Archive for the 'Information Technology' Category

My RHCE Exam Experience

On September 4th, 2009, I took the new 3.5 hour, single-section RHCE exam. This was my first time taking the exam, and to be perfectly honest, it was nowhere near as challenging as I thought it would be. While I don’t want to downplay the significance of the RHCE certification or give anyone the false impression that the exam is necessarily easy, I do want to emphasize that contrary to popular belief, it is quite possible to self-study your way to a perfect score. Not only was I able to breeze through the entire list of test objectives, I had time to thoroughly check and double-check all of my work with almost an hour to spare. This meant I was done long before anyone else in the room, including those who took the official Red Hat training courses. So how did I accomplish this feat?

The first thing you should know is that I had already been working professionally with Linux for ~10 years before I began studying for the exam. I doubt that so much experience is really necessary, but because the exam measures actual competency on live systems rather than your ability to memorize or “read between the lines,” I think it’s highly unlikely that someone with little or no working knowledge of Linux could successfully cram for this test. So I’d have to say that experience is an important prerequisite, but I’d also have to say that experience alone is almost certainly not enough. Considering the wide range of subject matter, it’s very likely that you’ll be tested on something you’ve never had to touch before.

In my opinion, the hardest thing about self-studying for the RHCE exam is knowing how much you need to know about each study point in the RHCE Prep Guide. In other words, when it says something like “you need to know basic configuration of x,” it’s hard to know exactly what “basic configuration” means. Although I’ve signed an NDA preventing me from revealing any details about the exam itself, I can tell you that nearly all of my study material came from two places:

  1. RHCE Red Hat Certified Engineer Linux Study Guide by Michael Jang
  2. Red Hat Deployment Guide

The rest of my study material came from search engines. I spent a lot of time comparing other people’s study notes to mine and reading anecdotes about the RHCE exam. While the result of this was usually a temporary blow to my confidence (there are a lot of horror stories out there!), I did find a few blogs and forum posts with some helpful information. Usually it was just an alternative way of doing something, but it was also nice to come across the occasional words of encouragement from a self-studied RHCE. Just knowing that other people had done it gave me a slight confidence boost (as I hope this post does for some of you).

The first thing I did was read Michael Jang’s book cover to cover, just trying to absorb what I considered to be the most important information from each chapter (i.e. concepts rather than commands). This took me a week or two. When I was done, I set up a test environment using VirtualBox. Then I decided it would help to have a condensed, single-page study guide to refer to, so I created a “wikified” version of the RHCE Prep Guide which I called my RHCE “cheat sheet”. For the next few weeks, I went through each study point on the “cheat sheet” and used my study material to test and document everything I thought I might need to know.

I found Michael Jang’s book to be a great study guide (the labs and example problems were a huge help), but I wasn’t completely happy with the amount of detail on some topics, as well as how it tends to follow the Red Hat course outlines rather than just sticking to the RHCE Prep Guide. I also found quite a few typos and just plain incorrect information, so that’s where the Red Hat Deployment Guide came in. While I was working on the RHCE “cheat sheet,” I would usually read the appropriate chapter(s) from Michael Jang’s book first, then I would supplement it with the appropriate chapter(s) from the Red Hat Deployment Guide. If I felt particularly weak in a certain area, I would also peruse the man pages and any HOWTOs I could find online. This was a long, arduous process, but it helped ensure that I wasn’t missing any important details.

For the last couple weeks, I tested myself by putting the “cheat sheet” away, doing a minimal install of CentOS in my test environment, and trying to configure everything I could without referring to any documentation whatsoever. Since the test takes place on a live system, I assumed from the beginning that the man pages would be available, but in order to save time, I wanted to make sure I could do everything off the top of my head. As a result, I ended up memorizing almost everything on the “cheat sheet,” which is probably why I was able to complete the exam so quickly.

Since I didn’t have a study partner, preparing for the troubleshooting section of the exam was a bit of a challenge. I did come across an interesting project called Trouble Maker which directly addresses this problem, but unfortunately, it has not been updated in several years and does not work on recent versions of CentOS (Update 2010-08-25: It appears this is no longer the case!). For a while, I actually considered writing my own trouble maker program, but I ultimately decided that this would be too much work. Luckily, I have a few friends who know enough about Linux to make a machine unbootable, so we made a game of it. I would give them my root password and challenge them to do something that would keep me from using my computer, then I would try to fix it as fast as I could.

When it was all said and done, I spent roughly six weeks (studying a few hours each day) to prepare for the RHCE. Considering how easy the exam was for me, I believe that I worked a lot harder than I needed to, but the results were clearly well worth the effort. The best advice I can give to prospective RHCEs is to take your time and practice until you can do everything in the RHCE Prep Guide off the top of your head. If you feel weak in anything, do yourself a favor and postpone the exam.

RHCE Exam Results

The results are in, and I got a perfect score!

Dear Michael T Conigliaro:
 
The results of your RHCE Certification Exam are reported below.  The
RHCE Certification Exam allows candidates to qualify for the
Red Hat Certified Engineer (RHCE) and Red Hat Certified Technician
(RHCT) certificates.  Please note that the RHCE designation is
understood to both include and supersede the RHCT designation.
 
RHCE requirements: score of 70 or higher on RHCT components (100 points)
                   score of 70 or higher on RHCE components (100 points)
 
RHCT requirement:  score of 70 or higher on RHCT components (100 points)
 
RHCT components score:                             100.0
RHCE components score:                             100.0
 
RHCE Certification:                                PASS
 
Congratulations -- you are now certified as a Red Hat Certified
Engineer!  Your RHCE Certificate number is 805009592042441.
The attached file is your personal print-ready certificate.
 
You are entitled to print this document and use it to demonstrate
that you are an RHCE, provided you remain an RHCE in good standing.
You may not modify or change the document's contents in any way, nor 
may you appropriate any elements of this document for use in other 
electronic documents or printed materials.  You may only print the 
document in its entirety.  Any other use of the document must be
approved by Red Hat, Inc.
 
Your RHCE number should be available for verification at Red Hat 
Certification Central:
 
http://www.redhat.com/training/certification/verify/?rhce_cert_display:certno=805009592042441&rhce_cert_display:verify_cb=Verify
 
You can verify the certificates of other RHCEs and RHCTs at 
 
https://www.redhat.com/training/certification/verify
 
Please visit RHCE Connection, our web site exclusively for RHCEs:
 
https://www.redhat.com/training/certification/
 
There you will find special offers from Red Hat, logo art, forums, job
listings, and more.  You can also use the site to manage your contact
information.  In order to access the site, you will need a PIN number.
You can have the PIN sent to the email address we have on file at
 
https://www.redhat.com/training/certification/lostpin.html
 
If you wish to connect to the forums directly:
 
https://certforums.redhat.com
 
Certification in Red Hat Enterprise Linux opens up new opportunities. 
We hope you will keep Red Hat updated with your experiences and successes
with Red Hat Enterprise Linux.
 
Please feel free to contact us with ideas and suggestions as to ways
we can enhance our Red Hat Enterprise Linux training and certification 
programs at
 
https://www.redhat.com/training/certification/comments.html
 
Thank you very much for your interest in Red Hat Enterprise Linux!
 
Red Hat Certification Central

So I’ve Decided to Take The RHCE Exam

According to the Internets, the RHCE is the “crown jewel of Linux certifications,” and since I don’t have any Linux certifications at the moment, I’ve decided to give it a shot. After taking the Pre-assessment Questionnaires about a month ago and seeing that I was in pretty good shape already, I decided to self-study using Michael Jang’s book and the Red Hat Deployment Guide rather than taking thousands of dollars worth of Red Hat classes. The more perceptive among you may have noticed my new wiki containing my RHCE “Cheat Sheet”. Hopefully someone else out there will find it useful.

Even though I can do about 95% of the stuff on the RHCE Prep Guide off the top of my head, the horror stories I’ve been reading about this test are making me pretty nervous. Any tips from current RHCEs would be appreciated. Also, if anyone is interested in logging into my test machine at home and doing something to make it unbootable, that would help me practice for the troubleshooting part of the exam.

I’m scheduled to take the test in Philadelphia on September 4th, which coincidentally, is my birthday. I guess I won’t know if that’s a good thing or a bad thing until I get the test results.

Getting Started in Information Technology is Easy

I have always insisted that getting started in IT is relatively easy compared to most other professions. I just can’t think of any other industry in which it’s possible to acquire so much practical working knowledge on your own with so little money, and it makes me wonder why more people (especially those who complain about being “stuck” in low-level tech support jobs) don’t take advantage of it. It’s my belief (and personal experience) that with a bit of time and motivation, anyone can gain practical, real-world experience (yes, the kind you put on a resume!) in their spare time, and have a lot of fun while they’re at it. This article is not really a step-by-step guide for getting a job in IT, but more of a list of simple things you can do in your spare time that will help you teach yourself the kinds of things you should know.

First, a few tips:

  • Clearly, it’s in your best interest to learn as much as you can about as many different technologies as you can get your hands on. However, don’t ever expect to know everything about anything. Definitely don’t waste your time memorizing things like command line options and function call parameters. That’s what reference manuals and Google are for. It’s infinitely more valuable to have a good understanding of concepts.
  • This is a bit of a continuation of the previous point, but you need to realize that what you know off the top of your head is a lot less important than knowing where to find the answers. Read the manual, learn how to use  Google and IRC, and as frustrating as it may be, only ask for help as a last resort after you’ve done your own research and tried everything else. This will teach you how to teach yourself.
  • Get in the habit of asking yourself “how would I do this if I had to do it 10000 times?” Computers are very good at performing repetitive tasks very quickly, but human beings are slow and error-prone. The sooner you learn how to leverage the computer to do your work for you, the better off you’ll be. Be lazy by eliminating as much manual work as possible. Your mantra should be “how can I automate this?”
  • Never settle for “good enough.” Be a perfectionist, and strive to do everything the correct way according to best practices and/or accepted conventions. If you find that you’re doing something wildly different than everyone else, you’re probably doing it wrong, and in the real world, it will almost certainly come back to bite you or your successor(s) in the ass later. You might also end up reading about yourself on a site like The Daily WTF.
  • Remember that the best solution is always the simplest and most elegant one. Avoid creating Rube Goldberg machines. Remember this quote by Antoine de Saint-Exupery: “A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away.”

And now the fun stuff:

  1. We’re going to start by building a network. In the unlikely event that you haven’t done so already, get yourself a broadband Internet connection and a cheap Linksys firewall/router. Make sure your ISP allows you to host a server (I strongly recommend DSL over cable for this reason alone). Contrary to popular belief, a static IP is not necessary, nor is lots of bandwidth.
  2. Build your own server from scratch. It doesn’t have to be anything really expensive or powerful, but it should be dedicated (i.e. a different physical machine from your primary desktop) and capable of running 24/7 for the foreseeable future. Don’t waste your money on an expensive video card. Use that money to buy a cheap interruptible power supply and/or a tape drive instead.
  3. Install Linux on your server. I don’t care what distribution you use, but you’ll find a lot more documentation for the more popular ones (I use Ubuntu server). Do the most minimal install for your distribution of choice (i.e. no optional packages). Do not install a GUI (or control panel) on your server under any circumstances.
  4. Once your server is up and running, you are not allowed to reboot it unless it’s to swap out hardware or to install a new kernel. Your goal should be to have the longest uptime possible (for reference, my home server’s record is somewhere around 450 days). When something breaks, figure out what went wrong and fix it. Don’t just reboot and hope the problem will go away, because that will be a waste of an opportunity to learn something.
  5. Without a GUI, you’ll need to do everything on the command line. Get familiar with Linux’s filesystem hierarchy. Learn about what files go where and why.
  6. Register a domain name, but do not purchase off-site email/web hosting. From now on, you’re not going to pay anyone else for anything you’re capable of hosting yourself.
  7. Make your domain resolve to your home Internet connection. If you don’t have a static IP address, get an account with a dynamic DNS provider and learn how to make this work.
  8. Install Apache and start a blog. You’ll use it to document things as you learn them. Learn how to open the necessary ports in your firewall to make your website available to the whole world. Does this scare you a bit? It should. Your server is now wide open to attack. Start learning how to secure your server. Be paranoid.
  9. Install SMTP, POP3 and IMAP servers, then tell everyone about your new email address. If you have other outside email accounts, forward them all to your new address and use it exclusively. Set up server-side spam filtering and learn how to defend yourself from spammers.
  10. If you haven’t already, get a cheap tape drive and learn how to back up and restore your files. Then learn how to take automated nightly backups of your critical data.
  11. Now that you’re your own webhost, give your friends accounts on your server and host their websites/email too (the more the better). Learn how to set quotas and secure your server on the inside so that your users can’t break it on you. Then test yourself by challenging them to break it on you.
  12. Turn off the crappy DHCP server on your router and install something more powerful (e.g. ISC-DHCP).
  13. Install BIND and use it in place of your ISP’s DNS servers. Learn how to edit zone files by configuring an internal DNS namespace and putting every device on your network into DNS. Don’t forget to create a reverse lookup zone. Learn how to use the dig command.
  14. Combine your knowledge of DHCP and DNS to set up dynamic DNS updates.
  15. Now that you know your way around a Linux box, back up your home directories and set everything back up again on a fresh install of Gentoo Linux.
  16. Learn a popular web-programing language (e.g. PHP, Python, Ruby) and build your own website/blog from scratch. For bonus points, start by creating your own MVC framework.

That aught to keep you busy for a few years! And once you’re done, you’ll have more working knowledge than 99% of the people out there applying for entry-level IT jobs.

A Scientific Method for Troubleshooting

I think there’s a new heavyweight champion in the world of Mike’s IT-related pet peeves. It’s called “just trying a bunch of random things until my problem goes away.” This is in no way related to troubleshooting, which I will define as “uncovering the root cause of an issue and then resolving it deliberately.” While there are many different techniques for troubleshooting specific problems, I’m going to attempt to show how the scientific method can provide a common framework for more effective troubleshooting.

Step 1: Describe the problem

This is the easy part. Find out what the problem is and reproduce it. That 2nd part is important, because the typical user can’t always be trusted to know what they’re doing. By reproducing the problem, you can rule out user error and verify that there actually is a problem.

Step 2: Gather and analyze data

This is the part everyone likes to skip, and is the real subject of this rant. Step two requires direct observations in order to find out exactly what is happening. How you do this is very much dependent on the problem at hand, but it involves things like log and packet analysis (which may very well require a 3rd party tool not included within the base OS). In any case, please don’t just take a wild guess about what’s causing the problem and then proceed down the path of random experimentation. I don’t know for sure how people acquire this bad habit, but I have a strong suspicion that it comes from working in Microsoft Land, where the computers have personalities, rebooting is the inexplicable fix for everything, and a sea of GUIs makes it easy for the novice sysadmin to miss what’s really going on.

Part of problem in Microsoft Land is that proper troubleshooting tends to be a lot more difficult than it needs to be. For example, there is a severe lack of useful diagnostic tools included within the Windows OS itself. Why are the Windows support tools, resource kit tools, and IIS diagnostic tools still separate downloads? The same question can be asked about the Sysinternals Suite (which Microsoft has owned for several years now). Why are they still shipping obsolete utilities instead of their newer replacements (e.g. nslookup, which was obsoleted by dig many, many years ago)? And lastly, why does Microsoft constantly try to hide any information that could be useful for troubleshooting? Anyone who has ever had to view the message headers on an email in Outlook knows exactly what I’m talking about here, but I digress.

Step 3: Form a hypothesis

It isn’t until you figure out what‘s happening that you can address the question of why it’s happening. Step three is where you use the information you gathered in step two to determine a logical course of action. Remember that a hypothesis beginning with “maybe” or “I think” with little or no direct evidence to back it up is often a dead giveaway for someone who doesn’t know what the hell they’re talking about.

Step 4: Test your hypothesis

Perform your planned course of action.

Step 5: Analyze results and draw conclusions

Check to see if the issue is resolved. If not, revert your changes and go back to step three. When drawing conclusions, ask yourself how this problem occurred in the first place. Was your most recent fix permanent or just a temporary band-aid? If the fix was temporary, make sure you schedule a time to implement a permanent fix.