Contrarian Systems Administration

So, all of the sudden “automation” is popular again.

Luke Kanies posted a “blog”:http://www.oreillynet.com/sysadmin/blog/2007/02/ on O’Reilly and sparked a discussion on “lopsa-discuss”:http://lopsa.org/cgi-bin/mailman/listinfo/discuss. My first response got fairly long and I ran out of time for that mail message. So, I decided to move the rest to a blog post. In the intervening time, I’ve actually gone back and read Luke’s original blog and some responses there. I have some real disagreements with Luke’s premise, although some of his conclusions are correct. However, what I wanted to discuss here was a response by “matt”:http://www.oreillynet.com/sysadmin/blog/2007/02/why_isnt_system_administration.html#comments to Luke (wow, that sounds kind of biblical, doesn’t it). matt suggested that more system administrators weren’t creating automation tools because they were scared of automating themselves out of a job.

By coincidence, if you read my response on lopsa-discuss, I’d already talked about doing that, and it was already the point of the blog post I was planning. My mail was based around a situation where I had done just that… I had automated myself out of a job. The difference between my situation and what I think is the fear of other admins, is that I chose to leave the job because I automated myself into boredom. It wasn’t that I automated myself into redundancy and the company let me go. It was my choice.

It was my choice, because I thought I had discovered (learned?) a basic truism about system administration. I thought the truism was that system administration wasn’t a career, it was a short term activity that every company needed, and that the real career was in consulting. Which sort of sounds like the same conclusion Luke came to. I thought the idea that a company would want and need ongoing, fulltime system administrators was ludacrous. SAs should go in, set up environments, automate everything, clean up trash, and go away because they were no longer necessary (until entropy set in and the cycle restarted). Of course, at that point, I was only working with small companies and small educational departments.

So, with the truism in mind, I went into consulting. And set about breaking the first rule of consulting (and matt’s concern). I often told my clients I was there to work myself out of a job. I told them my goal was to hear “Your services are no longer required.” The great thing about that goal… you can always reach that goal. And, generally, you can decide the timeframe to reach it. The bad thing about that goal, is that you can get there in a negative way!

But, I was able to repeatedly, successfully achieve that goal. I helped one company stop sending me on three hour (each way) drives across the state to fix systems by installing terminal servers. I helped another company reduce the time it took to replace a workstation (and get the developer back to work) from several hours to mere minutes. I helped a company that lost a day of work (and lots of revenue) every two weeks (like clockwork) by stabilizing the server to the point of it not crashing for six months until we moved locations and replaced the box completely. And, somewhere in there, I started working with larger and larger companies and realized my truism wasn’t true. I learned that some big companies had big environments with lots of change and needed fulltime admins, even with the tools, standards, and automation.

Thanks to starting a family, the dot-com bust, and other things, my goal changed from working myself out of a job to finding an environment large enough for me to work out of one job and quickly transition to another. Somewhere large enough that would always have something new to do.

And, I’ve found there are lots of companies like there. There is always change and always a need for SAs, even if you try to automate everything. Once you stop focusing on fighting fires and start focusing on preventing them in the first palce, you can start looking at how to take an environment to even the next level. Because there’s always something new, always a new tool, a new technology, a new challenge. Sometimes, you even find that the challenges and the gratification is not just from technology, it might be just as gratifying to figure out how to shave another point off the price you pay for a server (especially if you purchase thousands of them a year). Or, it can be gratifying to teach a junior SA how to move his career along. Or, to work with someone from a business to redesign their manual business processes and start leveraging technology to make their business more profitable.

So, I did learn a truism way back there… I learned not to be afraid of automating myself out of a job, even as a fulltime employee. I learned to STRIVE FOR IT!