How to suck at programming – Announce a feature hot-deployed to a live server without testing


(Loosely based on a true story… as a tribute to a new WordPress blog that I find very amusing… The idea here is NOT original and I can never be that witty!)

You’re the master of your domain. The one person responsible for making sure the build/deploy system is operational. Also, you’re the only developer who actually knows what’s going on in that tangled mass of technology that nobody else in the company uses but you. Sure it’s different and complicated… but that’s only because the rest of your company co-workers are weenies and don’t have what it takes to sit through 532 pages of text describing the ins and outs of Agile Software through the build system. All they know is their fancy wizards for compile and deploy, but you… yes you, the REAL software engineer know everything that goes on underneath. You can describe each tool involved in the build lifecycle down to the byte code level. That NullPointerException, that wasn’t a programming flaw! That was the team’s resistance towards your efforts externalizing the dependencies! They always assumed that they could reference a member variable directly without calling through the getter method.

At any rate, you’ve become professional. You’ve done hot deploys in your sleep while dreaming of sugar plums and dancing daisies. You once did a hot deploy remotely over dialup with a keyboard that was missing the enter key and space bar! Your expertise is matched only by the sheer number of both cold and hot deploy’s you’ve performed over the years! Why should this one be any different? Besides, the code has been tested on your development server hasn’t it? Forget about that staging area specifically set up for Quality Assurance which mirrors the hardware and configuration of the production machines! Completely bypass policies and procedures checks, and balances set up to protect the company from failure. You know better your your methods are fool proof. Just one command and you’re off and running! deployment never felt so sweet.

You draft an email to the project manager as well as the product owner as you watch the text whiz by in your command window while the hot deploy carries on. “New feature rolled out!”, goes the subject line as you inform all of a successful update completely leaving out the fact that it is still in progress as you type. The deployment completes only seconds after you hit send in Outlook. Time for an early lunch your stomach craves reminding you that you have absolutely no work left on your plate between 11:30 and 12pm while the rest of the company marvels at the new features you’ve unleashed into production.

You high five your buddy that sits across from you bragging about how awesome you both are for totally beating the deadline. You crack a joke about some of the corners that were cut and how nobody will ever notice the easter egg you snuck in as an added bonus. Alls yuh hafta do is enter the Contra code during the splash screen in your Swing based app and a mini Super Mario jumps out. This was included as an insider’s joke on the other hermit developer who actually called out “Mario” in the Olive Garden restaurant the company dined at last year thinking he saw his long lost cousin from Italy. You are totally unaware that your hot deploy only went to one of the 25 production machines situated behind the load balancer that your deployment tool talked to and that you have no security to manually update the others. You are also unaware of the impact this will have to the Fortune500 client that accesses your software remotely… totally unaware of the financial loss that is soon to come from the next seven hours of down time spent diagnosing and finally fixing the problem.

The phone call comes in just as you grab your shades ready to head out in the summer light on your way to an 11:30 dine in at the local buffet. It’s your manager on a call with the director that just got chewed by the CEO. Clueless and unaware that the director is on the line, you answer with arrogance because you two have this understanding from way back when you managed to fix a problem by remotely rebooting twelve Linux machines while on vacation in Jamaica. However the era of you saving the day have long passed (it’s all about what have you done for me lately) and he sounds unamused by the joke you mumble over the phone to him while his boss listens in… a rather serious man who has already begun paper work to hire an assistant manager who will likely assume a direct management over your department. Your manager was unaware of your plans to run the update and only found out about the fiasco after receiving a barrage of phone calls and emails which over took his inbox like angry Persians fighting the Spartans in the final scenes of 300. It’s too late to claim ignorance of the event as you’ve already sent the email company wide. It’s times like this you wish you knew more about automated integration testing. So if you wanna suck at programming, hot deploy to a production box at the same time as you claim success. You’ll only spend the next nine weeks looking for employment before taking a gig at reduced pay but the lesson will last a lifetime!

3 thoughts on “How to suck at programming – Announce a feature hot-deployed to a live server without testing

  1. Yeah Matt, you’re right! I can only imagine what would happen in a company like ours if a feature crept its way into production half finished.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s