When I first started thinking about how to approach this topic, my mind instantly went to the technical side such as centralized patch management and staggered deployments etc. It would be very easy to present a checklist of do's and don'ts pertaining to updates and patching. However, when you think about it, the "non-technical" side is just as important.
Consider this statement made by Robert Conquest in his book called "Reflections on a Ravaged Century":
Patching and updating is an area that can cause a massive flurry of activity and chaos, especially when there is a dangerous unpatched exploit(s) waiting to take advantage of unsuspected users (seems like it’s an all too regular of an occurrence these days). The usual reaction is to protect the network and systems which usually equates to "we needed those systems patched yesterday!!!!" (This also doesn't help to foster a "one big happy family" atmosphere between the security team and all the SAs since the SAs are usually the ones having to drop what they are doing to apply "security's" patches.) However, as stated above, you have to be certain that your actions will not make a bad situation worse. I bet if I took a poll, everyone reading this has experienced systems that have reacted badly to a patch or an update. It may have even been something as simple as a bad virus update. Having a procedure in place that will facilitate a methodical process for patching AND change management (though not the direct topic of this conversation, it is very intertwined) will help keep things organized and in control when you really want to run from the room screaming like chicken little. You have to have a method to ensure that your attempts to prevent a disaster don't create one in the short term or long term.
I'd really like to hear from some folks who have implemented good processes for patching as well as change management related to those patches and updates. If anyone has implemented ITIL or perhaps some portion of it or maybe you have achieved your ISO20000 certification, please shared what you have found worked and didn't work when it came to a methodical method of patching and updating. For anyone wondering if the "non-technical" side really matters, please take a few minutes and read "The Visible OPS Handbook". Being someone who has kept her head buried in the technical side, it was an eye opening read for me.
Please don't get me wrong, I will be happy to include discussion on the technical side if folks would like to send it in. If you have found a great method or solution to central patching or maybe have tips for deploying multiple patches to large organization geographically spread out, etc, please send them in!! My main goal is to not just focus on the technical side of patching, but to emphasis the equal importance of having controls and processes in place before changes are made.
To quote Robert Conquest again:
Oct 13th 2007
1 decade ago