Zotob reared its slimy head this morning. It exploits a defect in Windows systems (UPnP MS05-039) for which a patch has been available less than a week. Zotob is undoubtedly the first of what will be many Week Zero Worms exploiting this defect -- not quite a Zero Day worm, but close enough to wreck havoc. Every time this happens, the internet discussion forums are flooded with snide comments from smug systems administrators, along these paraphrased lines:
"I patched all 652 of my systems this week before the worm hit. Any organization being hit by this worm is incompetent."Well, probably not. These well-run one-man shops do impress with their ability to deploy patches quickly and offer some hope for the rest of the universe. However, the prima donna types that make it happen generally don't really understand the magnitude of the problem in a large corporation with, say, 50,000 TCP/IP devices, mostly running Windows. It's not just a matter of patching 77 times as many systems in that same week. The unstable tower of complex software architectures built up on top of the typical network of Windows systems in a large enterprise makes it quite a bit more difficult to plan and execute a system upgrade or a configuration change or even an operating system patch in a larger environment. Explaining this to management in large organizations isn't very hard. Getting them to agree to do something to fix the underlying problems, however, is almost impossible. The people in charge of keeping the engines running are not the same people in charge of all the complicated attachments that get connected to them. All of these arbitrary "business drivers" may be carefully considered by IT people, who conclude that they need to meet the needs of the "customer" (e.g. another business unit, which is often a profit center carrying clout with Senior Management) and concede to the complicated attachments. These other business units are often engaged, sometimes knowingly, in a game of externalized cost. They may buy a software system that must be deployed to every desktop, rather than one that users can access from a web server. Worse yet, they may build one, without divining the best practices which help prevent high-maintenance software architectures. An increasing burden builds up on the IT staff over time. Most of this stuff is extraordinarily difficult to measure. But these costs don't go away. They come back to bite. Other times support issues arise within the IT organization itself, and a clever solution is devised. Often entirely too clever. Unfortunately, this "can do" attitude of most IT shops is sometimes their undoing. Clever solutions interwoven through the layers of the distributed systems and the various creaky but mandated optional components combine to make an overall system architecture which is relatively brittle. Then a worm hits. In a panic, patches are applied, things break, and the mess is cleaned up later. A post-mortem is performed. In the post-crisis exhaustion, the IT organization struggles to put the pieces back together and move forward on the latest set of tasks from the latest set of business drivers. In the standard ongoing chaos, the recommendations are ignored. A few weeks later, another defect, another worm, another crisis which possibly could have been averted in a better world. It's a nasty vicious cycle, but it's definitely related to the sheer size of an organization and its network. So please, all you smug fully patched systems administrators, don't be so hard on your collegues who didn't get 50,000 PCs patched in the same week that you patched 700. This worm gave you several days to patch them, and it took you more than a day. The next worm could hit before the patch is available, and it could be you turning to the forums for advice on how to impede the spread of the worm on your network, contain the damage, and recover your systems. When your number comes up, these folks will have unfortunate experience that you might be able to draw upon.
Technorati Tags: zotob, adware, antivirus, botnets, hacker, malware, spyware, virus, Windows, worm
Comments