Wednesday, February 2, 2011

The Impact of Change Management on Technology Operations

For the last 18 months or so I have been researching and advocating for the need of having a better control of change in our environment. I strongly believe that Change Management is not “a thing for developers and project managers”.  It is widely known that many times we cause our own when we upgrade a system, or make a small change to a configuration, that triggers, sometimes not immediately, a sequence of events that can have a negative impact in our network operations and services that we provide. We have implemented a series of tools that have started to have positive impact in our Technology Operations.

Our Tools
The first tool that we introduced was DeviceExpert from ManageEngine, which added serve as our Backup tool for Network Gear and had a Change Management module that soon became popular. Because of this software, I started to understand how Change Management could help us improve troubleshooting, and operations of our network, and I started “evangelizing” about the need for a tool that took it beyond that area and covered as many systems and processes as possible

About six months our Technology Director and I started discussions around how it could help us and we soon added the rest of our Management team and we were all on board. Armed with all the input that we gave him, our Director designed and developed a Change Module that was added to our Resource Management and Systems Catalog portal which we named Techsystems. We use this module to log changes that we make to the systems and processes that we support. Granted, it is a manual process today, but it is already helping us resolve operation issues and it has great potential.

So How Does Change Management Impacts Operations?
In my opinion, one must have feature of a tool like this is the ability to create and schedule reports, and actually look at them. Both of our systems deliver reports that we review individually and then as a team during our weekly meeting. The result? Better picture of what’s happening and possible impact of changes made that were otherwise previously unknown.

I’ll now give you a couple of practical examples of how it helped our team overcome a couple of issues. One day we started receiving blank faxes from our Fax Server, which usually means that there is a communication issue somewhere.  I knew it was a missing setting in either our Cisco Unified Communications Manager or one of our routers that server a gateway to the server, but I couldn’t remember where it was. We spent the next couple of hours going over different settings in the Communications Manager and some of the routers until I finally remember that there was DeviceExpert and shouted it out! We immediately turned to it and ran a Change Management report, which showed the setting that was missing in one of the Routers. Issue resolved. Our problem was not looking at this report earlier because we had never used it. Since then, I have made this my first step to troubleshooting where it fits and I encourage our engineers to do the same. That’s where the value of the tool is. It’s just not a log that you never look at; it is a tool that can help that should be used.

Last week our Financial Systems Administrator found out that our calls had not been properly billed since January 11th. Not good in a Law Firm. He spent a couple of weeks trying to figure out what the problem was because other members of the team had been busy traveling or in projects. This past Monday we finally had time to talk about it and my first questions were to describe the situation and tell me when this started happening. With that information we went straight to Change Management and looked up the log to see what had happened around that time, and sure enough, there were some changes made in the server that host this process. Then, we were able to trace the problem back and revert to original configuration and in about 20 minutes the process was working properly.

If you change it, log it. If you log it,  look at it and share it. What are you doing that is helping you with Change Management?

No comments:

Post a Comment