How do you measure an IT department?

As part of a consulting engagement my client wanted to know how well their IT department compared to similar IT departments in the same industry. I began by collecting data comparing the customer's IT department to peer organizations in the same industry. I knew before I started that there may be some apples to oranges comparisons. What surprised me was that I was comparing apples to flying saucers. I discovered the real issue is determining how to define an IT department before making any quality judgements.

In this particular study I looked at organizations within the same somewhat regulated industry. Organizations with similar revenue and customer volume had IT departments that varied in size by over 200%. So what was really going on here? First, it was clear to me that everybody has a different names for the same thing. Second, there are no obvious rules about boundaries. Let's look at both dimensions sequentially.

Names can be deceptive. For example, within IT there is often an infrastructure group. Assuming that includes server support, does it also include network support? If that includes network support, does it also include the phone system? If it includes the phone system, it probably includes the automated call response software and support staff. If you include these staff, do you include the telephone operators? If the operators are included, then do you include the call centre and business help line staff? Now we have an organization that reaches directly from the external customer to the back-end server.

I know this example may be extreme. But where do you draw the line? Let's think about boundaries. Does IT end at the pure technology of servers and network gear? Or does it go further? Think about applications development. IT usually includes the programmers, but not always. What about the analysts that work with both the business and the programmers to develop the requirements? Sometimes they are called business analysts, sometimes systems analysts. They do the same work, but the systems analysts typically reside in IT and the business analysts do not.

Defining IT by roles performed simply doesn't do the trick. As I reviewed the organizations in the study, there seemed to be no consistency. One IT department ran the printing operations for the whole organization. Another included web content development. Contrary to intuition, you cannot define an IT organization by the functions it runs.These seemingly arbitrary distinctions depended on history, available skills, and ultimately, political machinations.

Names are nearly meaningless and boundaries fluctuate continually making comparisons challenging. But the rules creating IT boundaries are remarkably consistent across all organizations. The ability for IT to influence these changes is negotiating power.  The degree of negotiating power accumulated by IT defines its boundaries. Building negotiating power comes from cumulative value-building. Every transaction between IT and a customer creates or destroys the clients' perception of IT's value. If IT provides great service, clients are more willing to work with IT. As a result, they become more willing to support IT in political bargaining when the organizational boundaries are inevitably revised over time.

Back to the original question - how do you measure an IT department? I would propose that you define the worth of your IT department by its return on service (ROS). If clients feel they get great service, the long term reward to IT is greater scope - a higher ROS. Conversely, bad service creates negative ROS and a shrinking IT department. A good IT department is one that is growing because it is providing an optimal return on service to clients. The last full measure of an IT department is its return on service.

~