The Measurement of IT

A Metrics Toolkit for EffectiveManagement

Richard Swanson, professor of human resources at theUniversity of Minnesota, made a keen observation: Everythingin business is evaluated. Expanding on that notion, we wouldalso observe that what we can't measure, we can't improveupon, and improving all aspects of business operations isnot only critical to success, but also critical tosurvival. The recent pronouncement by the Federal ReserveBank Board of Governors that the decline of the stock marketand disclosures of corporate wrongdoing had prolonged asharp deceleration and that the economy is not going toimprove significantly anytime soon means thelong-anticipated turnaround is no longer right around thecorner. In fact, 2003 is shaping up to be a lot like 2002,and we expect that corporate budgets will remain extremelyconservative and that improving profitability in the face offlat or even declining revenue will continue to be a primarycorporate objective. As a result, measuring and improvingbusiness operations will be critical to survival.

From a CIO's perspective, this means another year of tightbudgets and a continuing imperative for IT to contributevalue by implementing technology-based solutions that reducecosts and create opportunities for new products andservices. However, given IT's track record, this is mucheasier said than done. We would recommend that CIOs focus ontwo objectives: The first is improving the efficiency andeffectiveness of IT, that is, getting more things donefaster and better with the same or fewer resources. Thesecond is demonstrating the value of IT to theorganization. The key to both of these is measurement. Inthe first case we want to measure our performance so that wehave a benchmark to improve against. In the second case wewant to use measures that connote value whether that valueis expressed in reducing the cost of doing business or innew revenue streams that are the direct result of aninvestment in information technology.

Having said this, what are the right measurements? The Gigaresearch organization recently collaborated in an effort toassist our clients in building an effective measurementtoolkit for IT. To make it as practical as possible, weorganized the metrics along job functions. Analysts wereasked to put themselves in the shoes of a client and ask thequestion, "What metrics would I consider to be key tosuccessfully managing or demonstrating the value of myfunction?" During this exercise we identified two majortypes of metrics in keeping with the two objectives - improving efficiency and demonstrating value. The firstmetrics are operational metrics. Focusing on these metricsenables you to improve the performance, productivity andefficiency of the specific IT function. Operational metricsinclude such things as cycle times, defect counts and outputrates. The second metrics are value metrics. Focusing onthese metrics enables you to demonstrate the function'svalue to the overall business. Value metrics tend to be costand/or revenue based. Examples would include cost perbusiness transaction, cost of IT per end user, revenue andprofitability of a new IT-enabled product or service, etc.

Metrics by themselves provide little value - it's how themetrics are implemented, reported and acted on thatdifferentiates successful measurement programs fromfailures. Research has shown that despite the bestintentions, four out of five measurement programs failwithin one year of their inception. A good metrics programwill have the following characteristics: