E-Procurement Measurement: It''s Not Broken - But It Needs to Be Fixed
There is a school of thought, and several recent articles have supported this, that suggests a fundamental overhaul of measurement activities is imperative in the new economy. Indeed, "E-business measurement as it exists today is broken,"1 was the conclusion of a recent study which argued that traditional measures such as ROI should be replaced, or at least extended, by more externally-oriented statistics (identified as end-customer success, hyper-partnering efficiency, and multi-unit financial performance across a company). Although we agree with many of the supporting arguments, we would not entirely support this particular statement in the specific context of e-procurement.
The advent of e-commerce and the consequent overhaul of internal procurement and supply chain mechanisms will require companies to re-evaluate their measurement criteria. Too many have been tempted to jump on the e-procurement bandwagon with little more than a hastily-scribbled business case to support them, and many early adopters now need to focus on ensuring that they do indeed realize the value they had anticipated. This can be done by tracking both the economic and the physical benefits. There are in fact many parallels with the early days of ERP implementation, with the focus primarily on getting systems in and only latterly on squeezing value out.
Doing the traditional basics of procurement measurement, while being prepared to make the occasional appropriate adjustment, will in fact take the fledgling e-procurement user a long way down the road toward ultimate success. The requirement is not for over-complication but for ensuring that the right things are measured for the right reasons.
Why Measure?
The ability to aggregate data in one central location and hence to provide almost limitless reporting opportunities on key criteria has been cited as one of the principal drivers for the adoption of e-procurement solutions. However, a number of early e-procurement adopters have recently been left bewildered by their failure to achieve (or at least to demonstrate that they have achieved) the benefits they anticipated. The explanation may lie in the failure of these companies to observe the basic principles of measuring business and implementation success. The simple truth is that the only way to realize these benefits is to measure how the business does those things that are fundamental - buying in the right way and at the right price.
While the fundamentals of the e-economy are essentially those of the traditional economy, the speed with which information diffuses through makes it vital that an organization measures the right things quickly. With internal and external customers empowered by the freedom with which they can access information, the company that is slow to measure its own performance risks learning of its failures only when its customers choose to leave. Conversely, in this new and more transparent economy with its increasing focus on performance comparisons, a company can only exploit comparative information if it has precise and timely information on its own performance. The need to measure is therefore paramount, but obtaining measurement data and transforming it into information may not be as difficult as some might suggest.
What to Measure?
"The firms that we see succeeding in e-business take conscious and deliberate steps to break away from traditional metrics and create measurements that address the unique interdependencies of the new business landscape."1 While the new environment does indeed impose new requirements, there is no need to break away entirely from the past. Boundaries are changing, most notably in terms of the speed at which companies can access information and forge external relationships, but the basic principles remain unchanged.
As with any major initiative, the success or failure of an e-procurement implementation depends on a few key factors; first to understand and define the goal of the project and secondly to have a clear strategy with measures or checkpoints incorporated along the way. Our experience has shown that the most successful projects are those where the e-procurement implementation is linked directly to the business process and needs - it is essential never to forget that an e-procurement implementation is a business and not a technology project.
The measures essential for a successful e-procurement implementation divide themselves into two clearly defined but closely interrelated categories of Key Performance Indicator (KPI). "Implementation Success KPIs" measure the penetration of e-buying (electronic identification of an item for purchase and subsequent generation of a purchase order) into the organization and identify the causes of poor uptake. "Benefits KPIs" provide the information needed to assess the extent to which anticipated benefits have been realized. While they can be measured separately, they must not be considered in isolation. To do so would be to risk incorrectly attributing a lack of success to the e-procurement concept itself, when the true cause may be a failure to measure inadequate levels of penetration and usage. A number of possible metrics are referenced in Tables 1 and 2.
Table 1 - Examples of Implementation Success KPIs
Table 2 - Examples of Business Benefit KPIs
Some of the KPIs, such as the percentage of spend going through e-procurement tools, will be directly measurable, while others, for example the proportion of time spent on purchasing as against strategic and tactical activities - a measure of the degree to which the purchasing team has been freed-up - are indirect. As ever, the temptation to focus on single metrics should be resisted. Simply measuring the number of catalog items available to a user could create an unrealistic picture of success or failure. A better solution might be to balance this with a measure that assesses the true value of the catalog to a user - for example, what percentage of the available items have actually been viewed or purchased using the e-procurement tool? Similarly, price will always be a key measure but other qualitative measures such as customer service or supplier flexibility must also be considered in parallel.
A focus on value, sometimes referred to as TCO (the Total Cost of Ownership), is as important as ever, and the aim should be to adopt a multi-tiered set of benchmarks in order to ensure that a balanced picture of success or failure is presented. In short, the old adage, "you get what you measure," is still valid. Decisions on precisely what to measure and how continue to require local ownership and acceptance if they are to engage buy-in both before, during, and after the implementation. Table 3 provides a few key principles. This selection is certainly not exhaustive.
Table 3 - KPI Implementation Guidelines
Current concerns have focused on e-procurement implementations that have failed to deliver anticipated benefits, but it is equally possible that casting the measurement net insufficiently wide could result in real but unexpected benefits going unnoticed. With increasing integration across the business environment, measurement needs to extend beyond pure purchasing and into other affected areas such as logistics, quality, manufacturing, R&D, and materials planning. Moreover the same principles can be applied to companies' increasing involvement, whether by investment or participation, in vertical and horizontal electronic marketplaces.
One area where some new measurement techniques are required is in the area of online bidding or e-auctioning. Nonetheless, while some new measures will need to be implemented - for example "percentage saved on traditional contracts by using an online auctioning tool," or "reduction of time taken to complete the tender process - some traditional measures will continue to apply. In the past, it has been usual to measure spend made with a company's tier-one suppliers; this need to rate suppliers remains a critical pre-activity for an e-auctioning program. Similarly companies will still have to measure ROI based on the projected and actual cost (including technical and process) of implementing an e-auctioning solution.
In general, just as e-procurement essentially supplements rather than replaces traditional procurement processes, so e-procurement metrics are not significantly different from those used in a traditional environment. The Supplier Use Index - the proportion of all suppliers who account for 90% of the total spend - is just as critical an indicator in e-procurement as in a traditional environment. The difference lies principally in the speed with which both benefits and mistakes make themselves felt, making it imperative for implementers to have their measurements in place much more quickly than on a traditional project. That in turn means defining not just what should be measured, but also the frequency and processes associated with recording measurements and communicating them (the latter done ideally through the organization's existing finance and procurement channels and systems).
While spending may have moved online, businesses will still find that the key measures they are interested in are compliance, administration (including process) and nature or spread of spend. To take an example, compliance was viewed by early adopters as a significant source of value, contributing as much as 30% of the total anticipated benefits. Unfortunately, the actual benefits realized were substantially less than had originally been expected. While the traditional measure for compliance may have been "percent of spend made against existing contracts," the same measure could be applied with just the extension "available within the e-procurement system."
Similarly, traditional techniques would also assist the effective measurement of process administration savings realized (anticipated by some early adopters as providing as much as 20% of total benefits). What has actually been seen is that predicted head count reductions have been largely compensated by increases in other areas such as catalog preparation or indeed the move towards a more strategic than tactical role for procurement personnel. Disappointment or discrepancies might have been avoided if better measures had been introduced more promptly. In this way implementers would have gained a better insight into variances from anticipated benefits as they developed, allowing practices and implementation procedures to be adapted accordingly.
How to Measure?
Ultimately, the success or failure of an e-procurement implementation can only be judged by measuring performance against the goals set at the outset. That requires the development of a balanced scorecard of KPIs - which is applied from the onset of the initiative. How are companies doing this and what tools are they using?
If the required measurements are to be assessed accurately without committing overwhelming personnel resources, information systems are needed to link the various supply chain partners. ERP systems have certainly helped but there is a pressing need to link them with new generation e-procurement systems. To date, supporting e-procurement technology has been comparatively immature but leading providers such as SAP and Oracle now recognize the need for applications that enable the user effectively to collate and analyze information from their e-procurement and other systems. These applications will come into their own as the ability to manage and mine the available data becomes more important.
At the same time many companies are extending their own internal tools and processes to incorporate e-procurement. One Accenture client, for example, has encouraged self-sufficiency among its people by developing an intranet site that helps site personnel manage their own implementation of e-procurement. The tool also asks sites to record development metrics and implementation progress in areas such as content development, and it is supported by a number of other tools designed to ensure collaboration across projects and between units. Another company uses an online database to track planned auctions from the "idea" stage to eventual results. Here, the ultimate plan is to link this automatically to the company's preferred auction provider to access information such as estimated savings and the status of planned auctions.
While intranet and Web-based technology is making it easier to internally disseminate purchasing information, external organizations are also providing tools to benchmark supplier capabilities. One such vendor offers a database of contractor capabilities on a website that includes benchmarking profiles of suppliers providing similar products and services.
Faced on the one hand with the evidence from early adopters of the consequences of failing to implement adequate e-procurement measures, and on the other with the suggestion that conventional measures are worthless in the new economy, companies embarking on e-procurement initiatives need to be reassured. Measurement has always been important and will become increasingly so. All of the traditional fundamentals of measurement and indeed many of the measures themselves can still be applied, they should be re-evaluated and fine-tuned but not necessarily discarded. Given a clear understanding of where the organization is coming from and where it is going to, a commitment to do the basic things well (following guidelines similar to those in Table 3), will go a long way not just to delivering the benefits identified in the original business case, but often to exceeding them.
References
1. "Measuring E-business Success," Forrester Research, Inc. December 2000
2. ''The Future of Purchasing and Supply: A Five- and 10-Year Forecast'' (Joint CAPS, NAPM, A.T. Kearney, Arizona State University and Michigan State University study)
3. ''New Measures Surface,'' By Roberta J. Duffy, Purchasing Today, November 2000
