Europe''s New Retail Market by Chris Trayhorn, Publisher of mThink Blue Book, March 11, 2004 The European Union plans to accelerate the pace of electricity and gas market liberalization and has introduced dates for the phased introduction of full retail competition (FRC) in both these markets. Some member states have already implemented FRC arrangements that meet the requirements of the revised directives. Many, however, have not. For the transmission and distribution systems operators (TSOs and DSOs) of these member states, the dates for market opening are looming large. Experience around the world shows that implementation of arrangements for FRC is a complex undertaking. Successful implementation can take years to complete. Those states that have implemented partial competition, and even those that have full competition but where there has been little interest in the mass market by external retailers, face a significant task in scaling up their systems and processes to meet the transaction volumes seen in fully competitive markets. Based on IBM’s involvement in the successful implementation of FRC in many jurisdictions, we’ll address: Challenges facing TSOs and DSOs around Europe; Models for FRC already implemented around the world; and Key factors that ensure a successful implementation for regulators, TSOs, and DSOs, including critical market design concepts, implementation approaches, and systems and technology options for retail markets. European Challenges The progress of implementing retail competition across Europe to date has been mixed. Some markets are fully opened and have experienced fierce competition, e.g., the UK electricity and gas markets and the Nordic electricity market. There are some markets that are still only partially open, such as France, and some that are theoretically fully open, but where fierce competition between retailers has yet to take hold for all customer classes. For these countries, current processes and systems may only have to cope with registering the switching of a few thousand customers. Over the next two to three years, we’ll see a significant change with far-reaching implications for those charged with implementing new market arrangements. There are four key factors that will lead to a very active mass retail market: Nondiscriminatory, cost-reflective arrangements for access by third parties to all network voltage or pressure levels; Competitive, transparent, and liquid markets in wholesale energy (either national markets, or regional markets with non-discriminatory arrangements for international interconnector or pipeline access); Clear rules and processes for changing retailers that minimize the transaction costs involved; and Sufficient scale (again, either nationally or regionally) to encourage entry by global mass market retail players. New EU directives require the designation of a regulator charged with approving the commercial terms for third-party network access and for the provision of balancing energy or flexibility services, legal and financial separation of all distribution and transmission network operations, and phased moves toward full market opening by 2007. In addition, the new regulation in the electricity market, and its expected equivalent in gas, will start to address the issue of charging for international interconnectors and the allocation of capacity on those interconnectors. This will increase the extent to which Europe will be regarded as a single market in electricity and should encourage players to think of regional rather than national wholesale and retail markets. This may encourage retail players to participate in markets that would, otherwise, have been below an efficient scale. Finally, given the legal mandate for full retail competition, regulatory institutions are likely to start considering the rules and processes put in place to allow customers to change retailer, and the extent to which they provide a framework for an active market. TSOs and DSOs in either partially open markets, or in markets that are as yet only theoretically open, will need to consider their response to questions that are likely to arise, such as: Can existing processes and systems ensure that delivered energy is appropriately accounted for and that customers can still be billed with churn rates of 10 to 20 percent? Given the requirement for a two-stage market opening (2004 for nondomestic and 2007 for full market opening), can the 2004 arrangements be future-proofed or will there need to be two separate programs of work? What are the potential costs of creating a fully liberalized market, and how will these be recovered? How will a significant internal and cross-industry work program be managed? Which market design options will be most appropriate? International Models The two broad models that have been implemented around the world – distribution-centric retail settlement, and direct wholesale market settlement – differ in the way retail participants interact with the wholesale market. Distribution-Centric Retail Settlement The distribution-centric retail settlement model shown in Figure 1 has been implemented in a number of electricity markets in North America. This model is based on a mandatory wholesale market (as is typical in the North American electricity sector). In this model, retailers receive their bills for wholesale energy from the distribution system operator rather than directly interfacing with the wholesale market operator. Retailers (and the DSO to the extent they have a retail business) strike financial hedges with producers to reduce exposure to volatile wholesale energy prices. Retailers can opt to bill their end customers directly (direct retailers), or have the distribution system operator bill the customers for energy on their behalf (indirect retailers). DSOs will also charge retailers for network access. In terms of payments to the DSO, direct retailers simply pay the wholesale price on their estimated consumptions. Indirect retailers receive the difference between their retail contract price and the wholesale energy price. Because retailers don’t interface and settle directly with the wholesale energy market, their consumption doesn’t have to be estimated immediately for the purposes of settling that day’s consumption. The wholesale market can be settled between producers and distributors, and then retailers can settle with distributors through a separate process. For example, their consumption could be settled on a monthly basis. Depending on typical meter reading frequency, this can help to remove the need to continually refine estimates of consumption as more recent meter reads are collected. The model is based around a single, mandatory market. It is this that makes it possible for the distributor to bill retailers for the energy consumption of their registered customers, as it provides a single independently determined market price. Hence it is most applicable to electricity markets. However, in Europe (in contrast to the US), even electricity markets are typically bilateral in nature, and do not involve centralized mandatory markets. Within such a context, it would be difficult for distributors to bill retailers for the energy consumption of their registered customers. Direct Wholesale Market Settlement Figure 2 shows the broad market mechanics for direct wholesale market settlement. This model has been implemented in a numberof European markets, including the UK electricity and gas markets, the Nordic electricity market, and in a number of states in Australia. It is being implemented in the Irish gas and electricity markets. This model is shown in the context of a bilateral contract market for wholesale energy, but could equally well be implemented alongside a mandatory market. In this model, retailers interface directly with the arrangements for imbalance settlement, either by themselves or as part of a balancing wholesale circle. Customer consumption needs are estimated under the same timescales as those used for the imbalance settlement process. If the imbalance settlement arrangements are based on a rolling settlement process a given number of days after delivery, customer consumption needs to be estimated daily and well within the defined timescale. Reconciliation of imbalance amounts among retailers will be required as actual meter reads become available. Depending upon the approach to profiling noninterval metered data and typical meter reading frequencies, this reconciliation process may run for over a year after the settlement day. All retailers bill customers directly, rather than having the option of allowing the DSO to carry out billing on their behalf. Key Factors for Success Implementing FRC is no small undertaking. The total cost across all participants of implementing FRC in the UK has been estimated at €1.5 billion. This cost is attributed to complex market arrangements and a high degree of systems development to implement the arrangements. Today a number of package solutions exist to meet the requirements of FRC. Coupled with advances in the use of Internet technology for market communication, this has reduced the cost of implementing a competitive retail market. However, even where simpler arrangements have been implemented using configured off-the-shelf solutions, costs have been significant. For example, central costs of around €20 million in some Australian states, and company-specific costs in Canada and Australia of around €20 million. The cost and complexity of these programs remains high, and FRC implementations around the world continue to be characterized by delays and cost overruns. The risks presented by these programs can be minimized through careful consideration of the following four factors. Market Design Features The failure to make appropriate decisions on critical market design features upfront in an FRC program often comes back to haunt the program during its implementation. The procurement and configuration of systems will be dependent upon these design features, as will be the interactions required between participants to effect market transactions. An ill-defined design is likely to change during the life of the program, resulting in increased costs to change the design being implemented by some or all participants involved. Even if the design does not change, there is still the risk of misinterpretation between participants that will require change on behalf of one or all parties. Therefore it is of critical importance to the success of the program that a number of design features are defined early and with sufficient authority. This will ensure that systems design decisions can be made with confidence. Approach to Program Management Given the size, complexity, and number of participants and stakeholders involved in implementing FRC, strong and effective central program management is critical to the success of the implementation. The key features of such program management are: Appropriate governance arrangements; Industry-wide clarity on roles and responsibilities; Resource planning; Design control; Systems and processes trialing; and Stakeholder management and communication. Systems Selection For those markets that have successfully implemented FRC to date, systems costs have been a significant part of overall expense. Fully automated systems solutions have replaced the simpler systems and manual processes that were used to support initial market opening. However, as international experience of FRC implementations has grown, so have the range of package system solutions that are available. It is now possible to avoid significant build and implement FRC using off-the-shelf solutions (using either single packages with a wide range of functionality or best-of-breed integrated solutions).This has dramatically reduced the costs of implementing FRC, while also improving the quality of the solution. The UK recently began reviewing the systems and processes that support FRC in gas and electricity in the belief that they were too expensive to operate and produced too many exceptions. The nature of the systems solution will depend upon the overall market design chosen, and on the approach taken to separation of network and competitive activities (and hence the use of legacy applications). However, in general, the implementation of FRC will have implications for systems performing the following functions: Customer registration and transfer; Meter data handling; Retailer settlement and billing; and Communications hub. In defining a systems solution for FRC, the capabilities and constraints of the existing legacy systems form a key consideration. While many of the package solutions will claim to meet all FRC requirements fully, the final solution architecture will typically include a number of legacy systems in one form or other. A number of vendors provide solution components for FRC. They include: Excelergy Corporation, ICF Consulting, ITRON, Lodestar Corporation, SAP, and SPL WorldGroup. While some of the packages can claim to meet the full scope of FRC requirements, the specifics of a particular market, key design aspects and the use of legacy systems (and associated constraints) will typically drive the solution chosen. In this context, compatibility and ease of integration need to be considered when selecting components from one or several package solutions. As Figure 3 shows, the operation of the UK arrangements is split over four roles: The Registration Service, which is at the center of the change-of-retailer process, is operated by the DSOs and maintains a range of details for each eligible metering point, including the retailer, the data collector and data aggregator, and the meter registers from which data is to be collected. The Data Collector, who is an agent of the retailer, collects data from interval metered and noninterval metered metering points. For non-interval metering the data collector provides an estimated annual consumption where a reading is not available, or an annualized meter advance where a meter reading has been made. The Data Aggregator calculates aggregated cumulative advance meter readings (or estimated annual consumptions where no meter advance has been collected) by each profile class. A Central Agent, who: Calculates and applies profiles to the aggregated data for each retailer; Receives data on total transmission system off-takes for each distribution area and adjusts profiled estimates to ensure that the aggregated estimates are consistent with this top-down figure; Aggregates retailers profiled consumptions nationally; Carries out initial wholesale market imbalance settlement using this data as an input; and Carries out subsequent reconciliations among retailers to update imbalance settlement calculations as improved estimates of retailer consumption (based on actual meter reads) become available. There is a private data network to manage the flow of information between all the relevant parties involved in the end-to-end processes of calculating each retailer’s total demand. While the wholesale settlement arrangements are, to a great extent, unaffected by the arrangements for retail competition, it is worth noting that final settlement for retailers does not occur until around 14 months after the energy has been delivered. This elapsed time allows a greater proportion of retailer consumption to be based on profiles applied to actual non-interval meter readings, rather than on the estimates of the readings used for initial settlement. The arrangements put in place in the UK electricity market to facilitate retail competition probably lie at the extreme end of the spectrum of complexity. For example: Each DSO operates their own registration database, and this is typically implemented as a physically separate system to the databases that the DSOs’ retail businesses use for customer care and billing purposes. The roles of interval meter operator and data collector were specified separately and open to competition from the start of operation of the new arrangements, with their non-half hourly counterparts opened to competition shortly thereafter. There are eight separate regression profiles, depending on customer class. Similarly, while the UK gas market arrangements were simpler initially (with the majority of the process being managed centrally within Transco) progressive regulatory moves to increase competition have added to complexity. Equally, the unbundling of the distribution networks within Transco may complicate the arrangements further. Finally, the technology underpinning the UK arrangements reflects the fact that they were designed and implemented in 1998-1989. For example, nowadays we would expect Internet technology and communication hub applications to be used in place of the UK’s centrally managed private data network. This sort of technology has been implemented in a number of North American markets and has recently gone live in the Belgian electricity and gas markets. A similar technology solution is being considered for the Irish gas market. Implementation and Testing In addition to the definition of a solution architecture, there are a number of other key factors in the detailed implementation program itself, which, in IBM’s experience, are key to success. Two such important factors are legacy data migration and cleansing, and market testing. Legacy Data Migration and Cleansing Existing meter and customer data may need to be migrated from the DSO’s legacy customer information system (CIS) to the new systems which support FRC. Alternatively, if the legacy CIS is to support FRC, the structure of the data may need to be modified. An appropriate data structure is central to the implementation of FRC. Retailers should be associated with metering points, and metering points with customers A retailer may serve many metering points, and an individual customer may be associated with many metering points. However, this basis of customer registration may not be consistent with the way in which data is stored in legacy systems. Equally, the data in legacy systems may not be of the quality required to ensure accurate accounting for energy delivered post-FRC. Many retail participants in mature markets continue to have problems billing customers directly as a result of data quality issues that were not resolved prior to the market going live. Market Testing Sufficient time should be allowed in the overall implementation program for individual system testing (the usual factory, site and user acceptance testing processes) and also for end-to-end market testing with market participants. Given the nature of FRC implementation, it is particularly important to ensure that all existing participants in the market prior to FRC are able to continue their operations after go-live. Failure to ensure this could, in the extreme, result in a failure to account appropriately for energy that has been delivered and the risk of utilities’ statutory accounts being qualified. Robust market testing is the key to mitigating this risk. This allows those participants already operating in the market to test their systems and business processes against the central systems in order to ensure that end-to-end market processes operate as intended, and that central and participant systems can communicate with each other. The most appropriate approach to market testing will need to be considered against the specific situation of each country, and the way in which eligible customers supplied by third-party retailers will be treated under post-FRC arrangements. For example, if existing eligible customers will continue to be managed under existing systems and processes and only migrated on new FRC systems at a later date (for example, when they next switch), then testing of the new systems and processes may have to involve fewer parties. Filed under: White Papers Tagged under: Utilities About the Author Chris Trayhorn, Publisher of mThink Blue Book Chris Trayhorn is the Chairman of the Performance Marketing Industry Blue Ribbon Panel and the CEO of mThink.com, a leading online and content marketing agency. He has founded four successful marketing companies in London and San Francisco in the last 15 years, and is currently the founder and publisher of Revenue+Performance magazine, the magazine of the performance marketing industry since 2002.