The European Union plans to accelerate the pace of electricity and gas market
liberalization and has introduced dates for the phased introduction of full
retail competition (FRC) in both these markets. Some member states have already
implemented FRC arrangements that meet the requirements of the revised directives.
Many, however, have not. For the transmission and distribution systems operators
(TSOs and DSOs) of these member states, the dates for market opening are looming
Experience around the world shows that implementation of arrangements for FRC
is a complex undertaking. Successful implementation can take years to complete.
Those states that have implemented partial competition, and even those that
have full competition but where there has been little interest in the mass
market by external retailers, face a significant task in scaling up their systems
and processes to meet the transaction volumes seen in fully competitive markets.
Based on IBM’s involvement in the successful implementation of FRC in
many jurisdictions, we’ll address:
- Challenges facing TSOs and DSOs around Europe;
- Models for FRC already implemented
around the world; and
- Key factors that ensure a successful implementation for regulators, TSOs,
and DSOs, including critical market design concepts, implementation approaches,
and systems and technology options for retail markets.
The progress of implementing retail competition across Europe to date has been
mixed. Some markets are fully opened and have experienced fierce competition,
e.g., the UK electricity and gas markets and the Nordic electricity market.
There are some markets that are still only partially open, such as France,
and some that are theoretically fully open, but where fierce competition between
retailers has yet to take hold for all customer classes. For these countries,
current processes and systems may only have to cope with registering the switching
of a few thousand customers. Over the next two to three years, we’ll
see a significant change with far-reaching implications for those charged with
implementing new market arrangements.
There are four key factors that will lead to a very active mass retail market:
- Nondiscriminatory, cost-reflective arrangements for access by third parties
to all network voltage or pressure levels;
- Competitive, transparent, and liquid markets in wholesale energy (either
national markets, or regional markets with non-discriminatory arrangements
interconnector or pipeline access);
- Clear rules and processes for changing retailers that minimize the transaction
costs involved; and
- Sufficient scale (again, either nationally or regionally) to encourage
entry by global mass market retail players.
New EU directives require the designation of a regulator charged with approving
the commercial terms for third-party network access and for the provision of
balancing energy or flexibility services, legal and financial separation of
all distribution and transmission network operations, and phased moves toward
full market opening by 2007.
In addition, the new regulation in the electricity market, and its expected
equivalent in gas, will start to address the issue of charging for international
interconnectors and the allocation of capacity on those interconnectors. This
will increase the extent to which Europe will be regarded as a single market
in electricity and should encourage players to think of regional rather than
national wholesale and retail markets. This may encourage retail players to
participate in markets that would, otherwise, have been below an efficient
Finally, given the legal mandate for full retail competition, regulatory institutions
are likely to start considering the rules and processes put in place to allow
customers to change retailer, and the extent to which they provide a framework
for an active market.
TSOs and DSOs in either partially open markets, or in markets that are as yet
only theoretically open, will need to consider their response to questions
that are likely to arise, such as:
- Can existing processes and systems ensure that delivered energy is appropriately
accounted for and that customers can still be billed with churn rates
of 10 to 20 percent?
- Given the requirement for a two-stage market opening (2004 for nondomestic
and 2007 for full market opening), can the 2004 arrangements be future-proofed
or will there need to be two separate programs of work?
- What are the potential costs of creating a fully liberalized market, and
how will these be recovered?
- How will a significant internal and cross-industry work program be
- Which market design options will be most appropriate?
The two broad models that have been implemented around the world – distribution-centric
retail settlement, and direct wholesale market settlement – differ
in the way retail participants interact with the wholesale market.
Distribution-Centric Retail Settlement
The distribution-centric retail settlement model shown in Figure
1 has been implemented in a number of electricity markets in North
is based on a mandatory wholesale market (as is typical in the
North American electricity sector).
In this model, retailers receive their bills for wholesale energy
from the distribution system operator rather than directly interfacing
market operator. Retailers (and the DSO to the extent they have
a retail business) strike financial hedges with producers to reduce
to volatile wholesale
Retailers can opt to bill their end customers directly (direct
retailers), or have the distribution system operator bill the customers
on their behalf (indirect retailers). DSOs will also charge retailers
In terms of payments to the DSO, direct retailers simply pay the
wholesale price on their estimated consumptions. Indirect retailers
difference between their retail contract price and the wholesale
Because retailers don’t interface and settle directly with the wholesale
energy market, their consumption doesn’t have to be estimated immediately
for the purposes of settling that day’s consumption. The wholesale
market can be settled between producers and distributors, and then
retailers can settle
with distributors through a separate process. For example, their
consumption could be settled on a monthly basis. Depending on typical
meter reading frequency,
this can help to remove the need to continually refine estimates
of consumption as more recent meter reads are collected.
The model is based around a single, mandatory market. It is this
that makes it possible for the distributor to bill retailers for
of their registered customers, as it provides a single independently
determined market price. Hence it is most applicable to electricity
in Europe (in contrast to the US), even electricity markets are
typically bilateral in nature, and do not involve centralized mandatory
Within such a
context, it would be difficult for distributors to bill retailers
for the energy consumption of their registered customers.
Direct Wholesale Market Settlement
Figure 2 shows the broad market mechanics for direct wholesale
market settlement. This model has been implemented in a numberof
the UK electricity and gas markets, the Nordic electricity market,
and in a number
of states in Australia. It is being implemented in the Irish gas
and electricity markets.
This model is shown in the context of a bilateral contract market
for wholesale energy, but could equally well be implemented alongside
In this model, retailers interface directly with the arrangements
for imbalance settlement, either by themselves or as part of a
Customer consumption needs are estimated under the same timescales
as those used for the imbalance settlement process.
If the imbalance settlement arrangements are based on a rolling
settlement process a given number of days after delivery, customer
needs to be estimated daily and well within the defined timescale.
imbalance amounts among retailers will be required as actual meter
reads become available. Depending upon the approach to profiling
noninterval metered data
and typical meter reading frequencies, this reconciliation process
run for over a year after the settlement day.
All retailers bill customers directly, rather than having the option
of allowing the DSO to carry out billing on their behalf.
Key Factors for Success
Implementing FRC is no small undertaking. The total cost across
all participants of implementing FRC in the UK has been estimated
billion. This cost is attributed to complex market arrangements
and a high degree of systems
development to implement the arrangements.
Today a number of package solutions exist to meet the requirements
of FRC. Coupled with advances in the use of Internet technology
this has reduced the cost of implementing a competitive retail
market. However, even where simpler arrangements have been implemented
solutions, costs have been significant. For example, central costs
of around €20
million in some Australian states, and company-specific costs in
Canada and Australia of around €20 million. The cost and complexity
these programs remains high, and FRC implementations around the
world continue to be characterized
by delays and cost overruns. The risks presented by these programs
can be minimized through careful consideration of the following
Market Design Features
The failure to make appropriate decisions on critical market design
features upfront in an FRC program often comes back to haunt the
its implementation. The procurement and configuration of systems
will be dependent
upon these design features, as will be the interactions required
between participants to effect market transactions. An ill-defined
likely to change during
the life of the program, resulting in increased costs to change
the design being implemented by some or all participants involved.
if the design
does not change, there is still the risk of misinterpretation between
participants that will require change on behalf of one or all parties.
is of critical importance to the success of the program that a
number of design
are defined early and with sufficient authority. This will ensure
that systems design decisions can be made with confidence.
Approach to Program Management
Given the size, complexity, and number of participants and stakeholders
involved in implementing FRC, strong and effective central program
management is critical
to the success of the implementation. The key features of such
program management are:
- Appropriate governance arrangements;
- Industry-wide clarity on roles and responsibilities;
- Resource planning;
- Design control;
- Systems and processes trialing; and
- Stakeholder management and communication.
For those markets that have successfully implemented FRC to date, systems
costs have been a significant part of overall expense. Fully
automated systems solutions
have replaced the simpler systems and manual processes that were
used to support initial market opening. However, as international experience
of FRC implementations
has grown, so have the range of package system solutions that
It is now possible to avoid significant build and implement FRC
solutions (using either single packages with a wide range of
functionality or best-of-breed integrated solutions).This has dramatically
the costs of implementing FRC, while also improving the quality of
The UK recently began reviewing the systems and processes that support
FRC in gas and electricity in the belief that they were too expensive
and produced too many exceptions.
The nature of the systems solution will depend upon the overall
market design chosen, and on the approach taken to separation
of network and
competitive activities (and hence the use of legacy applications).
However, in general, the implementation of FRC will have implications
for systems performing the following functions:
- Customer registration and transfer;
- Meter data handling;
- Retailer settlement and billing; and
- Communications hub.
In defining a systems solution for FRC, the capabilities and constraints of
the existing legacy systems form a key consideration. While many of the package
solutions will claim to meet all FRC requirements fully, the final solution
architecture will typically include a number of legacy systems in one form
A number of vendors provide solution components for FRC. They include: Excelergy
Corporation, ICF Consulting, ITRON, Lodestar Corporation, SAP, and SPL WorldGroup.
While some of the packages can claim to meet the full scope of FRC requirements,
the specifics of a particular market, key design aspects and the use of legacy
systems (and associated constraints) will typically drive the solution chosen.
In this context, compatibility and ease of integration need to be considered
when selecting components from one or several package solutions.
As Figure 3 shows, the operation of the UK arrangements is split over
The Registration Service, which is at the center of
The Data Collector, who is an agent of the retailer,
The Data Aggregator calculates aggregated cumulative
A Central Agent, who:
There is a private data network to manage the flow of information between
The arrangements put in place in the UK electricity market to facilitate
Similarly, while the UK gas market arrangements were simpler initially
Finally, the technology underpinning the UK arrangements reflects the
Implementation and Testing
In addition to the definition of a solution architecture, there are a number
of other key factors in the detailed implementation program itself, which,
in IBM’s experience, are key to success. Two such important factors are
legacy data migration and cleansing, and market testing.
Legacy Data Migration and Cleansing
Existing meter and customer data may need to be migrated from the DSO’s
legacy customer information system (CIS) to the new systems which support FRC.
Alternatively, if the legacy CIS is to support FRC, the structure of the data
may need to be modified. An appropriate data structure is central to the implementation
of FRC. Retailers should be associated with metering points, and metering points
A retailer may serve many metering points, and an individual customer may be
associated with many metering points. However, this basis of customer registration
may not be consistent with the way in which data is stored in legacy systems.
Equally, the data in legacy systems may not be of the quality required to ensure
accurate accounting for energy delivered post-FRC. Many retail participants
in mature markets continue to have problems billing customers directly as a
result of data quality issues that were not resolved prior to the market going
Sufficient time should be allowed in the overall implementation program for
individual system testing (the usual factory, site and user acceptance testing
processes) and also for end-to-end market testing with market participants.
Given the nature of FRC implementation, it is particularly important to ensure
that all existing participants in the market prior to FRC are able to continue
their operations after go-live. Failure to ensure this could, in the extreme,
result in a failure to account appropriately for energy that has been delivered
and the risk of utilities’ statutory accounts being qualified.
Robust market testing is the key to mitigating this risk. This allows those
participants already operating in the market to test their systems and business
processes against the central systems in order to ensure that end-to-end
market processes operate as intended, and that central and participant
communicate with each other. The most appropriate approach to market testing
will need to be considered against the specific situation of each country,
and the way in which eligible customers supplied by third-party retailers
will be treated under post-FRC arrangements.
For example, if existing eligible customers will continue to be managed under
existing systems and processes and only migrated on new FRC systems at
a later date (for example, when they next switch), then testing of the
and processes may have to involve fewer parties.