The European Union plans to accelerate the pace of electricity and gas market
liberalization and has introduced dates for the phased introduction of full
retail competition (FRC) in both these markets. Some member states have already
implemented FRC arrangements that meet the requirements of the revised directives.

Many, however, have not. For the transmission and distribution systems operators
(TSOs and DSOs) of these member states, the dates for market opening are looming

Experience around the world shows that implementation of arrangements for FRC
is a complex undertaking. Successful implementation can take years to complete.
Those states that have implemented partial competition, and even those that
have full competition but where there has been little interest in the mass
market by external retailers, face a significant task in scaling up their systems
and processes to meet the transaction volumes seen in fully competitive markets.

Based on IBM’s involvement in the successful implementation of FRC in
many jurisdictions, we’ll address:

  • Challenges facing TSOs and DSOs around Europe;
  • Models for FRC already implemented
    around the world; and
  • Key factors that ensure a successful implementation for regulators, TSOs,
    and DSOs, including critical market design concepts, implementation approaches,
    and systems and technology options for retail markets.

European Challenges

The progress of implementing retail competition across Europe to date has been
mixed. Some markets are fully opened and have experienced fierce competition,
e.g., the UK electricity and gas markets and the Nordic electricity market.
There are some markets that are still only partially open, such as France,
and some that are theoretically fully open, but where fierce competition between
retailers has yet to take hold for all customer classes. For these countries,
current processes and systems may only have to cope with registering the switching
of a few thousand customers. Over the next two to three years, we’ll
see a significant change with far-reaching implications for those charged with
implementing new market arrangements.

There are four key factors that will lead to a very active mass retail market:

  • Nondiscriminatory, cost-reflective arrangements for access by third parties
    to all network voltage or pressure levels;
  • Competitive, transparent, and liquid markets in wholesale energy (either
    national markets, or regional markets with non-discriminatory arrangements
    for international
    interconnector or pipeline access);
  • Clear rules and processes for changing retailers that minimize the transaction
    costs involved; and
  • Sufficient scale (again, either nationally or regionally) to encourage
    entry by global mass market retail players.

New EU directives require the designation of a regulator charged with approving
the commercial terms for third-party network access and for the provision of
balancing energy or flexibility services, legal and financial separation of
all distribution and transmission network operations, and phased moves toward
full market opening by 2007.

In addition, the new regulation in the electricity market, and its expected
equivalent in gas, will start to address the issue of charging for international
interconnectors and the allocation of capacity on those interconnectors. This
will increase the extent to which Europe will be regarded as a single market
in electricity and should encourage players to think of regional rather than
national wholesale and retail markets. This may encourage retail players to
participate in markets that would, otherwise, have been below an efficient

Finally, given the legal mandate for full retail competition, regulatory institutions
are likely to start considering the rules and processes put in place to allow
customers to change retailer, and the extent to which they provide a framework
for an active market.

TSOs and DSOs in either partially open markets, or in markets that are as yet
only theoretically open, will need to consider their response to questions
that are likely to arise, such as:

  • Can existing processes and systems ensure that delivered energy is appropriately
    accounted for and that customers can still be billed with churn rates
    of 10 to 20 percent?
  • Given the requirement for a two-stage market opening (2004 for nondomestic
    and 2007 for full market opening), can the 2004 arrangements be future-proofed
    or will there need to be two separate programs of work?
  • What are the potential costs of creating a fully liberalized market, and
    how will these be recovered?
  • How will a significant internal and cross-industry work program be
  • Which market design options will be most appropriate?

International Models

The two broad models that have been implemented around the world – distribution-centric
retail settlement, and direct wholesale market settlement – differ
in the way retail participants interact with the wholesale market.

Distribution-Centric Retail Settlement

The distribution-centric retail settlement model shown in Figure
1 has been implemented in a number of electricity markets in North
This model
is based on a mandatory wholesale market (as is typical in the
North American electricity sector).

In this model, retailers receive their bills for wholesale energy
from the distribution system operator rather than directly interfacing
the wholesale
market operator. Retailers (and the DSO to the extent they have
a retail business) strike financial hedges with producers to reduce
to volatile wholesale
energy prices.

Retailers can opt to bill their end customers directly (direct
retailers), or have the distribution system operator bill the customers
for energy
on their behalf (indirect retailers). DSOs will also charge retailers
for network

In terms of payments to the DSO, direct retailers simply pay the
wholesale price on their estimated consumptions. Indirect retailers
receive the
difference between their retail contract price and the wholesale
energy price.

Because retailers don’t interface and settle directly with the wholesale
energy market, their consumption doesn’t have to be estimated immediately
for the purposes of settling that day’s consumption. The wholesale
market can be settled between producers and distributors, and then
retailers can settle
with distributors through a separate process. For example, their
consumption could be settled on a monthly basis. Depending on typical
meter reading frequency,
this can help to remove the need to continually refine estimates
of consumption as more recent meter reads are collected.

The model is based around a single, mandatory market. It is this
that makes it possible for the distributor to bill retailers for
the energy
of their registered customers, as it provides a single independently
determined market price. Hence it is most applicable to electricity
markets. However,
in Europe (in contrast to the US), even electricity markets are
typically bilateral in nature, and do not involve centralized mandatory
Within such a
context, it would be difficult for distributors to bill retailers
for the energy consumption of their registered customers.

Direct Wholesale Market Settlement

Figure 2 shows the broad market mechanics for direct wholesale
market settlement. This model has been implemented in a numberof
markets, including
the UK electricity and gas markets, the Nordic electricity market,
and in a number
of states in Australia. It is being implemented in the Irish gas
and electricity markets.

This model is shown in the context of a bilateral contract market
for wholesale energy, but could equally well be implemented alongside
mandatory market.

In this model, retailers interface directly with the arrangements
for imbalance settlement, either by themselves or as part of a
wholesale circle.
Customer consumption needs are estimated under the same timescales
as those used for the imbalance settlement process.

If the imbalance settlement arrangements are based on a rolling
settlement process a given number of days after delivery, customer
needs to be estimated daily and well within the defined timescale.
imbalance amounts among retailers will be required as actual meter
reads become available. Depending upon the approach to profiling
noninterval metered data
and typical meter reading frequencies, this reconciliation process
run for over a year after the settlement day.
All retailers bill customers directly, rather than having the option
of allowing the DSO to carry out billing on their behalf.

Key Factors for Success

Implementing FRC is no small undertaking. The total cost across
all participants of implementing FRC in the UK has been estimated
at €1.5
billion. This cost is attributed to complex market arrangements
and a high degree of systems
development to implement the arrangements.

Today a number of package solutions exist to meet the requirements
of FRC. Coupled with advances in the use of Internet technology
for market
this has reduced the cost of implementing a competitive retail
market. However, even where simpler arrangements have been implemented
configured off-the-shelf
solutions, costs have been significant. For example, central costs
of around €20
million in some Australian states, and company-specific costs in
Canada and Australia of around €20 million. The cost and complexity
these programs remains high, and FRC implementations around the
world continue to be characterized
by delays and cost overruns. The risks presented by these programs
can be minimized through careful consideration of the following

Market Design Features

The failure to make appropriate decisions on critical market design
features upfront in an FRC program often comes back to haunt the
program during
its implementation. The procurement and configuration of systems
will be dependent
upon these design features, as will be the interactions required
between participants to effect market transactions. An ill-defined
design is
likely to change during
the life of the program, resulting in increased costs to change
the design being implemented by some or all participants involved.
if the design
does not change, there is still the risk of misinterpretation between
participants that will require change on behalf of one or all parties.
Therefore it
is of critical importance to the success of the program that a
number of design
are defined early and with sufficient authority. This will ensure
that systems design decisions can be made with confidence.

Approach to Program Management

Given the size, complexity, and number of participants and stakeholders
involved in implementing FRC, strong and effective central program
management is critical
to the success of the implementation. The key features of such
program management are:

  • Appropriate governance arrangements;
  • Industry-wide clarity on roles and responsibilities;
  • Resource planning;
  • Design control;
  • Systems and processes trialing; and
  • Stakeholder management and communication.

Systems Selection

For those markets that have successfully implemented FRC to date, systems
costs have been a significant part of overall expense. Fully
automated systems solutions
have replaced the simpler systems and manual processes that were
used to support initial market opening. However, as international experience
of FRC implementations
has grown, so have the range of package system solutions that
are available.
It is now possible to avoid significant build and implement FRC
using off-the-shelf
solutions (using either single packages with a wide range of
functionality or best-of-breed integrated solutions).This has dramatically
the costs of implementing FRC, while also improving the quality of
the solution.

The UK recently began reviewing the systems and processes that support
FRC in gas and electricity in the belief that they were too expensive
to operate
and produced too many exceptions.

The nature of the systems solution will depend upon the overall
market design chosen, and on the approach taken to separation
of network and
competitive activities (and hence the use of legacy applications).

However, in general, the implementation of FRC will have implications
for systems performing the following functions:

  • Customer registration and transfer;
  • Meter data handling;
  • Retailer settlement and billing; and
  • Communications hub.

In defining a systems solution for FRC, the capabilities and constraints of
the existing legacy systems form a key consideration. While many of the package
solutions will claim to meet all FRC requirements fully, the final solution
architecture will typically include a number of legacy systems in one form
or other.

A number of vendors provide solution components for FRC. They include: Excelergy
Corporation, ICF Consulting, ITRON, Lodestar Corporation, SAP, and SPL WorldGroup.

While some of the packages can claim to meet the full scope of FRC requirements,
the specifics of a particular market, key design aspects and the use of legacy
systems (and associated constraints) will typically drive the solution chosen.
In this context, compatibility and ease of integration need to be considered
when selecting components from one or several package solutions.

As Figure 3 shows, the operation of the UK arrangements is split over
four roles:

The Registration Service, which is at the center of
the change-of-retailer process, is operated by the DSOs and maintains
a range of details for each eligible metering point, including the retailer,
the data collector and data aggregator, and the meter registers from
which data is to be collected.

The Data Collector, who is an agent of the retailer,
collects data from interval metered and noninterval metered metering
points. For non-interval metering the data collector provides an estimated
annual consumption where a reading is not available, or an annualized
meter advance where a meter reading has been made.

The Data Aggregator calculates aggregated cumulative
advance meter readings (or estimated annual consumptions where no meter
advance has been collected) by each
profile class.

A Central Agent, who:

  • Calculates and applies profiles to the aggregated data for each
  • Receives data on total transmission system off-takes for each distribution
    area and adjusts profiled estimates to ensure that the aggregated estimates
    are consistent with this top-down figure;
  • Aggregates retailers profiled consumptions nationally;
  • Carries out initial wholesale market imbalance settlement using
    this data as an input; and
  • Carries out subsequent reconciliations among retailers to update
    imbalance settlement calculations as improved estimates of retailer
    consumption (based on actual meter reads) become available.

There is a private data network to manage the flow of information between
all the relevant parties involved in the end-to-end processes of calculating
each retailer’s total demand. While the wholesale settlement arrangements
are, to a great extent, unaffected by the arrangements for retail competition,
it is worth noting that final settlement for retailers does not occur
until around 14 months after the energy has been delivered. This elapsed
time allows a greater proportion of retailer consumption to be based
on profiles applied to actual non-interval meter readings, rather than
on the estimates of the readings used for initial settlement.

The arrangements put in place in the UK electricity market to facilitate
retail competition probably lie at the extreme end of the spectrum of
complexity. For example:

  • Each DSO operates their own registration database, and this is typically
    implemented as a physically separate
    system to the databases that the DSOs’ retail businesses
    use for customer care and billing purposes.
  • The roles of interval meter operator and data collector
    were specified separately and open to competition from
    the start of operation of the new arrangements, with
    their non-half hourly counterparts opened to competition shortly thereafter.
  • There are eight separate regression profiles, depending on customer

Similarly, while the UK gas market arrangements were simpler initially
(with the majority of the process being managed centrally within Transco)
progressive regulatory moves to increase competition have added to complexity.
Equally, the unbundling of the distribution networks within Transco may
complicate the arrangements further.

Finally, the technology underpinning the UK arrangements reflects the
fact that they were designed and implemented in 1998-1989. For example,
nowadays we would expect Internet technology and communication hub applications
to be used in place of the UK’s centrally managed private data
network. This sort of technology has been implemented in a number of
North American markets and has recently gone live in the Belgian electricity
and gas markets. A similar technology solution is being considered for
the Irish gas market.


Implementation and Testing

In addition to the definition of a solution architecture, there are a number
of other key factors in the detailed implementation program itself, which,
in IBM’s experience, are key to success. Two such important factors are
legacy data migration and cleansing, and market testing.

Legacy Data Migration and Cleansing

Existing meter and customer data may need to be migrated from the DSO’s
legacy customer information system (CIS) to the new systems which support FRC.
Alternatively, if the legacy CIS is to support FRC, the structure of the data
may need to be modified. An appropriate data structure is central to the implementation
of FRC. Retailers should be associated with metering points, and metering points
with customers

A retailer may serve many metering points, and an individual customer may be
associated with many metering points. However, this basis of customer registration
may not be consistent with the way in which data is stored in legacy systems.
Equally, the data in legacy systems may not be of the quality required to ensure
accurate accounting for energy delivered post-FRC. Many retail participants
in mature markets continue to have problems billing customers directly as a
result of data quality issues that were not resolved prior to the market going

Market Testing

Sufficient time should be allowed in the overall implementation program for
individual system testing (the usual factory, site and user acceptance testing
processes) and also for end-to-end market testing with market participants.
Given the nature of FRC implementation, it is particularly important to ensure
that all existing participants in the market prior to FRC are able to continue
their operations after go-live. Failure to ensure this could, in the extreme,
result in a failure to account appropriately for energy that has been delivered
and the risk of utilities’ statutory accounts being qualified.

Robust market testing is the key to mitigating this risk. This allows those
participants already operating in the market to test their systems and business
processes against the central systems in order to ensure that end-to-end
market processes operate as intended, and that central and participant
systems can
communicate with each other. The most appropriate approach to market testing
will need to be considered against the specific situation of each country,
and the way in which eligible customers supplied by third-party retailers
will be treated under post-FRC arrangements.

For example, if existing eligible customers will continue to be managed under
existing systems and processes and only migrated on new FRC systems at
a later date (for example, when they next switch), then testing of the
and processes may have to involve fewer parties.