Perhaps we should not measure the role of technology in realizing the Federal
Energy Regulatory Commission’s Standard Market Design by the volume of text that
it is afforded in the commission’s “Notice of Proposed Rule Making.” A scant five
pages outlines FERC’s technology vision.
Experience teaches us in the technology business that you need to get your
business requirements right if you want the technology to be right. So perhaps
FERC is correct in spending the vast amount of text on the design of the market
instead of the technology.
But FERC has outlined a tall order in its technology blueprint for Standard
Market Design (SMD). We need to manage expectations. SMD is a vision. The reality
will be something less. But the key is to keep pushing the vision.
FERC outlines three foundation attributes of the design’s technology:
Transparency: the ability to understand what the software does.
Testability: the ability to understand and compare performance.
Modularity: the ability to change software modules without changing other
These three attributes are all reasonable, perhaps noble, objectives. FERC,
however, frames them as somewhat equal. The reality is that there is more cause
and effect than appears on the surface. There are also big differences in the
achievability of any of these objectives in the near-term.
First, lets take modularity. As FERC notes, achieving this objective requires
standard interfaces. But actually it takes much more. It requires a common vision
on design and architecture within the components that make up the software systems
in an ISO/RTO. The data constructs and the methods utilized within the components
must also be standardized, or else the data that crosses these “standard interfaces”
will be useless.
An example would be you picking up the phone and calling France. This is seamless
and easy because long ago the interfaces between the U.S. network and the French
system were standardized. But if the person who answers the phone knows only
French, and you know only English, you are not going to get very far in the
conversation. It is the same for systems. To make these systems really modular
there will need to be a much deeper level of standards and integration. Modularity
is much more intrusive into the design of software components than just interfaces.
So what about testability? Certainly the notion of a standard benchmark test
suite for similar components is a good idea. Testability in the context of SMD
is really a proxy for certainty. That is, a known input will produce a known
output. The proxy for certainty so far in ISO implementations has effectively
been operational audits of the market systems that are part of many ISO/RTO
tariffs. These audits are effective, but they do not provide the degree of cross
market/operator benchmarking at which FERC is driving. FERC is shrewd in pushing
this point since such benchmarking would be a subtle but effective pressure
on vendors to conform and improve systems performance over time. If there is
anything a vendor likes least, it is to see their name last in a published benchmark
“bake off.” It is just too competitive out there.
Which leads us to transparency. This element of SMD is actually quite easy
to achieve yet so difficult to implement at the same time. Transparency is really
documentation. Public domain documentation. In an industry standard framework
and at an industry standard level of quality. For us working in this environment
every day, such documentation would be a breath of fresh air.
I will go out on a limb here. One significant act that FERC can take in regard
to technology in SMD implementation is to mandate that all SMD implementation
be documented at an industry standard level and placed into the public domain.
Of course, there will be some screams from the system vendors notwithstanding
their rhetoric in the market about supporting open standards, etc. That is because
they all believe that the “special” elements of their systems are their competitive
differentiator. And indeed in reality they may be right. Why else would so many
ISO/RTO executives feel “locked in” to a vendor?
I outlined FERC attributes in reverse order for a reason. They are not equal.
You arrive at modularity through good design, be it mandated or market driven.
Either way, you get standardization that then can be tested (compared) as FERC
envisions. To be testable, you must already be transparent. So there is a natural
sequence to these objectives. Modularity is the end point. Transparence is the
If Not Now, When?
So just how achievable is FERC’s vision in the medium- to long-term? (You will
understand as you read on, the short-term prospects are dim).
FERC’s basic vision of the technology is right. But as the saying goes, the
devil is in the details. The details will likely be messy and expensive.
First, one must recognize that the technical environment that SMD targets is
not green field; it is legacy. A sizable number of ISO/RTOs are already operational.
They have already made their first wave investments in market and operational
systems. No matter how visionary SMD is, implementation will be incremental
to their current environment.
Second, green field or not, the ISO/RTOs can only implement this vision if
the system vendors provide a suite of systems as envisioned in SMD.
Which leads to the third element. Vendors are clearly not all that motivated
to provide SMD “vision-like” technology fast or for free.
Figure 1 outlines the continuum of ISO/RTO systems. If you look at currently
operating ISO/RTOs, for the most part, their systems are unique. Unique in the
sense that a single market operator, without modifications to their systems,
could not just up and operate in another’s jurisdiction. The architectures of
the systems are generally specific to the market the system serves. This is
an artifact of the history and timing of each of the currently operating ISO/RTO’s
Figure 1: ISO/RTO Continuum
The next step up on the continuum is open documentation (read FERC transparency).
The level of documentation of ISO/RTO systems is actually more available than
many think. (Indeed it is remarkable the level of documentation on ISO/RTO in
the public domain already). But it is also true that there are large holes in
Indeed, a survey would likely reveal that each ISO/RTO has a large degree of
documentation regarding systems, processes, and operations residing in the cerebral
cortices of a few and consequently very important individuals at each of the
respective ISO/RTO and/or its vendor. As I outlined above, conforming this knowledge
to software industry best practice in terms of form and substance and making
it available in the public domain is a significant enabler for advancing the
state of ISO/RTO transparency.
Further up the continuum is the notion of componentization. This is the FERC
modularity element. It is also the notion that has formed the foundation for
best practice in software design and implementation for some time. But here
is where the Standard Market Design vision starts to be more difficult.
From the incumbent vendor’s perspective, breaking major portions of their offering
into plug-and-play components is not trivial technically or commercially. Most
of these systems have evolved from power engineering algorithms utilized during
pre-deregulation. Without question, the focus has been on the technical elements
of the algorithms instead of the eloquence of the integration design. It is
not that a more “eloquent” architectural design is not viewed as preferable
by these vendors. The question is who is going to pay and how long do they have
to complete the migration.
At the extreme end of the continuum is the notion of “plug-and-play.” FERC
uses this language directly in the NOPR. Plug-and-play conjures up notions of
the common RJ11 phone jack. No matter who manufactures your phone, the jack
is the same. When you plug it into the wall presto dial tone (the
This is where a reality check is needed. Plug-and-play, in the context of the
pedestrian concept of plug-and-play, within and/or between ISO/RTO systems,
will not occur anytime soon unless it is mandated by law (it is unclear if FERC
has the authority to make such a mandate) and it is subsidized (it is very clear
that the ratepayers and taxpayers would be the ones paying).
Why? The answer lies in the current market realities. The fact is that there
is little market gravity pulling vendors in FERC’s direction. First, the market
is quite small relative to the potential wallet of spending in the future. Thus
the return for making a strategic investment by any single vendor is not all
that good. The risk of both time and future market success would likely send
vendor corporate strategic planners looking for greener pastures for their scarce
investment dollars. Second, the rework required to current vendor systems is
extensive when one considers how much structural change is required to migrate
current systems to a technical architecture that supports the objective of Standard
Market Design. Last, the migration will require vendor cooperation and/or standards
(not vision but detailed technical standards). The former is not likely and
the latter will take a long time.
If you review the record for Standard Market Design, the above reality check
is not a surprise. FERC has been told in a number of technical conferences that
there is a gap between vision and reality. It is, however, remarkable that nearly
every presenter on the subject, be it a vendor, a market operator, or a market
participant, points to the same general technical solution. There is a remarkable
degree of convergence on a common view for integration in ISO/RTO systems looking
There are many names for this integration structure: common information bus,
application integrating bus, and enterprise application integration (EAI), to
name a few. The EAI approach is a predominant strategy in a number of industries.
EAI is best suited to an environment that requires an integration of operational
and business systems into a single system; that requires an integration of both
established legacy applications with new applications; and finally, has the
desire to implement systems that could be easily adapted in the future to new
and changing application requirements.
EAI uses the latest software advances to seamlessly integrate separate systems.
It is an integrating platform that allows use of best-in-class tools for each
operational requirement in an integrated manner.
Figure 2 is purposely a different depiction of EAI than you have likely seen
before. Indeed most pictures of EAI opt for the over-simplified “EAI Magic Bus”
(EMB). You just plug the applications into the EMB and you have plug-and-play.
Want to switch stuff around? No problem. Just unplug one component and plug
in a new one. If it were only that simple.
Figure 1: EAI Hierarchy
EAI is a very powerful technology for integration. And indeed as a foundation
for ISO/RTO systems in the future, it has the potential to take center stage.
But it is not magic. Much design, architecture, and new development are required
before the simple “magic bus” picture is anything close to reality.
Looking at the hundreds of pages on market design and the mere five pages on
technology design in the SMD NOPR, one can really get an idea of what FERC is
thinking. Technology is not considered the issue.
Indeed, FERC is right. The technology frameworks and standards to achieve FERC’s
vision are no longer bleeding edge technology. Transparent, testable, and modular
systems have been the design and market norm for quite some time.
Achieving FERC’s technology vision is a market issue. A market issue that starts
with the very premise of SMD in the first place: A single standard design for
electric wholesale markets in the United States. If such a standard did exist,
then all the vendors would be in a race to provide the systems to implement
this design. And operators would be in a race to ensure the new design cured
the issues of the past five to 10 years of implementations. I am sure if that
were the case, we would end up with modular, testable, and transparent systems.
So perhaps FERC was prudent in spending only five pages on technology vision
after all. As is frequently the case, technology is not the issue. Business