The long-term vision of MoP’s proposal for MBED is to ensure discoms save costs and electricity markets transition to a new form of equilibrium and efficiency, under a centralized market mechanism.
For this, the discoms will have to cede the powers which they enjoyed over their long-term PPAs with state, central and private generators. In return, a centralised market operator is expected to ensure least cost dispatch of all the PPAs in the country, through a mandatory common pool which will be run on a day ahead basis. The expectation is that the discoms will only gain from MBED.
We believe that the MBED may not be a positive-sum game for all the discoms and markets are way beyond equilibrium and efficiency.
The title of our article is borrowed from Nobel Laureate Friedrich Hayek’s seminal book, “The Fatal Conceit: The Errors of Socialism” which warns us of the dangers of a planned economy. The fatal conceit is the belief that the world can be shaped according to human desires and central planners can engage in rational constructivism to design a superior state-of-affairs as compared to decentralized interactions in an economy.
The Fatal Conceit in MBED proposal is our central planner’s belief and expectation that our concurrent and complex electricity sector should be directed in a top-down manner to improve upon the existing order. Our contention is that the MBED proposal is the essence of a big government, top-down planning with an authoritarian structure. Especially, that
Each and every discom in the country is not capable of running its portfolio optimally and need to be directed by wise and benevolent planners
The collective decentralised optimisation by discoms as per their own self-interest would not achieve the efficiencies that a central authoritarian design would achieve
What is good for the country will often coincide with what is good for each discom
Unfortunately, belief and expectation will always be insignificant for as big a matter as MBED through regulatory fait accompli.
At the outset, we would like to draw your attention to two key features of the Indian Electricity Markets. First, as per Indian Constitution, the devolution of power in the electricity sector lies with both the central and state governments which can respectively regulate and operate the markets in their own jurisdictions. Given that both central and state governments are responsible for its development, the electricity sector is highly complex. Second, the hosts to the markets i.e., Power Exchanges follow a voluntary participation for trades. The key word here is voluntary. Nobody restricts discoms to shift their entire procurement to Exchanges in search of higher efficiencies.
The MoP’s Policy Note on MBED alludes to benefits of thousands of crores, apparently accruing to discoms with the stroke of a senior bureaucrat’s pen. An admirable goal, that none of the states or their discoms asked for. With ~75 discoms operating in India today and some of the finest minds at Exchanges shaping, tracking, analysing Indian Power Market transactions for over 10 years, nobody saw these thousands of crores. Why was it visible to our central policy makers only?
The natural questions that arise:
Had the Spontaneous Order[1] existing in our power markets failed? What qualifies as a market failure?
Is there any precedent of MBED globally? How are the markets organised in the developed world?
It is hard to build and easy to destroy. Is MBED the creative destruction required for improving the market efficiencies? Are there better ways of achieving the same without stifling multiple institutions like Power Exchanges, POSOCO and SLDCs, etc. and the emergent order that they have created?
What is the guarantee that the good intentions of MBED will not lead to bad outcomes? What could be the Unseen or the Second Order effects of MBED?
Lastly, a technical question that has invited less attention than one would have ideally expected. Whether the day ahead market and the underlying algorithms used by Power Exchanges would actually result in unit commitment decisions for generators and least cost dispatch?
We would now attempt to unpack some of these questions one by one:
Failure of Spontaneous Order: If optimization is defined as satisfaction of a single objective function – least cost system operations, why did the discoms fail to recognize the benefits of dispatch through Exchanges. If nothing else, they could have simply bid their entire day ahead demand and offered their entire supply portfolio on Exchanges as suggested by IEX in its gross bidding petition. For details readers can refer to this article by IEX. Is MBED then the solution looking for a problem? because it may be solving a problem that doesn’t actually exist.
If we stick to one definition of optimization, it is easy to accuse the Spontaneous Order i.e., an order which emerges as result of the voluntary actions of discoms and generators, as a failure. But if we look at other facets of optimization, there is merit in believing that Spontaneous Order has found its optimality in the current equilibrium.
Day ahead transactions through Exchanges have delivery and offtake commitments frozen way ahead of actual supply and delivery of power. To be precise, in the current order, it is as high as 12 to 36 hours. Any changes to these advance commitments in real time are associated with uncertainties in both prices and volumes which therefore have costs and operational frictions.
Under the current self-scheduling mechanism, discoms respond to unforeseen changes in demand and supply commitments of their day ahead planning by rescheduling (right-to-revision) of generators contracted through long-term PPAs, and at zero costs. Important point to note here is that in the current order rescheduling is a free option available to discoms for managing their demand supply balance near real time.
While there might be merit in arguing that supply side uncertainties would be mitigated through freezing of schedules on a day ahead basis, so the problem isn’t that big. However, we are to understand that in current system operations, supply side uncertainties like tripping of a committed generator are only manageable to an extent through rescheduling of other generators, and discoms may have to resort to market based purchases, which is a compulsion, not a deliberate choice.
So, there might be merit in believing that the uncertainties in demand far outweigh the uncertainties in supply and this is more amplified with bad weather forecasting, lack of reliable data acquisition infrastructure, etc. Certainly, not a failure of Spontaneous Order in our markets.
Therefore, if benefits are to be assessed, a simple deterministic optimization across aggregated demand of states is misleading and theoretical if not anything else. Benefit of doubt may be accorded to the Spontaneous Order, wherein participants have settled for an optimization objective function of minimum regret, i.e., I am ready to forego the benefits of a centralized optimization for avoiding the incremental costs associated with rigidities imposed by a frozen schedule under the assumption that the costs of correcting positions in real time to respond to demand uncertainties far outweigh the costs of day ahead optimisation through Exchanges.
We are aware of many discoms like WBSECL, MSEDCL, BSES, TSDISCOMS, GRIDCO, MPPCL, RUVNL etc. having benefited massively over the years by optimizing their portfolios while considering their long-term PPAs and the prices on the day ahead market of Exchanges. Therefore, the universal claims of failure of Spontaneous Order cannot be made.
If the benefits of a centralized dispatch are to be assessed, it should quantitatively assess the costs and benefits comprehensively, while internalising the incentives and negative externalities of such optimisation.
Precedence of MBED globally: The simple answer is a NO. We have two extreme models of the developed world, the voluntary European Market model and the mandatory US Market model i.e., PJM, MISO, NYISO etc. Even though the US model consisting of ISO/RTO is mandatory, the choice of joining a pool along with entry /exit decisions from the pool are voluntary. FERC regulations allow for multiple pools in the US and incentivise discoms to join pools. While RTOs/ISOs lure discoms by delivering reasonable prices and managing risks.
Both the market models trade almost 100% of volumes through Exchanges and RTO/ISOs respectively. To address the issues of rescheduling, Europe has developed a deep intra-day market and the footprints are huge. For the US, risk is mitigated by ISO re-dispatching generation with updated demand forecasts like the SCED model in India, ensuring that the benefits of a centralized dispatch are accrued, and risks of uncertainty are contained within tolerable limits.
Instead of identifying the reasons why Europe, despite being a voluntary market, has been able to attract higher volumes on the markets? And the US despite being a Federal polity has been able to successfully push reforms which were not necessarily top-down? and we haven’t been able to do so? We are trying to shove a cocktail of diagonally opposite and a rigid market model down the throat of our discoms rather than creating conducive conditions for participants to voluntarily shift volumes to day-ahead market while providing a mechanism for using the real time market as a balancing tool. It would be better to undertake reforms in some states rather than pushing something on each and every state. If then, there is success in one state, demonstration effect will prevail and other states would also follow.
MBED as Creative Destruction: Ever since the electricity markets were liberalised, most experts have been banging their heads into the wall over the centralized vs. decentralized markets dilemma. Centralisation and decentralisation are both good as well as bad. Naturally, there are plenty of grey areas in between as shown in the break-up below.
The centralized market takes care of pretty much everything and is usually easy to use. Centralisation is important for accessibility and serviceability. It offers benefits like stability, efficiency, symmetrical information flows etc. But decentralization offers benefits too like choice, ownership, controlling your own resources and data, understanding the market narrative while having an element of privacy.
The centralised markets can lead to positive sum transactions. But decentralised markets too result in positive sum transactions and financially beneficial outcomes if the discoms pursue their self-interest, and the market is guided by right institutional structure, rules for market access, dispatch and price systems. Adam Smith called such a system the “Invisible Hand” of markets.
The arguments thus circle back to the grey area continuum of the centralisation and decentralisation dilemma. The grey areas or the tradeoffs (as we will call it now) existing between the centralisation and decentralisation may be the second-best solutions amongst the two outcomes. But these are still better economic mechanisms which can get us the benefits of both the centralisation and decentralisation without many of their underlying costs and frictions.
The concurrent legislative power of Indian Constitution has provided electricity markets a secure political foundation with both the state and central control i.e., a decentralised centralisation (grey area in the middle). The constitutional architects deliberately adopted this model to suit the political economy of the country and the very objective that welfare should be the preserve neither of the Center nor the States alone. The tradeoff here was the fine market preserving balance between authoritarian structure and market access, control and self-ownership and accessibility and serviceability etc.
It has taken many years since the enactment of the Electricity Act, 2003 to develop institutions including ALDCs, SLDCs, SERCs at the state level and NLDC/RLDCs, Power Exchanges and CERC at the national level. These institutions have provided a proper balance between the center and state jurisdictions along with a reasonable direction for operating and improving our electricity markets.
The question that therefore arises is whether MBED is the creative destruction that we need for putative efficiency gains and whether it will only destroy or dilute our fledgling institutions. By destroying these institutions, are we also destroying what might have been?
Unseen effects of MBED: There is only one difference between bad policy makers and the good ones. The bad ones confine themselves only to the visible effects of the policy while the good ones consider both the effects that can be seen and those effects that must be foreseen.
While MBED has the least cost dispatch as the seen effect, but the unseen effects are also many and inevitable. We believe that MBED may turn out to be a zero-sum game for market participants creating winners and losers, where in the gain for some discoms and generators will come at the cost of the other discoms and generators. Let us try to see some of these unseen effects of MBED.
Discoms: MBED has two unintended consequences on discoms. Firstly, the original beneficiaries of long-term PPAs may end up paying higher energy costs to the generators. CIL’s coal allocation continues to be rationed and the sector has been in a deep crisis for decades now. There is a strong possibility that the generators with lower variable charges are dispatched more such that they breach the ACQ limits of their FSAs early in the year. Thus, they would have to pay higher incentives to CIL for additional coal or procure higher priced e-auction coal. This will increase the costs for the original beneficiary discoms of the generator. Secondly, in the absence of resource adequacy and reliability mandates at the discom level, some discoms to avoid fixed cost burdens may not go for adequate cover for peak demand through long-term PPAs. They may rather prefer to free ride on the long-term contracts of other discoms having higher peak demand coverage. This could lead to more volatility in the day ahead market prices, affecting discoms adversely.
Generators: MBED may also have unintended consequences for different ownership categories of generators i.e., state, central and private. MBED will lead to a change in the dispatch levels of the generators as compared to levels seen in the current system. This will have financial consequences for central vs state vs private PPAs. Some government owned discoms, for internal fiscal management of states do not want to dispatch cheaper cost central and private generators, but rather prefer to dispatch the state owned generators to keep the money within their state. Also, MBED will expand the liquidity of day ahead markets, which are essentially “Energy only Markets” multiple times. The unique feature of energy only markets is that it gives signals for new investments but does not necessarily incentivise new capacity additions. This is attributed to the “Missing Money Problem” of energy only markets (more about it here). Expansion of the day ahead markets may thus aggravate the missing money problem for merchant generators more adversely, which in turn only exacerbates the pervasive NPA problem.
Transmission: MBED will dramatically change the direction and magnitude of power flows on interstate tie-lines, imports and exports of various states and regions in the country. Our transmission grids, both at the intra-state and inter-state level were not originally designed for a centralized dispatch. Most discoms have a dependency on short term merchant markets which includes Exchanges and Bilateral contracts. The discoms use them for their daily as well as seasonal demand supply balancing. Any putative benefits to discoms from MBED can be easily wiped out by unforeseen transmission congestion, import and export restrictions affecting discoms adversely.
Market Clearing Algorithm: All the hype around MBED is worthless if underlying algorithms don’t lead to its stated objectives of unit commitment and least cost dispatch.
The MoP’s Concept Note highlighted the broad economic benefits of MBED through a day ahead market of Exchanges for a single day and extrapolated it for the year 2019. The note stated that the day ahead market of Exchanges would lead to the unit commitment decisions, which were presently not considered in the SCED and unit commitment would accrue additional savings to discoms.
POSOCO in its report “Detailed Feedback on Expanded Pilot – SCED of Inter-state Generating Stations pan-India” dated March 2021 evaluated and quantified the benefits of unit commitment by running an SCUC algorithm along with the SCED. POSOCO has conducted an extensive mock exercise for an 8-month period from June 1, 2020 to Jan 31, 2021 for all the inter-state generating stations in the country.
No such extensive data-driven simulation exercise was conducted by MoP to quantify the benefits of using the Exchange based algorithms for undertaking unit commitment decisions. Validating whether the unit commitment decisions, which are so critical for additional savings, can be taken through a day ahead algorithm of Exchange is critical?
The objective functions and constraints of the two algorithms i.e., Exchanges and POSOCO’s SCUC/SCED are different and produce different results/benefits. While Exchange algorithms maximise the Economic Surplus of the buyer and seller trades considering the bid types and transmission constraints; the SCUC/SCED determines least cost schedules and dispatch considering the various technical and operating constraints of the generators and grid.
We believe that SCUC/SCED allows for explicit modeling of the constraints while in Exchanges, these constraints need to be internalized through limited bid types and bidding structures that it offers to sellers and therefore may be less efficient for unit commitment decisions.
The lessons and challenges faced by the European and US power markets (here and here) which are based on exchange and SCUC/SCED algorithms respectively are instructive in arriving at the most effective algorithm for scheduling and dispatch.
Thank you if you have made it this far. We are about to conclude now.
Final Thoughts
The Fatal Conceit in MBED lies in the belief that the states and their incumbent discoms should be directed through top-down interventions, that this design would be successful and would improve upon the existing order of our markets. Like we have witnessed in the past, such actions of centralisation, whether it was nationalisation of banks, coal, demonetisation, GST, COVID vaccines and agriculture etc. have in the hindsight proved to be fatally flawed, contested and with disturbing seen and unseen effects. The lesson here is that the governance has to be through negotiations, consensus and consent of the states.
As MoP begins the campaign for re-designing our electricity markets, they would also do well with few instructive lessons from two of the greatest proponents of liberal markets ever - Friedrich Hayek and Adam Smith and attempt reforms that are not top-down plans but ones which transfer power back to states/discoms along with their nascent institutions. This fascination with creating national systems like one nation grid one price backed by state coercion should end. It is much better to have organic evolution all around the country with different systems owned by different governments and private players. There is an equilibrium and efficiency in our state-central decision making construct, which should be conserved; too much intervention and there will be contestation, not enough innovation and there will be stagnation.
History will not be kind to the command-and-control mechanism as MBED. This time the costs would be a clean energy transition impacting millions of human lives, their livelihoods, and wellbeing of the country. By the time the flaws of these conceited actions are revealed, it might be too late.
Thank you for reading, and if you enjoyed the Jai & Veeru’s Newsletter, please do share and subscribe!
[1] Spontaneous Order, first introduced by Adam Smith and advocated by Frederick Hayek, is the idea that society and markets, like languages, cannot be centrally directed or planned
Can I just say how wonderfully helpful, concise, well-articulated, structured, researched (and many more adjectives) all of your writings are. Thank you very, very much for your efforts. I hope you get many times more good luck than what you have given us through this substack!