Paul Joskow, professor emeritus, Department of Economics
Richard Schmalensee, professor, Sloan School of Management
Francis O’Sullivan: From MIT, this is the Energy Initiative. I’m Francis O’Sullivan and welcome to the podcast. Today we’re talking about the future of electricity markets with Paul Joskow and Richard Schmalensee. Paul is Emeritus Professor of Economics at MIT and Dick is Professor of Economics and Management at the MIT Sloan School. Paul, Dick, thank you so much for being here today. It’s particularly exciting for me because over the last couple of years I have been spending a lot of time trying to immerse myself in the changing power sector, and we are absolutely in the midst of some very profound change. That’s coming from person of my generation, but both of you, with all due respect, have obviously been around a good deal longer. And indeed, you wrote a very important book, Markets for Power, back in when? 1983?
Paul Joskow: ’82.
FO: Let me put it like this. That’s how long I’ve been alive for. You’ve seen a lot of change in the power sector, I suppose, since that book has been published. What I’d like to start out with today is to ask for your reflections back at that time when you were working on that book. When you were looking at the power sector and what really motivated that. Because I think we’d like to move through to today, but I think it’s important for us to start with the genesis of that particular work. Because I think there are a lot of lessons to be learned going forward from it.
PJ: You really have to go back to that period in time. Because the context in which we came to do a study that led to the book was the deregulation movement of the late 1970s. It deregulated the airline’s prices and entry, deregulated interstate trucking prices and entry. They largely deregulated the railroads. That was more of an exit in pricing, to some extent. There was interest in, why can’t we do that for the electric power industry? Dick and I said, well, because the electric power industry is not like airlines.
Richard Schmalensee: We heard “why can’t we do this with electric power” from two economists in the Reagan administration. We went, what, wait, huh?
PJ: We said, you mean, just stop price regulation? They said, yes, you’re going to have real problems with that. But they came back to us and asked us to think about, how would you do it? What would you have to do to create competitive opportunities, at least in some sectors? It’s really that question that led us to write the book.
RS: There were two questions I would say. One is, they were power pools, particularly in New England. This is efficient economic dispatch, which looks a whole lot like a competitive market. Then there was the question, which we spent a lot of time on, what about investment? How would you do that?
PJ: The power pools, and there was a wholesale market. It was basically trading hourly electricity. It was competitive. But it was quite clear when you looked at the number, it was basically a short run marginal cost market. You couldn’t support investment in that market. The question is, how would you possibly reform the market? What kinds of structural changes? What kinds of contractual arrangements would you need that would, in the long run, support efficient investment?
FO: Since that time, there’s been the emergence of a variety of flavors of efforts with respect to market structures. I’m not quite sure whether some have been more successful than others. I’m going to ask you guys to reflect on that. But I am curious if you can characterize, broadly speaking, the approaches that did emerge through the deregulation process. The general features that became common across the U.S. and maybe a few of the international markets as well.
RS: It took a little while. Markets started focusing on energy and efficient dispatch and short run marginal cost and then there became the question of, how do you ensure capacity? Texas has decided that you can do it with energy markets as long as you don’t cap the short run price. Every place else they’ve fiddled with capacity markets to try to make up for the fact that they have caps on energy prices. Capacity markets have differed in a range of ways. Paul has looked at them more closely than I. But they’ve all required a lot of tinkering to try to fill in the missing money.
PJ: I think the design of markets was not the first step. It was a restructuring effort, which is really very important. We focused a lot on that in our book. The general view was that the generation was competitive. It could be competitive in the short run, it could be competitive in the longer run, in terms of entry of new capacity. But transmission and distribution were natural monopolies, and would continue to be regulated. The regions of the country that decided they wanted to go with a competitive model also separated generation from transmission and distribution. In some cases also disintegrated the existing companies to reduce concentration in the market. In California, the major utilities—PG&E, San Diego Gas and Electric, Southern California Edison—divested their generating plants but they had to do it in packages so that in the end, rather than there being three generating companies, there were more like 10 generating companies. Similarly, New England did that. New York did that. Texas did the structural separation. I don’t think they restructured generation, though, to make it more competitive. But that was the starting point, in a way, the precondition for having competitive markets. I would say that there was a lot of focus on creating competitive energy markets and ancillary services markets. The model that’s emerged almost everywhere in the U.S. is a day ahead and day of bid base, security constraint bid base market, that integrates the pricing of energy and ancillary services with the management of congestion. That’s the market in New England, it’s PJM’s market, it’s the market in New York, it’s the market in Texas now, which has locational pricing. It’s now the market in California. I think that market design has taken over. As Dick said, there was very little attention to what we thought was the most important question from an efficiency perspective, how do you attract entry? I think some of it was actually bad economics. There were people who just didn’t get that to support capital investment, you either had to have capacity pricing or contracts or very high prices during some hours to just to be able to cover the capital costs. There was a long literature in planning for electric power systems that would have led you to that conclusion if you really had incorporated it.
RS: But nobody really looked at that literature. It’s funny, we thought that the energy market would be relatively simple. It took a little while, as California illustrated, to make it work, but it has proven to be relatively simple. Capacity has been rather another matter. The other thing that happened, of course, was the development and evolution of the independent system operator regional transmission authority model. It’s a strange model but it’s developed and it seems to mostly work.
PJ: An alternative model was the UK model, where the independent system operator was the grid operator. I think that had certain advantages, because the grid operator also could manage the transmission system, maintain it, and invest in it. Whereas the ISO [Independent System Operators] model separates that and the ISO does planning and then in one way or another, tries to induce the transmission owners, of which there are many in many of these areas, to make investments that serve the collective good.
FO: This brings up an interesting feature from my point of view. If I look around the world at the plethora of models that exist… on the energy market front, you guys mentioned the nodal model that we generally rely on here. Most other countries do not. Coming from the electrical engineering point of view, the nodal model makes a lot of sense. But it adds a lot of complexity.
RS: It also originated at MIT so we naturally have sympathy for it.
FO: This is very true.
PJ: It’s not just the nodal model that’s been rejected in most other places. It’s also the idea that you basically have a central market that was managed by the system operator. The UK market, which began in the 1990s, was a central market. You put in bids. The system operator and the grid owner had some incomprehensible computer program that took the bids and solved for some kind of least-cost solution and set up a dispatch and did the dispatching based on prices.
FO: Let’s talk about where we are today and where we’re possibly going. I’m going to contend that power markets, broadly, have proved reasonably effective to date. I’m curious on your perspectives on this. They seem to function reasonably well. It certainly does feel that the missing money problem, the question about incenting new entrants to build capacity has been an ongoing challenge. But nonetheless, the lights have stayed on, more or less constantly, and therefore, it feels the system has worked reasonably well. We are now entering into a new era in the power sector as we see the proliferation of renewables, intermittent renewables in particular, and resources that have a very different economic characteristic than many of the resources these original markets were designed for.
RS: I’d argue that while intermittency is important, it’s also important that, in particular solar but also wind, as long as production is subsidized, have very low marginal cost. When you have an economically-efficient dispatch with marginal cost pricing, you all of a sudden have no money in the energy market. Pretty significantly, the energy market doesn’t support investment. It may have been inadequate in the past to support investment, but it becomes more inadequate now. And the investment in the renewables tends to be subsidized or required. You need fossil capacity to deal with intermittency and you don’t get it out of the energy markets. So you have to figure out how to get it.
PJ: I agree with that. I think the missing money problem is going to become more and more important because, especially as solar diffuses into the market at high volumes, you get a lot of hours during the day when you have very, very low prices. Those prices not only aren’t high enough to support conventional generation, they’re not high enough to support renewable generation, which raise finance basically outside the market through subsidies or contractual agreements with distribution companies. I think the whole issue of how you generate quasi-rants, short run margins to cover the capital costs of these facilities, if you’re going to do it through the market, becomes more and more challenging.
RS: We heard a seminar last spring here at MITEI. A young woman who was active in solar development made the point that nobody puts a solar project through the capacity market. They’re all funded long term with contracts with load serving entities that have renewables obligations. She said, “I love markets. These markets don’t work. What are we going to do?” I think that’s the $64 question.
FO: The rise of renewables—and you’ve mentioned this, Dick—obviously, there has been a dramatic reduction in the cost and so there are instances now where the economic competitiveness of these resources is drawing them into the market. In addition, of course, a lot of the momentum that these resources now have has come about because of mandates, policy mandates and so on, that come down through states, typically. Those mandates sit on top of, generally, the power market structure. From my point of view, looking at that, it feels like there is tremendous potential for a lot of distortive effect. Is it fair to say that? Are we at that point?
RS: It is and it isn’t. The fact that solar is being pulled into the market by government policy, the policy makers would argue that’s not a distortion, that’s a response to climate change. That’s a response to CO2 emissions and our desire to own distort. We don’t have a price on carbon. We don’t tax the bad thing so we subsidize the good thing. But it then makes it hard for markets to work. The FERC [Federal Energy Regulatory Commission] has wrestled with this. You have some resources that are subsidized by the state, competing in a capacity market with other resources that aren’t subsidized. How should that work? They don’t have an answer.
PJ: I think some of these challenges have been hidden by the fact that there is this large stock of existing dispatchable fossil generation. Basically wind and solar could free ride on them, in a sense. They provided the backup services. They were getting paid enough to cover their operating costs, but not much in the way of covering what would have been their ongoing capital costs. They’re exiting the market slowly. At some point, enough will exit the market where this issue is going to have to be confronted. How are we going to incent storage backup generation to come into the market given the current structure that we have? The current structure is unlikely to work. I see this whole thing potentially unraveling where you have mandates for wind and solar subsidies, required contractual arrangements with distributors, then you’re going to do it for storage, then you’re going to do it for rapid response gas turbines, and before long, we’re going to have two markets, a contract market and an energy market.
RS: I think we’re getting there. You see patches on the current system when Utility A or Generating Entity A says, we’re going to retire this plant. The system operator says, you can’t retire that plant, we’ll pay you to keep that plant going. What kind of market is that? That’s central planning. There are more and more of those.
PJ: Does it make any sense to have this period of very low energy prices and very low capacity prices where they have capacity markets and start having the nuclear plants retiring? Which are zero carbon emitters. While at the same time you’re subsidizing more zero carbon emitters to come into the market for exactly that reason. That’s kind of what’s happening.
RS: It seems to me the answer has to be, you’re putting patches on patches to try to preserve some sort of semblance of a market. The question is, are we institutionally and politically capable of biting the bullet and saying, the ISOs really need to do integrated resource planning at a regional level and think about the most efficient way to have the capital stock they need regionally, have debates about what that capital stock is, procure it efficiently. Instead of trying to tweak a market design that is plainly not working, that’s a big leap politically. But it may be a necessary leap.
PJ: It’s also easier to do in the states that have single state ISOs like New York and California and Texas. Where the public policies and the behavior of the ISO can be coordinated in many dimensions. Think about a large ISO RTO [Regional Transmission Organization] like PJM with states on the one hand, like New Jersey, which wants to go all green, and Maryland, and states like Ohio that don’t want any part of that. How are you going to harmonize all of these different public policies in one electric power system? I think it’s a real challenge for the multi state ISOs.
FO: It certainly feels today these issues that both of you have just been articulating, they’re beginning to rise to the surface across the various ISOs and RTOs, to different extents in different systems and so on. But people are getting worried. I recently was speaking with a wind developer, very focused in Texas, who is extremely worried about the growth of solar, which is going to be quite aggressive in Texas shortly, and the impact it’s going to have on that particular market. I’m hearing more and more people asking, we feel like there’s a problem, but how do we begin the process of going about putting in place, whether we’ll use the term “markets” or at least new institutional paradigms, that are fit for purpose, given the new technologies we have at hand?
RS: The people in the southeast, at Duke and Southern [Company], can kind of smile at this. Because they have the institutional setting. They can do this as central planning. I have a son living in Hawaii, so I follow developments in Hawaii. Hawaii has a lot of solar. More solar proportionally than California. But they don’t have a market so the utility and the regulators sit down and figure out what needs to be done. Then they do it. In California, trying to cobble together a mix of markets and mandates and subsidies, and God knows what, is harder. Not that Hawaii is a model but there’s more flexibility in that system than there is in one that’s supposedly market-driven, where you have to change the market design as the world changes.
FO: That leads me back to where we started. When you guys were writing your book and the principles you were trying to draw out, if you take that and reflect on where we are today, are there things that have surprised you in terms of how things have played out? Or do you feel like that what you had in mind at the time still holds? It’s just that we’re going to have to perhaps click “refresh” a little bit, in order to put in place that more fit-for-purpose paradigm.
PJ: I’m not sure we had anything specific in mind. I think we paid much more attention to the challenge of attracting investment. We thought the problem of creating competitive short-term energy markets would be simple. It wasn’t as simple as we thought. I think our favorite model was one in which, basically, you had distribution companies having a plan, taking competitive bids, and you get competition that way from suppliers with some kind of incentive-based contracts. Then those would be taken by the system operator and dispatched economically. That was the model we thought that would support investment. But it was not a model that was adopted here. There was a view the short-term market would cover it. I said before, it was bad economics. Going back to the late 40s and 50s, one thing everyone realized, if you only paid the peaker its marginal operating costs, you couldn’t cover the cost of the peaker. That undermined the whole equilibrium of the system. Somehow that got lost in all of this restructuring.
RS: I went back and looked at the book a few months ago. It was all about what kind of long-term contracts. You need long-term contracts to finance new capacity: what kind, who would do them. There’s a sentence, which I could probably find again, where it says, we doubt the ability of market prices to provide adequate guidance for investment decisions. Nobody else doubted that, really, for a while. But I think we’re back to the future, in a way. If you look at that book, it says we need intelligent decisions about the capacity mix and long-term contracts to enable financing. Here we are.
PJ: I think maybe the worst thing is to back into this with your eyes half-closed. If we’re going to a system where we’re basically doing integrated resource planning again, we’re going to be deciding on wind and solar and batteries and quick start and fast response generators and we’re going to be contracting those up to get them and keep them in the market. Maybe we should just recognize that and start putting into place mechanisms to make a system like that work efficiently, rather than waiting until we’ve driven all of the incumbent capacity out of the market, then trying to figure out how we’re going to make it work.
RS: If you compare the Southern Company, which isn’t aggressive on renewables, but take Hawaii. If you compare Hawaii Electric with California, they’re doing integrated resource planning and vetting the plan with the regulator. They have a 100% carbon-free goal as well. They have handicaps because of size that California doesn’t have. But it just seems smoother. They’ve had their hiccups. But it just seems smoother without trying to tweak a market. Let’s just see what we want, what we need, and let’s do it.
FO: The other element in what you’ve just said there, Dick, that I would like your brief reflections on, is the fact that going back to the original restructuring, and in particular to the breaking up of some of the incumbents initially during the restructuring process, the focus, was on generation and that made a lot of sense. But that was in an era when the demand side was not really viewed as having a whole lot of flexibility. Today we’re entering a very different world, obviously, where there is, through digitalization a host of other things, the potential for a lot more of that dynamic. Have we found ourselves today, particularly in the RTOs or ISOs here in New England, in a situation where we have one hand tied behind our back, in terms of our ability to actually bring a dynamic part of the system to bear and to leverage that in delivery of the overall system?
PJ: Certainly, all of the ISOs have tried to accommodate demand side. I recognize that demand is more flexible. Especially in areas where there are competitive retailers. They see this as a service that they can sell, both to retail customers, as well as into the market. I think we’re going to see that happening. The fact that we’re seeing more and more distributed rooftop solar, especially, creates opportunities for on-site storage. With storage, you can make heating and air conditioning much more flexible. I think those opportunities are coming and the ISOs have thought about how you integrate those into market mechanisms. There are challenges as to what’s the baseline, how do you make this work, are we really willing to charge small customers real time prices? Which sometimes can get very high. My view on that is, that’s what competitive retailers are going to do. They’re going to offer you a deal and say, we’re going to give you a fixed price. But in return for that you have to let me manage your load during 12 summer days and 12 winter days. I’m not going to freeze you out but I may curtail you for some period of time or put you on some kind of rotating schedule. I think that’s one of the potential benefits of retail competition. Other than that, I’ve never seen any great benefit to retail competition.
RS: I think with all the smart meter deployment, there’s enormous potential to do that. But you look at retail competition in Massachusetts, and it has not moved in that direction. I’m not sure it has in many other places. I do want to clarify something I said a few minutes ago. I think the vertically integrated model may have advantages in terms of flexibility. But the whole argument for restructuring was efficiency incentives in constructing generation and operating generation. Those are still there. The question is, you decentralize what you can decentralize, you centralize what you have to centralize, and that’s the right approach. It’s just being a little clear-minded about the need to centralize some things that we haven’t.
PJ: The pressure for restructuring, and one of the oddities of it, most of the restructuring movement took place in blue states. These were not the states that were wild about free markets in general, but they were reacting to the high costs of nuclear plants that had been put in and some of the high costs of public policy interventions. The way they applied PURPA [Public Utility Regulatory Policy Act] in terms of promoting co-generators with the creamy contracts, energy efficiency programs, the cost got put on the wires. Part of this was an effort, the proponent’s thought is that they would lower prices and put more pressure on the costs of building and operating generating facilities. I think to some extent that’s been successful.
FO: You think so?
PJ: Yes. There is much more pressure on merchant generators to perform efficiently. Dick mentioned the Southern Company which has more flexibility but they’re also completing two of the world’s most expensive nuclear power plants, which their customers are going to pay for. It’s that tail risk that I think people were trying to avoid.
RS: Absolutely.
FO: The only reason I asked that question, do you think so, is that here in Massachusetts or in New England, we still have, ultimately, the end user pays very high rates. On the wholesale side, there has been a lot of, you would argue, and I’m quite happy to take your point that there’s been efficiency gains there. But we’ve had a lot of other charges come online, some of them due to mandate and so on, some of them due to efficiency and other issues. I still think there’s a perception that in these markets, electricity, at least at the end user point of view, electricity is still expensive.
RS: Well, if you live in California, that perception is accurate.
FO: Absolutely.
PJ: I haven’t done the numbers for 2018, but in 2017, the average energy price in the New England ISO was $35 a megawatt hour, three and a half cents a kilowatt hour. My retail price was 12 cents a kilowatt hour. How do you make up the difference? There are losses, that’s about 10%. There are capacity charges, which at that time was maybe another 20%. Then they offer you a two-year fixed price. That’s got to be hedged. Not only they hedge the price, but they hedge the quantity. It kind of adds up. If someone could offer me, we’ll charge you the real time price plus 10%, it would be very much more risk but it would be a lower price.
FO: That’s fair. The final topic I’d like you to reflect on a little bit today is something that I think is coming down the pike very rapidly, and that’s the issue of storage. We’ve mentioned storage a few times here. Storage feels to me like a technology, when we say storage, I suppose specifically things like batteries and so on, but feels to me like a technology that has the potential to be profound in altering and adding flexibility to the system. It also feels like a technology that’s going to be very difficult to integrate into the markets, or at least feels that way to me because it seems like there are a lot of things that storage can do, some people claim simultaneously, and how you actually integrate that into existing systems feels complicated. I’d love your thoughts on it. I know you’re thinking a lot about that now as you’re thinking about the evolution of the system in general.
RS: We’re thinking about it in the context of the ongoing Future of Storage study. I must say, I don’t know where Paul is, we haven’t talked about this much, but part of me says, wait, it’s not that hard. We’ve got markets for the things that solar does that are market-driven and we have needs of distribution utilities for storage and the distribution systems. How hard can it be? Then the other part of me says, no, it’s the simultaneity and the multifunction and the layering that makes it hard and I’m not sure I know how to think about that. I can’t tell at this stage. I don’t know where you are, Paul.
PJ: I think there are some challenges here. I think maybe we’ve overcomplicated it.
RS: It’s possible.
PJ: I’ll go back to some of the discussion we had before. If you look at the numbers for California, which now has a lot of solar, you see that at the end of the day, the prices go way up for four hours basically. That creates an arbitrage opportunity. You can buy during the day and you can sell during that period of time. You can use batteries which are very, very fast. We come back to the question I raised before. Is the margin large enough to support the cost of the batteries? The answer today is, no. That is going to be resolved either by coming up with some new way of pricing that service, or through subsidies, which is what’s happening now, subsidies or mandates, or potentially large reductions in the costs of those storage devices. Dick is right. There is a simultaneity problem here. You’re not going to see that the large margin for the four-hour ramp until you have a lot of solar in the system in particular. But you can’t have a lot of solar in the system without having this backup capacity or else it’s not going to work properly. That’s why I think this missing money problem is going to become more and more important. If we don’t let the prices get really high on some of these days when you really have to ramp a lot very, very fast, you’re never going to be able to support it with energy prices. Maybe there will be a capacity pricing mechanism for it, at least we’re looking at right now, it’s mandates that are coming through.
RS: It’s mandates without, at least as I read them, without much notion of what solar, what storage will do. It’s just that we’ve got intermittent generation, so we’re going to need some storage, so put some storage in.
PJ: I think we’ve been a little bit misled with the fact that a lot of the existing experiments, I’d call them, with storage have found a lot of the benefits are on displacing grid investments. The fact that grid investments and storage, or potentially a substitute, it’s also true of generation. If you’ve got congestion, if you got a pocket, one way of solving it is to build more transmission. The other way is to put generation in the pocket. In that sense, it’s not really that much different from generators.
RS: Or you put storage in the pocket.
PJ: Or you put storage in the pocket in this case. But the incentive problems in the markets we have are the same in both cases. If we don’t have high prices inside the pocket to attract entry or generators, or we build the transmission capacity too soon so the prices are never realized, you’re not going to get a market solution.
RS: This goes back, Frank, to your earlier point about pricing. There’s a nice study, at least conceptually nice study, about Hawaii that says, how do we do 100%? What would the cost be? If we can price dynamically, there’s all kinds of non-fancy storage that can be done. You can pre-make ice for air conditioning. There are opportunities for pumped hydro because of the terrain. You don’t need new high-tech batteries to cut the cost enormously if you provide the economic incentives by getting the prices right. We tend to think of storage as really advanced technology, super high lithium stuff. There’s a lot of ways to store. If you get the prices right, you provide incentives to do those things, as well as to use the fancy stuff.
PJ: On the demand flexibility side, at least for residential and small commercial customers, it is air conditioning and heating that are both the largest fraction of consumption, also the most flexible. We know that consumers like air conditioner cycling programs. If you give them a discount, you tell them on some number of days we’re going to cycle your air conditioner on for half an hour and off for half an hour and you get a discount. They like that. We know there’s a potential for doing that but we haven’t created the right incentive structure to make that work.
RS: I learned some time ago that the Miami airport buys it wholesale. Because I got stuck there overnight missing a hurricane. By five o’clock in the morning, it was 45 degrees in that airport. They clearly bought at night, they chilled it down, and it made perfect sense except it was really cold.
FO: Dick and Paul, this has been fantastic. I could certainly sit here and ping silly questions at you guys and learn a lot for a lot longer. But we’ll wrap it here. I will say this though. In reflecting on what you both said, certainly there’s a sense of back to the future in many respects. All change again. In many respects, it’s all the same. I think the all the same is really ultimately one that I’m sure you as economists would appreciate. That is, we should focus on getting the price right. That still remains the core principle and I’m sure in all of our work going forward that will continue to be a message that we’re going to be keen to deliver to the broader world. Thank you so very much indeed. It was an absolute pleasure.
PJ: You’re welcome. Thank you.
RS: Thank you.
FO: Show notes and links for this episode are at energy.mit.edu/podcast. Tweet us @mitenergy with your questions, comments, and show ideas, and subscribe and review us where you get your podcasts. From the MIT Energy Initiative, I’m Francis O’Sullivan. Thanks for listening.