How quants shaped the modern energy markets

When deregulation spread through US and European power markets in the early 1990s – sweeping away fixed prices and guaranteed markets – utilities realised they would need to transform their business models. Formerly focused on volumes and costs, businesses now needed to extract maximum value from every asset to achieve competitive advantage.

Applying the mathematical models used in financial markets to energy markets was much harder than anyone anticipated. The unique properties of energy prices – the existence of big price jumps and mean reversion, for example – meant that many financial models didn’t work. It wasn’t a case of making a few adjustments, but rather it required large-scale bespoke rebuilding.

As part of the Energy Risk at 30 series – which celebrates Energy Risk’s three decades – we are looking back at seminal events that explain the background to the landscape we see today. The article below, first published in 2014, tells the sometimes-painful story of how quantitative analysis of power markets began and evolved.  

Quantitative analysis revolutionised financial markets in the 1970s and 1980s, but its use in the energy world was limited until the 1990s, when deregulation spread across natural gas and power markets in the US and Europe. As the old order of fixed prices and guaranteed markets came to an end, gas and power companies became exposed to substantial price and volume risk for the first time, rendering risk management an urgent new priority.

Faced with the need to calculate exposures to market prices and value physical assets and contracts, energy firms turned to the tools and models of quantitative finance that were being used in financial markets. Engineers in Houston, traders in New York and academics the world over saw new opportunities presented by the deregulated world and set about applying and creating quantitative tools to help extract optimum value from the optionality that exists throughout energy markets.

“The energy markets are saturated with options,” comments Vincent Kaminski, Houston-based professor at Rice University’s Jesse H Jones Graduate School of Business. “Practically any flexibility or rigidity in a contract or a physical asset can be translated into some sort of a financial option.”

Prior to liberalisation, the production and distribution of energy was very much a co-operative endeavour, with the emphasis on minimising costs rather than finding competitive advantage. “Firms didn’t mind too much if they gave an option away, as there was no market in which to monetise it,” says Kaminski. “After deregulation, firms realised they couldn’t give away options any more. Now, for the first time, they needed to find a way of pricing options – putting them into a portfolio and managing it in a systematic way.”

For the energy industry, the introduction of financial market modelling represented a clear break with the type of models that had been used in the past. “Before deregulation, analytical modelling was based on fundamentals, but not in order to understand the market, rather in order to justify to regulators why rates should be as they are,” says Krzysztof Wolyniec, managing partner at Millwright Capital Management, a Connecticut-based investment manager. “All of a sudden, the approaches had to be adjusted to actually understanding what’s going on in the markets.”

[Since the crisis], people have become less trustful of models and moved more towards qualitative and subjective measures

A new breed of merchant energy companies with large quant teams emerged in the early to mid-1990s. These included North Carolina-based Duke Energy, Atlanta-based Mirant, Dallas-based TXU, Oklahoma-based Williams, as well as Houston-based firms Calpine, Dynegy, Enron, El Paso and Reliant. Quantitative finance became the backbone of the merchants’ business models, and engineers – who had formerly been in the driving seat – found themselves playing second fiddle to traders.

“Traders started looking at areas that had been the preserve of engineers or procurement,” says Carlos Blanco, managing director at California-based consultancy Black Swan Risk Advisors (BSRA). “The engineers had been the ones with the most power when it came to investment or divestment decisions. After deregulation, engineers were still needed to operate the facilities, but decisions became based not on maximising volumes and minimising costs, but on making as much money as possible from the asset.”

The merchant energy firms underwent a huge cultural change very quickly, in an effort to bring quantitative finance into newly deregulated markets. “I remember at Enron in the early 1990s, we went through a period of internal training to make employees aware of the concept of optionality and the business of options in fixed assets and contracts,” says Rice University’s Kaminski, who joined the firm as a quant in 1992 and went on to become head of its quantitative modelling group.

Early challenges

Initially, the idea of applying the tools of quantitative finance to the energy markets seemed a straightforward one. “In the beginning, the perception was that this would be a walk in the park and that we’d just be able to make a few adjustments to the tools being used in the financial markets,” says Kaminski. “We thought that things could be transplanted mechanically to energy markets.”

But by the early to mid-1990s, market participants had realised that the unique properties of energy prices – such as the existence of big price jumps and mean reversion – meant that more bespoke tools were needed. Another complicating factor was the preponderance of spreads compared with other markets. “Most things being modelled in energy are spreads – the difference between oil and refined products, or electricity and coal, for example – so you not only have to model the underlying sources of uncertainty, but you also have to model the joint evolution of them,” says Chris Strickland, co-founder of Sydney-based software vendor Lacima.

Although some basic quantitative analysis had been in use in the oil markets since the early 1980s, it had been carried out on a very small scale. Options trading had been in existence for several years following the hugely successful launch of the New York Mercantile Exchange’s West Texas Intermediate crude oil futures contract in 1983. These options were mostly plain vanilla and linked to futures or over-the-counter swaps. In comparison with financial markets, the energy options market was very small and not considered quite as important, so it was often assigned to the less senior traders.

“Typically more junior people with less experience were given options books,” says Millwright’s Wolyniec. “Because the financial models didn’t fit too well it was considered a fairly risky market and that was what kept it small.”

The starting point for pricing most oil options was the ground-breaking Black-Scholes model of options pricing developed by Fischer Black and Myron Scholes in 1973, which was revised in 1976 by Robert Merton. However, it soon became apparent that the assumption that prices follow geometric Brownian motion – small, random movements – severely limited its applicability to energy markets, where large price spikes are common and mean reversion often occurs.

By the late 1980s, market actors were turning their attention to modelling energy prices and behaviour in an effort to produce energy-specific models. One of the first such models was developed and described in a paper in 1990 by Rajna Gibson and Eduardo Schwartz, entitled Stochastic convenience yield and the pricing of oil contingent claims. This paper developed and tested a model for pricing financial and real assets contingent on the price of oil. It is considered to be an important milestone in commodity-specific research and prepared the ground for two later papers by Schwartz (see Must-reads for energy quants box below).

The early 1990s were a time of experimentation for quants in energy markets. But as with Black-Scholes, it was soon discovered that not all the models being imported from the financial world were appropriate for energy markets. Practitioners say this became particularly apparent in 1993 and 1994, as the trend towards ever more complex options led to the collapse of some of the financial models that had been brought across into energy. “The energy industry rejected those transplants, rather like an immune system,” recalls Rice University’s Kaminski. “But what happened as a result is that the end-users came back and asked for tools that better reflected the optionality embedded in energy-related contracts and fixed assets.”

Not all tools from the financial markets were doomed to failure. It was found that certain types of complex option, such as Asian options, spread options, contingent options and binary options, could be adapted for energy markets very well, as they adequately captured the properties of energy prices. Then there was the development of swing options – contracts that give the holder the option to change, or ‘swing’, the quantity of energy purchased a certain number of times over a set period. These were developed in the early 1990s to address the need to manage volumetric risk in energy markets.

The period between 1992 and 1996 is often considered the heyday of innovation in energy quantitative finance. “This was the time when we really broke ground,” remembers Kaminski. “Everything that has been done since is really just tweaking what was created at that time… We had the vision of transforming the energy markets and making them more efficient.”

Two particularly important books came out at the end of that period: Managing energy price risk was edited by Kaminski and published in 1996, while Dragana Pilipovic’s Valuing and managing energy derivatives came out in 1998. Both helped to establish the discipline of quantitative energy finance in its own right. Other important papers were published around this time, but despite the growing importance of energy risk management, the literature was still sparse.

Lacima’s Strickland recalls being approached by an Australian energy company in the late 1990s to value a portfolio of call options and Asian options linked to power. “Because electricity prices are different from nearly all financial time series, we looked at the existing literature to see what had been written to help us and we realised there really wasn’t anything applicable out there,” he says. “There were very few energy-specific books or papers written, and even those that had been written usually used pure financial techniques for pricing.”

This spurred Strickland to make his own contribution to the field, along with Les Clewlow, an associate fellow at the University of Technology, Sydney, with whom he founded Lacima. Clewlow and Strickland co-wrote the book Energy derivatives: pricing and risk management, which was published in 2000.

According to Strickland, one of the challenges they wanted to address was the difficulty of actually applying quantitative models to real-life situations. “Coming up with a model that captures the distribution and the time series properties of a data set is just the first stage,” he says. “The second stage is actually using it to value a refinery, a power station, a
gas storage facility or a tolling agreement, taking into account all the associated optionality and constraints of the asset or contract.”

The emphasis in the early models was very much on understanding and capturing price behaviour rather than the applicability of the model, say practitioners. “The early models were mostly about price behaviour – spikes, mean reversion, non-linear relations between different assets, the behaviour of forward curves, things that are unique to energy,” says Blanco at BSRA. “The first generation of models didn’t look at the challenge of liquidity, for example. Ever since then, the effort has been focused on bringing in the additional constraints, such as transportation, pipeline and power plant operating constraints.”

Up to the early 2000s, quantitative finance began to build up such a head of steam that energy companies were put under extreme pressure to change their way of doing business or risk not surviving, recall market practitioners. “The merchant energy players were growing aggressively and there was an overriding feeling that a market-centric approach was needed,” says Blanco. “The environment for unregulated players was such that you had to adapt and grow or you would be put out of business.”

But even as merchant generators proliferated, extreme price moves were throwing up deep challenges to established quantitative methods. In the US Midwest during the week of June 22-26, 1998, power prices surged from around $25 per megawatt hour (/MWh) to as much as $7,500/ MWh, highlighting the potential price volatility of deregulated power markets. Then, just as the use of quantitative techniques in energy appeared to be at its peak, quants faced their biggest challenge to date: the 2001 collapse of Enron, which created a crisis of confidence for many merchant traders. During the episode, many well-respected quants lost their jobs and were forced to leave the industry. But perhaps the biggest damage quantitative energy finance suffered was to its reputation.

“There was a negative feeling around the tools of quantitative finance after the collapse of Enron. It was felt that very complex models could be developed essentially to justify any decision that management wanted,” says Wolyniec at Millwright. “It was a shame, because some of the things the merchant firms had done were genuinely hard to do – they were creating new markets where there were no valuations – but the idea that you can use models to produce any number you want led to a big mistrust of these models.”

A new cycle

With the leading proponents of quantitative finance now outside the market and its reputation in tatters, it seemed the energy market’s experiment with financial tools had come to a rather abrupt end. However, the entry of investment banks to the void left by merchant generators marked the start of a new cycle, during which the discipline began to earn newfound respect. “When the banks came in the models began to be taken more seriously again,” notes Wolyniec.

Between 2003 and 2008, there was a gradual maturation of quantitative finance in energy markets, as the use of such techniques grew in popularity again. As in the early days, the use of quantitative modelling by the firms involved was not without its perils. Like utilities and other companies before them, banks frequently underestimated the distinctiveness of energy markets by applying financial models on an indiscriminate basis, say practitioners. But crucial lessons had been learned. “One of the big improvements was the realisation that most of the models transplanted from the financial markets were assuming unrealistic flexibility and efficiency of commodity markets,” says Kaminski at Rice University. “A lot of effort went into recognising constraints and rigidity embedded in physical assets and this produced more realistic valuation models for power plants or natural gas storage facilities. This was a major improvement.”

By this stage, many industry participants had realised that greater complexity did not necessarily mean greater results. “One of the biggest lessons of the early 2000s is that the most complex models are not necessarily the best,” says Millwright’s Wolyniec. “That is a very difficult lesson for a quant to learn, as it isn’t natural… People put enormous resources into creating very complex models of their physical systems and those models failed overall. It was expensive to implement them and they didn’t work well.”

Lacima’s Strickland agrees. “We worked out years ago that you can make more and more complex models but you can end up with something that no-one understands, and therefore no-one can calibrate or use.”

Despite these improvements in energy-specific modelling, the reputation of quantitative finance has since been tarnished by the 2008 financial crisis. Although the energy market had little to do with a crash triggered by poor US subprime mortgage loans, the fallout dented enthusiasm for quantitative methods across the board. “The 2008 financial crisis has frequently been blamed on the failure of computer models,” says Brett Humphreys, New York-based executive director of commodity risk management, at Morgan Stanley. “People have become less trustful of models and moved more towards qualitative and subjective measures for decision-making,” he says.

Many practitioners argue that quantitative techniques have been unfairly maligned as a result of the financial crisis. They argue that models were incorrectly used to justify poor decisions by senior management, and that more scrutiny must be placed on the way these models are deployed in future. A New York-based quant with an investment firm stresses the importance of marrying quantitative models with strong subjective judgement. “I’ve never met a quant who wasn’t aware that his model had bias and errors, but sometimes that acknowledgement gets lost, as the model’s findings are presented higher up the company,” he says.

Today, quantitative modelling is much lower down the agenda of many energy trading firms than it was prior to 2008. Instead, the focus has shifted to regulatory compliance and attempts to understand changing market fundamentals, such as the boom in shale oil and gas sweeping North America. Industry practitioners say this is taking managerial support and IT resources away from quantitative research – a worrying development that could leave energy companies unprepared for future challenges. But quants remain hopeful that the tide will turn once again.