MISG Project: Diagnostic Testing for Earnings Simulation Engines in the Australian Electricity Market

About Integral Energy

Integral Energy’s Retail business sells electricity to customers by purchasing wholesale electricity from the National Electricity Market (“NEM”), and on-selling this at a margin to retail customers, who vary greatly in terms of load demanded. During this process, Integral Energy incurs a commodity price exposure equal to the difference between its wholesale purchase price for electricity from the NEM, and its sale price to its retail customers. By purchasing electricity at a variable price, and on-selling it to consumers at a fixed price, electricity retailers are exposed to Earnings at Risk (EaR).

Integral Energy reduces EaR exposure by entering into financial hedges, including electricity swaps, caps and other derivatives. These contracts offset an electricity retailer’s physical exposure to the NEM by effectively fixing the price at which wholesale electricity is purchased. Derivative contracts are purchased in order to keep EaR levels within established acceptable limits, the levels of which are determined in accordance with the risk appetite of the retailer.

The cost incurred by an electricity retailer is a product of two stochastic quantities: load and price. Since the underlying joint distribution for load and price is unobservable, EaR must be determined using a simulation engine. By generating simulations of load and price, an electricity retailer can estimate the earnings distribution for a given future time interval, and subsequently determine the associated EaR.

The aim of this project is to develop a comprehensive set of diagnostic tests which can be applied to the output from an earnings simulation engine (ESE). While there are many approaches which can be used to generate load and price simulations for the purposes of computing EaR, currently there is no established means by which a simulation engine can be assessed relative to historically observed data. Furthermore, there is no established approach for comparing alternative simulation methodologies. This project seeks to develop a standard by which ESEs can be evaluated and compared.

Problem Outline

National Electricity Market

Earnings are defined as the difference between income and expenses. For an electricity retailer, income arises from selling electricity to consumers at a predetermined fixed rate. The costs arising from supplying this electricity are determined by the actual cost (load times price) of purchasing the consumed load from the National Electricity Market (NEM). Note that each state (e.g. NSW, QLD, VIC, SA) has its own spot price and state load.

Purchasing electricity directly from the NEM results in highly volatile cash flows. This volatility is reduced by using financial hedges, such as swaps, futures and caps, which allow the holder to lock-in the price at which they purchase electricity in the future. Since the electricity load which will be consumed is unknown, the volume of hedges required cannot be determined with certainty.

The distribution for earnings is a function of consumed load and the spot electricity price. Purchases from the NEM are settled at half hourly intervals. Electricity derivatives are also written with payoff functions that settle at half hourly intervals. Historical load and price display high, positive correlations, and the marginal density for price displays a skewness in excess of a lognormal distribution. As a consequence of these features, simulations are the most efficient way to estimate the earnings distribution of an electricity retailer.

The Earnings at Risk (EaR) is defined as the maximum negative earnings to an electricity retailer, relative to expected earnings, for a given time interval and simulation quantile (or confidence level), if the current portfolio is held to maturity, and is fully exposed to the volatility of the spot price and committed load. Typical confidence levels are 95%, 97.5% and 99%. EaR provides a means to measure a retailer’s exposure to adverse spot price movements, and as such, this risk metric guides the hedging policy a retailer applies to committed load.

Simulation Engine Diagnostics

Currently there exist many feasible methods which can be used to generate joint simulations for load and pool prices for a given state. However there is currently no means by which these established simulation engines can be evaluated. There are several criteria that must be assessed to determine the effectiveness of an ESE.

The historical data will need to be analysed in order to identify the key statistical quantities which must be reflected by the simulations. Examples include moments and correlations. Since the times series are generated at half hour granularity, it is also necessary to determine at which time scales the statistics should be assessed e.g. annual, quarterly, etc. The rate of convergence for the statistics generated by each ESE must also be measured.

There is scope for additional criteria to be added as well. Potentials include analysing the spikes (frequency and location) generated by the simulation engine, and the implied term structure generated by the mean and volatility of the simulation engine.

In addition to selecting all the pertinent criteria for testing an ESE, it is necessary to develop suitable tests that can measure the suitability of the simulations. The primary concern is to assess a simulation engine relative to the observed historical data. It is also necessary to compare and contrast the output from different simulation engines using the same diagnostic tests.

Desired Outcomes

The primary goal for Integral Energy is to develop diagnostic tests for earnings simulation engines, and apply this test to several sets of simulations generated by various engines. The diagnostics should be able to benchmark a given set of simulations against historical data as quickly as possible. It should also be capable of measuring the quality of a simulation engine, and allow comparisons between competing simulation engines.

A set of criteria is sought for benchmarking simulation paths against observed historical data. This includes an analysis of the statistical properties, and also the convergence behaviour of each ESE.

The diagnostic tests should be applicable to any provided set of simulation paths for load and/or price, where accompanying historical data is also made available. The tests should be developed such that one can write a routine capable of taking the simulation paths and historical data as an input, with a complete diagnostic report generated as the output.

The performance measures should be specified such that it is possible to compare the relative performance of simulation engines which have been calibrated on the same historical data. It should be noted that it is perfectly acceptable for a simulation engine to perform well for some of the diagnostics, whilst performing poorly for others. This will provide the opportunity to compare and contrast different earnings simulation methodologies.