Energy trading: demystifying the Price Forward Curve

This blog post has been co-authored with Marc Recht from FORRS, a Quasar integration partner

What is so special about trading electricity?

Electricity is special.

Unlike other commodities, it is very challenging to store; for most consumers, it is entirely infeasible. As such, purchasing and storing electricity in advance of demand is not viable; hedging a position is only possible through purchasing Futures contracts that promise delivery of electricity at a future date.

However, the problem with Futures contracts is that they only offer delivery over extensive periods. For example, if consumers wish to buy electricity more than a year in advance, they may have to settle for a contract covering the entire year.  For shorter periods, an Over-the-Counter (OTC) market is needed.

The electricity market aims to trade contracts several years in advance to one hour. So-called Hourly Price Forward Curves (HPFC) are used to price such contracts. What if you’re not interested in specific hours? That’s when you need the daily Price Forward Curves (PFC).

The HPFC essentially gives the future electricity price, as seen today.

The literature describes several ways of determining the HPFC. Typically, one first gathers seasonal patterns from historical spot prices, Including hourly, daily, weekly, and seasonal shapes. Expected seasonal effects are then incorporated into these patterns, which are fitted arbitrage-free to the quoted futures prices.

Most of the data used is historical daily (baseload) spot prices and historical hourly spot prices per day. These are used to calculate intra-day, weekly, and yearly shape factors (e.g., via linear regression). The forward curve is constructed to be consistent with all quoted futures prices.

Renewable energy challenges the infrastructure

Due to the increasing share of renewables, seasonal patterns are becoming harder and harder to define. Why? Because renewables increase volatility and price uncertainty. Since renewable energy depends on the weather, suddenly, you need to incorporate frequent weather forecast updates.

This results in an increasing need to refresh the Price Forward Curves, which creates challenges because the system architecture of energy trading companies has typically not been designed to handle this refresh frequency and demand flexibility.

Intra-day (real-time) curves are used, among other things, to price contracts. For this, they must be constantly recalculated, e.g., every 5 or 15 minutes. Then, for auditing purposes, the calculated curves must be persisted to maintain traceability between curves and reported prices (e.g., to a customer).

Data lifecycle

The lifecycle of a PFC, is typically:

• Read spot and future prices
• Depending o the model, read historical spot prices and previous curves (either real-time or end of day)
• Read the calendars
• Calculate the curve
• Publish and store the curve for auditing purposes

What kind of data do you consume to build a curve. If we take an hourly power forward curve, this is what we could have:

• 15- and 60-minutes spot prices (and history)
• Weekend, quarter, month, and year baseload and peak load futures
• Calendars

Where to store the data?

Historically firms have been using data warehouses to handle the data workflow. They do a fine job when the refresh frequency is daily; however, as we saw, renewable energies stress these infrastructures.

Even assuming that you have an infinite amount of money to run your IT department, data warehouses will create significant challenges the minute data freshness matters.

The need to run complex workloads on large data sets, refreshed on-demand with speed and convenience while delivering a meager Total Cost of Ownership (TCO) motivated the creation of Quasar.

Quasar is a high-performance, fully integrated data platform optimized for timeseries data that delivers instant insights whether the data is from 3 years ago or 5 seconds ago. It comes with powerful analytics (also known as AI) such as order book rebuilding from raw order data, waveform processing, and trend analysis.

Ingestion

For the HPFC model to calculate the real-time curve, it needs input data, like spot and future/forward prices for the various delivery times and the previous curve (either EoD or the latest RT curve). The resulting curve is then written back to be retrieved by other models and applications, e.g., pricing, PnL, or risk management .

With the rise of renewable energy in the mix, we saw one order of magnitude increase in the volume and frequency of updates, making it very difficult for systems relying on a centralized data warehouse to keep up. A data warehouse typically struggles with frequent updates, and if you’re not careful, you can end up with a Rude Goldberg machine to circumvent those shortcomings.

That’s where a system like Quasar replaces or supplements the central data warehouse and ensures systems can keep up with data updates while keeping a simple architecture.

We like to consider Quasar as the ultimate impedance adapter, petabytes of data directly accessible from your Python program (or Excel spreadsheet), refined in real-time in a form that is ready to use.

Quasar is optimized for high-throughput, low latency updates, a domain data warehouse, or data lakes struggle with. The system can be connected to a streaming engine such as Kafka or Kinesis. It’s also possible to ingest directly from the data provider with a connector that uses the low-level API.

How does it achieve it? One unfair advantage is the protocol that Quasar uses, the client and the server communicate to optimize the data transfer in a highly compressed format. In other words, the client will package the data perfectly to minimize the server’s workload, while the transactional engine of Quasar can handle live updates transparently.

Editing the PFC

While the authors of this document don’t necessarily love the OLAP/OLTP dichotomy, it can help frame a discussion.

One additional challenge that comes in with PFC is the fact that data needs to be edited and versioned. OLAP systems typically struggle with this, and OLTP systems can be lacking in the analytic department.

Even if your OLAP system supports updates (or edits), updates don’t solve a typical problem people working with PFC encounter.

Indeed, you may want to keep all the edits because, in some instances, you may want to run a computation “as it was when the event happened”, whereas in another case, you want to run it with “the freshest data”.

Quasar has a direct solution for this and supports optimized queries on multiple versions of the same data with a “RESTRICT TO” extension to SQL.

The “RESTRICT TO” allows you to use a column as a version tracker. For example, “restrict to the latest version” or “restrict to the version as of this date”. Because it’s built into the language (and the core of database engine), it gives one or two orders of magnitude faster queries than complex stored procedures or user-defined functions, which is essential for this kind of query.

Pricing

Pricing is central to intra-day tasks (for example, updating the price for your counterparty) and end-of-day tasks (PnL). Pricing complexity grows faster than the volume of data you process and generates additional data.

Each pricer is specific to every firm. However, you can offload much processing to your data infrastructure to simplify your development tasks or even your complexity processing.

While end-of-day tasks usually have fewer constraints, intra-day jobs always benefit from more speed. If you can reach an interactive pricing update, this can give you an edge over the competition and even allow you to increase the richness of your model.

There’s no reason to leave this alpha to the competition with an IT infrastructure that is not up to date. We firmly believe that higher software quality is correlated with higher business efficiency.

Conclusion

The share of renewable energy is only going to rise, which means that firms will have to incorporate larger and larger data sets to remain competitive.

With sound IT planning and tools like Quasar, we have been very successful in the past in helping energy trading firms upgrade their game.

If you’d like to know more, FORRS and Quasar are more than happy to showcase several successful implementations. Don’t hesitate to reach out!