Energy trading: order book
Context
As the share of renewables grows in the energy mix, electrical supply and demand unpredictability puts massive stress on trading firms. In particular, building the order book becomes much more computationally intensive.
The order book is the list of buys and sells organized by price levels. It provides valuable trading information, which helps traders and improves market transparency.
In a nutshell, what Quasar delivered
- Deployment of complete market and weather data capture, from start to finish, in less than a week
- Instant order book rebuilding at any point in the day while using less cloud resources
- Storage footprint divided by 10
- Future proof solution thanks to built-in scalability
Challenge
Because of renewables, electricity production can suddenly drop or increase, creating sudden demand, which, in turn, puts stress on order book construction.
There can be an increase of at least one order of magnitude in the volume of orders, making the order book two orders of magnitude more computationally expensive.
Energy trading firms typically built an architecture relying on a relational database. That was sufficient to handle previous volumes. This approach does not work anymore regardless of the increase in computational power dedicated to the relational database. Building the order book is more and more expensive as the number of orders grows.
The solution
Quasar is connected directly to the market data provider and captures the data without a third-party tool. Capture is done with virtually no latency.
Quasar uses its unique suite of compression algorithms to compress the data faster than it arrives and use its micro-indexes to reduce future lookup and feature extraction time. This results in significantly reduced disk usage without impacting the queryability of the data.
Each instrument is stored as a timeseries a two dedicated tables (one for orders, one for trades). Weather data is organized by source.
Tables can be tagged at will, enabling a flexing querying mechanism based on those tags. Tags can be queried recursively. For example, obtaining all the instruments of an index or all the weather data of a specific region.
Queries use a SQL interface, dedicated API, or one of the numerous connectors available. For example, traders can load data directly into Tableau using the Quasar ODBC driver. Quants can load timeseries directly into Python Pandas using the Quasar Pandas API.
Quasar has a built-in order book computation module that enables sub-second order book rebuilding at any time of the day, even for deep order books. These order books are then accessible from a web browser using a dedicated dashboard built using the Quasar framework.
Benefits
- Firms can keep up with the market and have linear scalability capabilities thanks to the clustering capabilities of Quasar, increasing their agility and competitiveness
- Compatibility with all standard analytical tool suites for maximum flexibility
- Faster time to a solution: Quasar abstracts away all data engineering complexity
- Outstanding TCO thanks to the combination of data compression and rapid order book rebuilding, saving on disk storage, CPU power, and development time
Liquidity Provider: pool management
Context
A liquidity provider has many responsibilities; one of them is to help their customers validate that they are compliant with best execution. Another one is to analyze order books to advise their customers, for example, about their order book spread or which trading opportunities they may miss.
Best execution ensures you conform to the auction rules of the trading you are participating in (for example, sell to the highest bidder). The order book is the list of buys and sells organized by price levels.
In a nutshell, what Quasar delivered
- Replaced a constellation of custom tools, spreadsheets, and scripts to deliver a consistent and immediate view of the liquidity pool. What took a day of work is now instant
- Greatly increased the quality of advices given to the users thanks to increasing analytical capabilities
- Future proof solution thanks to built-in scalability
Challenge
The challenge to adequately cater to the needs of your customer for a liquidity provider are multiple.
As order data comes in, rebuilding the order book takes longer and longer since there are more orders to go through. This creates pressure on the data system, especially during peak activity. When doing end-of-day analysis, building the complete order book means going through all the day's orders.
However, building the order book is half the battle since the liquidity provider needs to estimate the order book spread to see if a participant could improve their trading with a tighter book.
Connecting the order book with the relationship matrix (who may trade with whom) helps to advise users regarding what opportunity they would have if they expanded the breadth of actors they are trading width.
In addition to the order book, participants need to ensure they are complying with best execution. For example, ensuring they are selling to the highest bidder. This is done by matching a trade with orders available at that time.
These specialized processes are complicated to do with a regular database management system and often require specialized tools or programs that lack flexibility and generally result in an expensive, complex toolchain.
The solution
Quasar is connected directly to the market data provider and captures the data without a third-party tool. Capture is done with virtually no latency.
The data is stored, compressed, and managed by Quasar. Information is organized in timeseries and is tagged in a way that makes queries easy and convenient. This approach is future-proof and allows you to change your mind "whenever you want" without touching the data itself.
Relationship matrixes are also ingested in the database as they get updated.
Queries use a SQL interface, dedicated API, or one of the numerous connectors available. For example, traders can load data directly into Tableau using the Quasar ODBC driver. Quants can load timeseries directly into Python Pandas using the Quasar Pandas API.
Quasar has a built-in order book computation module that enables sub-second order book rebuilding at any time of the day, even for deep order books. These order books are then accessible from a web browser using a dedicated dashboard built using the Quasar framework.
Quasar can also compute the top and the spread of any order book (for the whole pool or a given actor). From there, evaluating the spread of a specific actor relative market activities is a straightforward task and allows to provide high-value advice to a customer.
Using the dashboard, the liquidity provider can also see if a given actor order book would be matched by another actor they are currently not trading with and thus
Lastly, best execution can be done using advanced joins available in Quasar (ASOF JOINS) that instantly match a trade with an order at a given time.
Benefits
- Liquidity providers can leverage Quasar to give high-value feedback to their users without relying on custom development or third-party tools; increasing value offered to customers
- Firms can keep up with the market and have linear scalability capabilities thanks to the clustering capabilities of Quasar, increasing their agility and competitiveness
- Compatibility with all standard analytical tool suites for maximum flexibility
- Faster time to a solution: Quasar abstracts away all data engineering complexity
- Outstanding TCO thanks to the combination of data compression and built-in aggregation functions, saving on disk storage, CPU power, and development time.
ITCH Data: Order Book building
Context
Market data mainly consists of orders and trades.
The order book is necessary to evaluate how orders may be filled and is a powerful tool to identify actors' intentions.
On top of that, reconciling orders with trades is a legal requirement: actors need to conform to best execution.
The order book is the list of buys and sells organized by price levels. It provides valuable trading information, which helps traders and improves market transparency.
In a nutshell, what Quasar delivered
- Deployment of complete Nasdaq market data capture, from start to finish, in less than a week
- Instant order book rebuilding at any point in the day while using less cloud resources
- Storage footprint divided by 10
- Future proof solution thanks to built-in scalability
Challenge
Nasdaq enables subscribers to track the status of each order from the first time it is entered until it is either executed or canceled via the ITCH protocol. To properly participate, one thus needs software capable of decoding the protocol.
Once the data is decoded, one needs to store that data in a form that can be later processed to rebuild the order book, which means processing every order since the opening of the markets.
Lastly, it is necessary to match orders with executed trade to ensure that best execution has been respected. This means parsing every order and finding the best match at the time of the trade.
Nasdaq data volumes are in the region of 2 TB per day.
These volumes make it impossible to rely on standard data warehousing technologies.
Typically, firms will store data in files processed by custom-made software for historical data analysis and keep one or two days of data in a dedicated operational database.
The solution
Quasar has an ITCH data importer that can decode and capture the data at very high speed without any intermediary conversion. This ensures data capture is as fast and efficient as possible.
Data is compressed using Quasar optimized compression algorithm, significantly reducing the disk footprint of the history.
Market data is organized in the following way: two (2) tables representing timeseries data per security, one (1) for the trades, one (1) for the orders. Using tagging, it's possible to group different tables for any query. This organization that is made possible thanks to Quasar tagging capability increases compression efficiency and significantly boosts querying speed.
Benefits
- Equity trading firms get instant vision on the order book and best execution without relying on custom development or third-party tools, increasing alpha
- All market data (history and current day) is contained in the same system, significantly reducing errors and increasing productivity and, thus, competitiveness
- Compatibility with all standard analytical tool suites for maximum flexibility
- Outstanding TCO thanks to the combination of data compression and rapid order book rebuilding, saving on disk storage and CPU power