ITCH Data: Order Book building

Context

Market data mainly consists of orders and trades.

The order book is necessary to evaluate how orders may be filled and is a powerful tool to identify actors’ intentions.

On top of that, reconciling orders with trades is a legal requirement: actors need to conform to best execution.

The order book is the list of buys and sells organized by price levels. It provides valuable trading information, which helps traders and improves market transparency.

In a nutshell, what Quasar delivered

• Deployment of complete Nasdaq market data capture, from start to finish, in less than a week
• Instant order book rebuilding at any point in the day while using less cloud resources
• Storage footprint divided by 10
• Future proof solution thanks to built-in scalability

Challenge

Nasdaq enables subscribers to track the status of each order from the first time it is entered until it is either executed or canceled via the ITCH protocol. To properly participate, one thus needs software capable of decoding the protocol.

Once the data is decoded, one needs to store that data in a form that can be later processed to rebuild the order book, which means processing every order since the opening of the markets.

Lastly, it is necessary to match orders with executed trade to ensure that best execution has been respected. This means parsing every order and finding the best match at the time of the trade.

Nasdaq data volumes are in the region of 2 TB per day.

These volumes make it impossible to rely on standard data warehousing technologies.

Typically, firms will store data in files processed by custom-made software for historical data analysis and keep one or two days of data in a dedicated operational database.

The solution

Quasar has an ITCH data importer that can decode and capture the data at very high speed without any intermediary conversion. This ensures data capture is as fast and efficient as possible.

Data is compressed using Quasar optimized compression algorithm, significantly reducing the disk footprint of the history.

Market data is organized in the following way: two (2) tables representing timeseries data per security, one (1) for the trades, one (1) for the orders. Using tagging, it’s possible to group different tables for any query. This organization that is made possible thanks to Quasar tagging capability increases compression efficiency and significantly boosts querying speed.

Benefits

• Equity trading firms get instant vision on the order book and best execution without relying on custom development or third-party tools, increasing alpha
• All market data (history and current day) is contained in the same system, significantly reducing errors and increasing productivity and, thus, competitiveness
• Compatibility with all standard analytical tool suites for maximum flexibility
• Outstanding TCO thanks to the combination of data compression and rapid order book rebuilding, saving on disk storage and CPU power