Quasar June 2022 Community update

As we are getting closer to our official 3.13.4 release, we felt the moment was right to share with our community what to expect from this new beta release.

Hot backups

Let’s start with a significant new feature: hot backups. In 3.13.4, you can now backup a live node incrementally, with control over the number of revisions. Backups happen in the background, and while a node is backed up, you can continue to ingest data and run queries.

Some important points to consider with the hot backup feature:

• Backups happen in the background and will thus use some of your I/O capacity. It’s important to understand that this can impact your ingestion speed, even if backups have lower priority.
• Quasar instances can be humongous (petabytes). One does not simply back up 10 petabytes of data.

Enhanced querying capability

In this update, we’ve enhanced the capabilities of the RESTRICT TO keyword and added more flexibility to ORDERBOOK:

• RESTRICT TO can now accept more complex criteria making it very easy to handle multiple versions of the same data at a fantastic speed.
• The ORDERBOOK and SLIPPAGE functions now offer more granularity and flexibility when selecting the levels (depth) range.

We’ve also fixed some quirks of the query language to make it behave more coherently and logically. For example, the treatment of the timestamp by the “first” aggregation was surprising for people coming from SQL databases. We fixed that.

Performance

Performance is the central focus of this release. Speed matters a lot at Quasar,  because our users typically have large datasets.

Based on your feedback, we’ve spent an (unreasonable) amount of time to work on several performance bottlenecks impacting queries across the board. These improvements revolve mostly around internal structure optimizations and improving the client-server communication logic to avoid unnecessary work.

These performance improvements can be drastic, queries which used to take 20ms now can take less than 1ms (20X improvement). These improvements also solve issues of specific queries taking an abnormal amount of time (sometimes a minute, even more) given their complexity.

The ORDERBOOK virtual machine, the piece of code responsible for executing raw orders to build the order book, has been reworked and we managed to get up to 100% speed improvement.

Performance isn’t just server-side but also client-side. For example, the Python API can be up to 20X faster thanks to a generalized usage of zero-copy.

Lastly, we’ve done several improvements that should result in an overall lower memory usage for identical loads.

Compatibility

Finally, we’ve made several quality-of-life improvements to improve compatibility of Quasar with containers and OS X.

Quasar should work on a broader range of Macs, out of the box, without needing to recompile, e.g. the Python API. If Quasar does not work correctly on your mac, please send a mail to [email protected], whether you’re a paying user or not, and we will look into the issue.

Another quality-of-life improvement we’ve made is to ensure the CPU verification code runs on a more extensive set of platforms. This way, if you run the Haswell optimized build on an older workstation, you will get a clear error message inviting you to download a different build.

Lastly, the .NET API could sometimes be unable to load, especially from a docker container. We fixed the issues that were causing that and made generally speaking the loading process more resilient.

Expected release date

We are working very hard to release Quasar 3.13.4 in June 2022.

Meanwhile, feel free to download Quasar, join our community channel on Slack, subscribe to our newsletter, or to contact us!