Thursday June 23, 2022
As we are getting closer to our official 3.13.4 release, we felt the moment was right to share with our community what to expect from this new beta release.
Let’s start with a significant new feature: hot backups. In 3.13.4, you can now backup a live node incrementally, with control over the number of revisions. Backups happen in the background, and while a node is backed up, you can continue to ingest data and run queries.
Some important points to consider with the hot backup feature:
In this update, we’ve enhanced the capabilities of the RESTRICT TO keyword and added more flexibility to ORDERBOOK:
We’ve also fixed some quirks of the query language to make it behave more coherently and logically. For example, the treatment of the timestamp by the “first” aggregation was surprising for people coming from SQL databases. We fixed that.
Performance is the central focus of this release. Speed matters a lot at Quasar, because our users typically have large datasets.
Based on your feedback, we’ve spent an (unreasonable) amount of time to work on several performance bottlenecks impacting queries across the board. These improvements revolve mostly around internal structure optimizations and improving the client-server communication logic to avoid unnecessary work.
These performance improvements can be drastic, queries which used to take 20ms now can take less than 1ms (20X improvement). These improvements also solve issues of specific queries taking an abnormal amount of time (sometimes a minute, even more) given their complexity.
The ORDERBOOK virtual machine, the piece of code responsible for executing raw orders to build the order book, has been reworked and we managed to get up to 100% speed improvement.
Performance isn’t just server-side but also client-side. For example, the Python API can be up to 20X faster thanks to a generalized usage of zero-copy.
Lastly, we’ve done several improvements that should result in an overall lower memory usage for identical loads.
Finally, we’ve made several quality-of-life improvements to improve compatibility of Quasar with containers and OS X.
Quasar should work on a broader range of Macs, out of the box, without needing to recompile, e.g. the Python API. If Quasar does not work correctly on your mac, please send a mail to [email protected], whether you’re a paying user or not, and we will look into the issue.
Another quality-of-life improvement we’ve made is to ensure the CPU verification code runs on a more extensive set of platforms. This way, if you run the Haswell optimized build on an older workstation, you will get a clear error message inviting you to download a different build.
Lastly, the .NET API could sometimes be unable to load, especially from a docker container. We fixed the issues that were causing that and made generally speaking the loading process more resilient.
We are working very hard to release Quasar 3.13.4 in June 2022.