Quasar June 2022 Community update
As we are getting closer to our official 3.13.4 release, we felt the moment was right to share with our community what to expect from this new beta release.
Hot backups
Let’s start with a significant new feature: hot backups. In 3.13.4, you can now backup a live node incrementally, with control over the number of revisions. Backups happen in the background, and while a node is backed up, you can continue to ingest data and run queries.
Some important points to consider with the hot backup feature:
- Backups happen in the background and will thus use some of your I/O capacity. It’s important to understand that this can impact your ingestion speed, even if backups have lower priority.
- Quasar instances can be humongous (petabytes). One does not simply back up 10 petabytes of data.
Enhanced querying capability
In this update, we’ve enhanced the capabilities of the RESTRICT TO keyword and added more flexibility to ORDERBOOK:
- RESTRICT TO can now accept more complex criteria making it very easy to handle multiple versions of the same data at a fantastic speed.
- The ORDERBOOK and SLIPPAGE functions now offer more granularity and flexibility when selecting the levels (depth) range.
We’ve also fixed some quirks of the query language to make it behave more coherently and logically. For example, the treatment of the timestamp by the “first” aggregation was surprising for people coming from SQL databases. We fixed that.
Performance
Performance is the central focus of this release. Speed matters a lot at Quasar, because our users typically have large datasets.
Based on your feedback, we’ve spent an (unreasonable) amount of time to work on several performance bottlenecks impacting queries across the board. These improvements revolve mostly around internal structure optimizations and improving the client-server communication logic to avoid unnecessary work.
These performance improvements can be drastic, queries which used to take 20ms now can take less than 1ms (20X improvement). These improvements also solve issues of specific queries taking an abnormal amount of time (sometimes a minute, even more) given their complexity.
The ORDERBOOK virtual machine, the piece of code responsible for executing raw orders to build the order book, has been reworked and we managed to get up to 100% speed improvement.
Performance isn’t just server-side but also client-side. For example, the Python API can be up to 20X faster thanks to a generalized usage of zero-copy.
Lastly, we've done several improvements that should result in an overall lower memory usage for identical loads.
Compatibility
Finally, we’ve made several quality-of-life improvements to improve compatibility of Quasar with containers and OS X.
Quasar should work on a broader range of Macs, out of the box, without needing to recompile, e.g. the Python API. If Quasar does not work correctly on your mac, please send a mail to [email protected], whether you’re a paying user or not, and we will look into the issue.
Another quality-of-life improvement we’ve made is to ensure the CPU verification code runs on a more extensive set of platforms. This way, if you run the Haswell optimized build on an older workstation, you will get a clear error message inviting you to download a different build.
Lastly, the .NET API could sometimes be unable to load, especially from a docker container. We fixed the issues that were causing that and made generally speaking the loading process more resilient.
Expected release date
We are working very hard to release Quasar 3.13.4 in June 2022.
Meanwhile, feel free to download Quasar, join our community channel on Slack, subscribe to our newsletter, or to contact us!
QuasarDB is now Quasar
I am thrilled to announce that QuasarDB has become Quasar.
If you're already a Quasar customer, this will have no impact on the product and service. You may notice that our staff will write you from our new domain: quasar.ai.
This change is to reflect our mission statement: enable AI to create wealth.
Think of AI not as Artificial Intelligence but as Automated Intelligence. AI is that software chain that refines information to present us with actionable choices.
The challenge of AI today isn't the algorithms but the impedance difference between AI and data. As your data volume grows, it becomes exponentially more challenging to leverage it accurately.
At Quasar, we believe that more data isn't a problem: it's an opportunity.
Suppose you're currently doing predictive maintenance, network analysis, or quantitative analysis. What prevents you from feeding more data into your models? How much could you improve?
Quasar is a better answer than data lakes and data warehouses.
Quasar simplifies your data engineering platform by allowing you to capture an unlimited amount of raw timeseries data and extract features on the fly.
Quasar lets your analysts work interactively and forget about data engineering issues altogether.
If your curious about what Quasar can do for your business, our new website has an extensive list of case studies.
These last years have been a crazy ride, and I'm incredibly proud of our team.
I would like to take this opportunity to thank our investors and our customers for placing their faith in us when we were a tiny company with everything to prove. Thank you!
Happy Thanksgiving!
-Edouard, founder and CEO of Quasar