Quasar "Seneca" 3.13.3 Beta released

This is the beta 3.13 branch, not to be confused with the stable 3.10 branch.

We are very happy to announce the immediate availability of QuasarDB 3.13.3, a beta version update from 3.13.2.

In this release we have focused on enhancing the analytical power of Quasar, to support complex use case such as bitemporal timeseries data, orderbook slippage computation, new indicators, enhanced order book syntax, and suffix based search.

Get QuasarDB 3.13.3 here!

QuasarDB is also available on the AWS marketplace.

Note: 3.13.3 is not wire compatible with 3.13.2.

32-bit and 64-bit ARM binaries for Linux are now included with every release.

Change log

  • Protocol version 48
  • [daemon] Add built-in automatic version check
  • [daemon] Improve performance for very short lived queries
  • [nodejs] Fix several stability issues in the nodejs API
  • [protocol] Improve backward compatibility with new versioning logic
  • [query] Add support for ARRAY_ACCUM
  • [query] Add support for AS_TABLES (virtual tables built on keys)
  • [query] Add support for Coinbase market data in the ORDERBOOK function
  • [query] Add support for Relative Strength Index (RSI)
  • [query] Add support for RESTRICT TO (multi version management)
  • [query] Add support for SLIPPAGE (order book slippage computation)
  • [query] Add support for suffix in FIND
  • [query] Add support for TOP and FULL computation in the ORDERBOOK
  • [query] Fix several minor issues related to moving averages and exponential moving averages
  • [query] Fix subquery bug when parent and child share selected columns
  • [query] Changed the syntax of the ORDERBOOK function

by Edouard Alligand

Quasar "Seneca" 3.13.2 Beta Released

This is the beta 3.13 branch, not to be confused with the stable 3.10 branch.

We are very happy to announce the immediate availability of QuasarDB 3.13.2, a beta version update from 3.13.1.

This release brings important performance improvements across the board, and bring major new features such as a brand new low-latency, high-performance, cryptocurrencies exchange feed handler, support for EMA and MACD, and support for Apache Arrow as input and output.

Last but not least, real-time order book reconstruction is now out of private beta and available in this release, directly from the query language.

Get QuasarDB 3.13.2 here!

QuasarDB is also available on the AWS marketplace.

32-bit and 64-bit ARM binaries for Linux are now included with every release.

Change log

  • Protocol version 46
  • [api] Query results can optionally return Apache Arrow structures
  • [api] Remove deprecated bulk inserter
  • [api] Simplify symbol API
  • [api] The batch inserter can optionally accept Apache Arrow structures for input
  • [client] Greatly improve multithreading performance
  • [crypto] Low latency crypto feed handler with native Coinbase support
  • [kernel] Greatly improve cache heuristics for lower I/O usage
  • [query] Add support for "infinite" shard size
  • [query] Add support for escaped characters in quoted strings
  • [query] Add support for Exponential Moving Average (EMA)
  • [query] Add support for Moving Average Convergence Difference (MACD)
  • [query] Add support for ORDERBOOK for real-time order book reconstruction
  • [query] Fix a bug where GROUP BY + WHERE could incorrectly split a range
  • [query] Fix an edge case with last aggregations on certain GROUP BY queries
  • [query] Greatly improve the performance of DELETE WHERE DUPLICATE
  • [query] Greatly increased WHERE performance across the board
  • [security] Add support for transitive permissions on tags
  • [shell] Add a command to reload the user configuration manually

by Edouard Alligand

Quasar "Seneca" 3.13.1 Beta Released

This is the beta 3.13 branch, not to be confused with the stable 3.10 branch.

We are very happy to announce the immediate availability of QuasarDB 3.13.1, a beta version update from 3.13.0.

This version includes two important bug fixes. If you are using the new symbol column type or the Java API you are strongly encouraged to upgrade to 3.13.1.

Get QuasarDB 3.13.1 here!

QuasarDB is also available on the AWS marketplace.

32-bit and 64-bit ARM binaries for Linux are now included with every release.

Change log

  • Protocol version 46
  • [api] Fix a client-side data corruption when pushing multiple symbol columns at the same time
  • [api] Fix an issue with symbol and batch API parameters
  • [daemon] Fix a performance issue with ALTER TABLE ADD
  • [daemon] Properly take hierarchical cgroup memory limits into account
  • [java] Fix several stability issues
  • [query] Add support for ALTER TABLE DROP column
  • [query] Add support for ASOF INTERPOLATE and LOOKBACK
  • [query] Add support for direction for ASOF LOOKBACK
  • [query] Full support for moving averages

by Edouard Alligand

Quasar 3.10.1 "Marcus Aurelius" Stable Released

This is the stable 3.10 branch, not to be confused with the 3.13 branch.

We are very happy to announce the immediate availability of QuasarDB 3.10.1, a stable version update from 3.10.0.

This update features improvements to the ASOF JOINS capabilities as well as several bug fixes. We also updated the .NET API with the latest batch writer.

Get QuasarDB 3.10.1 here!

QuasarDB is also available on the AWS marketplace.

32-bit and 64-bit ARM binaries for Linux are now included with every release.

Change log

(w.r.t. 3.10.0)

  • Protocol version 37
  • [daemon] Add support for cgroups v2
  • [daemon] Add support for rate limit
  • [daemon] Add support for user dynamic configuration reloading and update
  • [daemon] Fix rare permission error in transactions
  • [net] Add support for the new batch API
  • [net] Fix several stability issues
  • [query] Accept ip() in INSERT/UPDATE
  • [query] Add support for % operator
  • [query] Add support for ASOF INTERPOLATE and LOOKBACK
  • [query] Add support for DATEPART
  • [query] Add support for TRIM TABLE
  • [query] Fix LOOKUP error on aggregation
  • [query] Fix several ASOF + GROUP BY errors
  • [query] Improve cast syntax flexibility

by Edouard Alligand

Quasar "Seneca" 3.13.0 Beta Released

Where is 3.11?

You may have noticed that we skipped the 3.11 (beta) and 3.12 (stable) releases altogether. We took the decision to kill a branch of Quasar and work on a major improvement of the I/O engine. Since there were internal releases named 3.11, to avoid any confusion we decided to number the official new beta release 3.13.

New "Seneca" QuasarDB engine

This new beta branch introduces a new version of the QuasarDB engine.

The new engine, named after the Roman stoic philosopher Seneca the Younger, is the result of one year of intense R&D focused on greatly optimizing I/O for large (beyond 100 TiB) data sets. Rest assured that smaller data sets will also greatly benefit from these improvements!

We have analyzed hundreds of petabytes of data sets coming from our customers to flesh out a new data organization and improve our stochastic indexes to reduce read and write amplification (up to 20 times!).

These changes mean that 3.13 is not backward compatible with 3.10 or 3.9 and a data migration is required to upgrade.

We have not just improved persistence, with this release we are introducing three major new features:

  1. Continuous queries - Queries results can now be pushed from the server to the client. This is more than a convenience: polling the server for updates can be extremely inefficient.
  2. Symbol tables - Native support for "infinite" enumerations. Data is stored as integers, but the user is presented with strings. This greatly improves disk usage and query performance. All the symbol table upkeeping is done transparently and the load is distributed over the cluster.
  3. Sub queries - We are working on greatly extending the capabilities of the query language and have introduced sub queries in this release.

This is the first release of this new engine. Expect more from the Quasar team over the year 2022: we aim at nothing less than the impossible.

Get QuasarDB 3.13.0 here!

32-bit and 64-bit ARM binaries for Linux are now included with every release.

Change log

  • Protocol version 45
  • [daemon] Add support for cgroups v2
  • [daemon] Add support for user configuration remote update
  • [daemon] Upgraded to RocksDB 6.14.6
  • [kernel] Add more statistics for improved monitoring
  • [kernel] Add support for continuous queries (e.g. push notifications)
  • [kernel] Add support for priorities on asynchronous queues (disabled by default)
  • [kernel] Greatly improved ingestion rate for asynchronous writes (up to 10X)
  • [kernel] Native support for "symbols" column type (e.g. "infinite" enums)
  • [net] Add support for the new batch API
  • [odbc] Improve ODBC compatibility
  • [persistence] Improve compression, up to double the efficiency!
  • [persistence] Improve tiered storage support
  • [persistence] New indexes and data layout which can deliver up to 20X speed improvement on cold data
  • [query] Add support for DATEPART
  • [query] Add support for TRIM TABLE
  • [query] Improve DELETE speed
  • [query] Improve disk size computation accuracy
  • [query] Support for subqueries

by Edouard Alligand

QuasarDB 3.10.0 Stable Released

We are very pleased to announce the immediate availability of QuasarDB 3.10.0. You can get it here.

The previous stable version, 3.8.0, was released on April 6th, 2020, almost 18 months ago!

3.10 has been battle-tested on multi-petabytes datasets with continuous data ingestions over several months.

3.10 is our most resilient, fastest, and most efficient release ever.

Compared to 3.8, some queries run several orders of magnitude faster, and memory usage is massively reduced. We also greatly increased the write resiliency, for example in the case of a brutal shutdown.

This release does not just improve performance. It also delivers on powerful new querying capabilities, enabling you to aggregate, pivot, and join data in the most flexible way possible using our tagging  logic.

3.10 is a direct branch of 3.9.9 with the improvements listed below. Switching to a stable release means we will now be much more conservative in the future 3.10.x releases, focusing on API stability and bug fixes.

If you want to have the new shiny and powerful features (such as continuous queries, symbol table, etc.), wait for the 3.13 release. We expect to release 3.13 later this month. 

Where is 3.11? Let's just say it's now in a better place.

Important: the default binaries are optimized for Haswell architectures. You will need to use the Core2 builds if your architecture is older than Haswell.

Fixes and improvements

(w.r.t 3.9.9)

  • Protocol version 37
  • [daemon] Add statistics for slow operations
  • [daemon] Change the configuration of sessions and multithreading to improve clarity
  • [daemon] Fix "alias not found" bug after changing a table schema
  • [daemon] Fix data duplication bug
  • [daemon] Fix DELETE ... WHERE DUPLICATE performance bug
  • [daemon] Greatly improve statistics collection performance
  • [daemon] Overall query performance improvement and memory usage
  • [daemon] Properly honor the "disable WAL" RocksDB setting
  • [daemon] Slight compression performance upgrade, both in speed and ratio
  • [persistence] Added more configuration parameters
  • [query] Add LIMIT to SHOW TABLES
  • [query] Add support for slope
  • [query] Add support for sum of absolute changes and sum of absolute values
  • [query] Extend GROUP BY capabilities
  • [query] Fix LOOKUP parsing ambiguity
  • [query] Greatly improve qualified FIND performance
  • [query] Improve FIND() capabilities
  • [query] New, zero copy, ultra-high-performance, batch insert API (the old API is still available but will be deprecated in the next stable release)
  • [shell] Correctly parse timeouts with the units
  • [shell] Do not print footer in CSV mode
  • [shell] Fix column layout print issues

by Edouard Alligand