Zero-Copy Integration: How Quasar Speaks Native in Every Language
When databases claim to offer “native language APIs,” what that often means in practice is yet another translation layer. Data is serialized, deserialized, copied, and converted across abstraction boundaries. While this keeps APIs user-friendly, it introduces a silent performance killer — memory copies, unnecessary conversions, and runtime overhead.
At Quasar, we designed our client APIs differently. Whether working in Python, Java, Go, .NET, or C, your application talks directly to the same high-performance native engine. That means fewer layers, less overhead, and — critically — zero-copy integration wherever possible.
This article explains how our architecture enables native-speed performance across languages and how we leverage each ecosystem’s capabilities to avoid unnecessary work. We’ll then use Python as a case study to show what this means in practice.
Most Client Libraries Are Just Protocol Wrappers
Traditional database APIs are usually written in the target language — a Java client written in Java, a Python library written in Python, and so on. These libraries are responsible for:
- Implementing the database’s wire protocol
- Managing connections and retries
- Serializing and deserializing data to and from the database’s internal format
- Translating between native language types and wire formats
This design works, but it comes with hidden costs:
- Multiple memory copies when converting between language-managed and native types
- Format conversions that often require full data traversal
- Loss of efficiency when data transformations happen in the high-level language rather than at the system level
Each step introduces latency and burns CPU cycles — especially at scale.
Quasar’s Approach: One Native Core, Many Smart Bridges
Instead of rewriting our protocol in every language, Quasar exposes a high-performance C API — the same API used internally by the database engine. This C API is not a dumb client: it contains logic to understand cluster topology, distribute requests, and even execute parts of operations locally when it makes sense.
All other language APIs — including Python, Java, Go, and .NET — are thin bindings built on this shared native layer. We use each language’s foreign function interface (FFI) to bridge between the runtime and our C API.
This means:
- There’s a single, consistent core of client logic across all languages
- We avoid reimplementing protocol details repeatedly
- Language bindings can remain lightweight, robust, and deeply optimized
And most importantly, we get native performance characteristics without compromising developer ergonomics.
Case Study: Native-Speed Python with NumPy
Python is often seen as slow, but it doesn’t have to be. Thanks to NumPy, developers can work with raw, native memory buffers in a high-level language. Quasar’s Python API takes full advantage of this.
Native Memory, Zero Copies
NumPy arrays are off-heap: their memory is allocated outside the Python interpreter, and they store data in tightly packed C-style arrays. That makes them ideal for zero-copy integration — if the API handles them correctly.
When you pass a NumPy array to Quasar’s Python API, we:
- Extract a direct pointer to the memory buffer
- Bypass serialization and Python object conversion
- Feed the pointer directly into our C API
If the NumPy array’s data type matches the target column in Quasar, no conversion occurs. This means we can insert millions of values per second — directly from Python — without duplicating or transforming your data.
Fast Conversion When Needed
If a data type mismatch exists (e.g. inserting an int64 array into a column of doubles), the conversion is performed in native C++ code using SIMD vectorized instructions. These conversions are still orders of magnitude faster than Python-based casting and happen without allocating intermediate buffers. Even here, the Python runtime stays out of the way.
Queries Work the Same Way
Zero-copy doesn’t just apply to ingestion. When you query data from Quasar, we construct NumPy arrays directly from the native memory backing the query result. That means you get the data in a format ready for analysis — with no translation steps or copies in between.
Designed for Every Language
Python is just one example. Our philosophy applies across the board:
- In Java, we use the Java Native Interface (JNI) to access the same zero-copy principles, allocating direct byte buffers when possible.
- In .NET, we use span-friendly memory models to achieve the same effect.
- In Go, we use cgo with strict control over memory ownership and pointer lifetimes.
- In C/C++, you talk directly to the native API — nothing is in the way.
In every case, Quasar ensures that the data you pass is not unnecessarily copied, reshaped, or repackaged — provided you use compatible types and layouts.
Why This Matters
Zero-copy isn’t just a buzzword — it has a measurable impact:
- Lower memory usage, especially for large datasets
- Reduced latency for inserts and reads
- Faster data pipelines with less CPU pressure
- Better performance predictability at scale
But perhaps the most compelling reason is this: it lets developers work in high-level languages like Python or Java without giving up native-level performance.
Conclusion
The fastest database operations are the ones that don’t touch your data more than once. By deeply integrating with language runtimes through their native FFI layers and offloading critical data handling to our shared C core, Quasar delivers actual zero-copy performance — across all supported languages.
If you’re working in Python and already use NumPy, you’re just a pointer away from native ingestion speed. And if you’re building systems in Java, .NET, or Go, you’ll find the same design philosophy throughout our APIs.
Ultimately, it’s not about writing less code but doing less work.
If you want to dive deeper into Quasar’s capabilities, download our technical whitepaper: Meet Quasar.