Meet Turbovec: A Rust Vector Index with Python Bindings, and Built on Google’s TurboQuant Algorithm

Vector search underpins most retrieval-augmented generation (RAG) pipelines. At scale, it gets expensive. Storing 10 million document embeddings in float32 consumes 31 GB of RAM. For dev teams running local or on-premise inference, that number creates real constraints.

A new open-source library called turbovec addresses this directly. It is a vector index written in Rust with Python bindings. It is built on TurboQuant, a quantization algorithm from Google Research. The same 10-million-document corpus fits in 4 GB with turbovec. On ARM hardware, search speed beats FAISS IndexPQFastScan by 12–20%.

The TurboQuant Paper

TurboQuant was introduced by Google’s research team. The Google team proposes TurboQuant as a data-oblivious quantizer. It achieves near-optimal distortion rates across all bit-widths and dimensions. It requires zero training and zero passes over the data.

Most production-grade vector quantizers, including FAISS’s Product Quantization, requires a codebook training step. You must run k-means over a representative sample of your vectors before indexing begins. If your corpus grows or shifts, you may need to retrain and rebuild the index entirely. TurboQuant skips all of that. It uses an analytical property of rotated vectors instead of a data-dependent calibration.

How turbovec Quantizes Vectors

The quantization pipeline has four steps:

(1) Each vector is normalized. The length (norm) is stripped and stored as a single float. Every vector becomes a unit direction on a high-dimensional hypersphere.

(2) A random rotation is applied. All vectors are multiplied by the same random orthogonal matrix. After rotation, each coordinate independently follows a Beta distribution. In high dimensions, this converges to Gaussian N(0, 1/d). This holds for any input data — the rotation makes the coordinate distribution predictable.

(3) Lloyd-Max scalar quantization is applied. Because the distribution is known analytically, the optimal bucket boundaries and centroids can be precomputed from the math alone. For 2-bit quantization, that means 4 buckets per coordinate. For 4-bit, it means 16 buckets. No data passes are needed.

(4) The quantized coordinates are bit-packed into bytes. A 1536-dimensional vector shrinks from 6,144 bytes in FP32 to 384 bytes at 2-bit. That is a 16x compression ratio.

At search time, the query is rotated once into the same domain. Scoring happens directly against the codebook values. The scoring kernel uses SIMD intrinsics — NEON on ARM and AVX-512BW on modern x86, with an AVX2 fallback — with nibble-split lookup tables for throughput.

TurboQuant achieves distortion within approximately 2.7x of the information-theoretic Shannon lower bound.

Recall and Speed: The Numbers

All benchmarks use 100K vectors, 1,000 queries, k=64, and report the median of 5 runs.

For recall, turbovec compares against FAISS IndexPQ (LUT256, nbits=8, float32 LUT). This is a strong baseline: FAISS uses a higher-precision LUT at scoring time and k-means++ for codebook training. Despite this, TurboQuant and FAISS are within 0–1 point at R@1 for OpenAI embeddings at d=1536 and d=3072. Both converge to 1.0 recall by k=4–8. GloVe at d=200 is harder. At that dimension, TurboQuant trails FAISS by 3–6 points at R@1, closing by k≈16–32.

On speed, ARM results (Apple M3 Max) show turbovec beating FAISS IndexPQFastScan by 12–20% across every configuration. On x86 (Intel Xeon Platinum 8481C / Sapphire Rapids, 8 vCPUs), turbovec wins every 4-bit configuration by 1–6%. It runs within ~1% of FAISS on 2-bit single-threaded. Two configurations sit slightly behind FAISS: 2-bit multi-threaded at d=1536 and d=3072. There, the inner accumulate loop is too short for unrolling amortization. FAISS’s AVX-512 VBMI path holds the edge in those two cases (2–4%).

Python API

Installation is a single command: pip install turbovec. The primary class is TurboQuantIndex, initialized with a dimension and bit width.

from turbovec import TurboQuantIndex

index = TurboQuantIndex(dim=1536, bit_width=4)
index.add(vectors)
scores, indices = index.search(query, k=10)
index.write("my_index.tq")

A second class, IdMapIndex, supports stable external uint64 IDs that survive deletes. Removal is O(1) by ID. This is useful for document stores where vectors are frequently updated or deleted.

turbovec integrates with LangChain (pip install turbovec[langchain]), LlamaIndex (pip install turbovec[llama-index]), and Haystack (pip install turbovec[haystack]). The Rust crate is available via cargo add turbovec.

Marktechpost’s Visual Explainer

# Python
pip install turbovec

# Rust
cargo add turbovec

from turbovec import TurboQuantIndex

index = TurboQuantIndex(dim=1536, bit_width=4)

# Add vectors (numpy float32 array, shape [n, dim])
index.add(vectors)
index.add(more_vectors)  # incremental adds are fine

# Search: returns top-k scores and positional indices
scores, indices = index.search(query, k=10)

import numpy as np
from turbovec import IdMapIndex

index = IdMapIndex(dim=1536, bit_width=4)

# Map vectors to your own uint64 external IDs
index.add_with_ids(vectors, np.array([1001, 1002, 1003], dtype=np.uint64))

# Search returns your external IDs, not positional indices
scores, ids = index.search(query, k=10)

# O(1) delete by external ID\nindex.remove(1002)

from turbovec import TurboQuantIndex, IdMapIndex

# TurboQuantIndex  —>  .tq
index.write("my_index.tq")
loaded = TurboQuantIndex.load("my_index.tq")

# IdMapIndex  —>  .tvim
index.write("my_index.tvim")
loaded = IdMapIndex.load("my_index.tvim")

# LangChain
pip install turbovec[langchain]

# LlamaIndex
pip install turbovec[llama-index]

# Haystack
pip install turbovec[haystack]

use turbovec::TurboQuantIndex;

let mut index = TurboQuantIndex::new(1536, 4);
index.add(&vectors);

let results = index.search(&queries, 10);

index.write("index.tv").unwrap();
let loaded = TurboQuantIndex::load("index.tv").unwrap();

Key Takeaways

• No codebook training. turbovec indexes vectors instantly — no k-means, no rebuilds as the corpus grows.
• 16x compression. A 1536-dim float32 vector shrinks from 6,144 bytes to 384 bytes at 2-bit quantization.
• Faster than FAISS on ARM. turbovec beats FAISS IndexPQFastScan by 12–20% on ARM across every configuration.
• Near-optimal distortion. TurboQuant achieves distortion within ~2.7x of the Shannon lower bound — provably near the theoretical limit.
• Fully local. No managed service, no data egress — pairs with any open-source embedding model for an air-gapped RAG stack.

Check out the Repo here. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us