Compress 10 million 768-dimensional embeddings into 4GB and search 4.4 times faster than FAISS
Large-scale embedding data that previously required tens of gigabytes of memory can now be reduced to 4GB while improving search speed. turbovec is an in-process index library that stores vectors compressed to 2-bit or 4-bit and searches them in that compressed form. Vectors added to the index are immediately searchable without a training phase, and because data does not leave the process, it can be used in offline environments.
The principle of searching compressed data directly
turbovec compresses vectors to 2-bit or 4-bit based on the TurboQuant algorithm from Google Research. It searches the compressed data directly using SIMD instructions, eliminating the need for a decompression step. Vectors become searchable immediately upon addition, and deletions are handled in microseconds. Calling sync(path) writes only the changed parts to the file, ensuring crash safety. Both Python and Rust provide the same API, and integration packages are supported for major frameworks such as LangChain, LlamaIndex, Haystack, and Agno.
Performance figures confirmed by benchmarks
Benchmarks compared turbovec against FAISS using 1 million OpenAI embeddings (d=1536, d=3072) and GloVe (d=200) data. With 4-bit compression, search speed is 4.0 times faster on ARM and 4.9 times faster on x86, while 2-bit compression is 2.05 times and 2.35 times faster respectively. Memory usage is 7.8 times lower than float32 for 4-bit and 15.5 times lower for 2-bit. Accuracy (Remember@1) remains comparable to FAISS, ranging from 0.959 to 0.981 for 4-bit and 0.901 to 0.931 for 2-bit. Specifically, with 1 million vectors, latency was recorded at 0.13 to 0.25 ms per query, a figure achieved when using all 8 cores.
Limitations to check before applying
Because turbovec scans all vectors, HNSW or IVF indexes are more suitable for datasets with hundreds of millions of vectors where strict latency limits are required. Accuracy may drop due to quantization; while OpenAI data shows 96 to 98% accuracy with 4-bit compression, lower-dimensional embeddings may suffer greater loss. As a pure library without network APIs or replication features, it must be combined with other stacks if server functionality is needed. Vector dimensions must be a multiple of 8, with support up to 16,384 dimensions. It is important to verify accuracy in advance using your own data samples.
