Google Research has published a paper introducing TurboQuant, a compression algorithm designed to reduce a key memory bottleneck in AI inference by at least six times while maintaining full accuracy. The paper is scheduled for presentation at ICLR 2026 and drew immediate attention online following its release on Wednesday. Cloudflare CEO Matthew Prince described it as Google’s DeepSeek moment, and shares in memory hardware companies including Micron, Western Digital, and Seagate declined on the same day.
TurboQuant targets the KV cache, the portion of GPU memory that stores everything a language model needs to retain during a conversation. As context windows expand toward millions of tokens, these caches can grow to hundreds of gigabytes per session. According to Google, the real bottleneck in large language model deployment is not processing power but raw memory capacity.
Conventional compression approaches shrink these caches by reducing numerical precision — moving from 32-bit floats down to 16, 8, or 4-bit integers. However, they must store additional quantization constants alongside the compressed data to preserve model performance, and those constants add one to two bits per value, partially offsetting the efficiency gains. TurboQuant claims to eliminate that overhead entirely through two component algorithms.
The first, PolarQuant, separates magnitude from direction within vectors. The second, QJL (Quantized Johnson-Lindenstrauss), reduces the small residual error that remains to a single sign bit — positive or negative — with no stored constants required. Google states the combined approach produces a mathematically unbiased estimator for the attention calculations that underpin transformer models.
In benchmarks conducted using Gemma, Mistral, and Llama, TurboQuant matched full-precision performance under four times compression, including perfect retrieval accuracy on needle-in-haystack tasks at context lengths up to 104,000 tokens. Extending a model’s usable context without degrading output quality has been one of the more persistent challenges in large language model deployment, making those results notable to researchers in the field.
The claim of zero accuracy loss, however, carries important qualifications. It applies specifically to KV cache compression during inference and not to the model’s weights, which represent a separate and more difficult compression problem that TurboQuant does not address. What the algorithm compresses is the temporary memory holding mid-session attention computations, which is considered more forgiving because that data can in principle be reconstructed.
There is also a gap between controlled benchmark conditions and large-scale production environments. TurboQuant was evaluated on open-source models rather than on Google’s own Gemini infrastructure at scale. Unlike the efficiency improvements associated with DeepSeek, which required architectural decisions made during initial training, TurboQuant requires no retraining or fine-tuning and is said to add negligible runtime overhead, meaning it could in theory be integrated directly into existing inference pipelines.
That compatibility with current hardware is what unsettled the memory sector — if the algorithm performs as described in production, major AI laboratories could run more efficiently on their existing GPU infrastructure without purchasing additional memory capacity. The paper remains a research publication ahead of ICLR 2026, and the zero-loss claim has yet to be validated outside laboratory conditions.
Originally reported by Decrypt.
