The KV Cache Compression Race: TurboQuant vs OSCAR vs EpiCache

Long-context language models face a memory bottleneck where cached key and value vectors grow so large they can exceed the model's own weight footprint, making decoding slow and expensive. Recent compression methods fall into five families: token eviction, quantization, low-rank projection, merging, and architectural sharing, with recent work focusing on ultra-low-bit quantization to shrink these caches. Three competing approaches tackle different aspects of this problem: one uses theoretical optimization without examining data, another uses calibration and attention-aware techniques optimized for deployment, and a third addresses the specific challenge of multi-turn conversations rather than single long contexts. Each method represents different tradeoffs between generality, theoretical guarantees, and practical production readiness.