i just beat Google DeepMind's turboquant
157.343 görüntüleme
35 Yorum
Yorumlar X (Twitter) üzerindeki orijinal gönderiden alınmıştır

@krishgarg
github:
35 beğeni

@pupposandro
@AIBuddyK @GoogleDeepMind Cool work! but nobody is using Llama-3.1-8b. Would love to see this working with Qwen3.6 MoE or 27b
19 beğeni

@no_stp_on_snek
Interesting direction. The asymmetry is the important part here, not the headline compression ratio. K and V behave fundamentally differently under attention, and treating them identically leaves performance on the table. The memory caveat matters too: a compressed cache is only half the story if decode still needs a second working copy. If you store a tiny cache but also build dense scratch/reconstruction buffers to actually use it, the real RAM bill is closer to “compressed cache + working cache,” not just the advertised stored-cache size. That is where these systems get interesting: decode-time distribution shift, materialization overhead, transient workspace growth, and attention-weighted error behavior rather than raw reconstruction MSE. Related: * Asymmetric K/V Cache Compression: * Sparse V Dequantization: * Why MSE Fails for KV Quantization:
18 beğeni

@will_ea
@GoogleDeepMind This needs a KVTC baseline
13 beğeni

@sakurayukiai
@GoogleDeepMind NIAH is basically a solved metric at this point. Run this through SWE-bench and let's see if the attention heads actually survive a 10x squeeze.
10 beğeni

@xbin12345
@GoogleDeepMind I explored this but it only works with llama? Can you get it to work with Qwen3.6 Moe because then its a Game changer!
7 beğeni

@ancrth
@GoogleDeepMind The benchmark I'd want to see: does Shard hold quality across 50-turn agent loops, not just single-shot evals? KV bloat is the actual bottleneck once tool calls + retries start stacking.
6 beğeni

@dhruvmt2
@GoogleDeepMind Yep, low rank tensors are everywhere. The tricky part has always been, as you found, the decomp kills throughput
5 beğeni

@AshleyH1656
Love what you're doing, and really enjoyed the well written write up. One thought - Turboquant is data-oblivious - Shard fits a per-layer PCA basis + DP bit allocation on prefill activations. Have you compared it to other data-dependent baselines in same regime like KVTC, AsymKV etc?
4 beğeni

@iamthemoat
@GoogleDeepMind
3 beğeni

@geekfoxcharlie
@GoogleDeepMind 10x with NIAH 1.000 is wild. Curious how Shard handles attention head heterogeneity — flat ratio across all heads feels suboptimal. Real agent workflows with tool calls stress KV patterns differently than LongBench prose. Would love multi-turn agentic benchmarks.
3 beğeni

@shamsharoonn
@GoogleDeepMind ur tuff
2 beğeni

@utosrad
@GoogleDeepMind genuinely the tuffest
2 beğeni

@SOHAMPATRA18
@GoogleDeepMind @grok how he have done this , process , and got this accurately
2 beğeni

@AgiWhen
@GoogleDeepMind @grok how efficient is this in respect to deepseek's context compression? on a scale of 1-10, two decimal digits allowed
2 beğeni

@vibecoder_dc
@GoogleDeepMind 10x compression, zero loss? Every compression paper says this on its own eval. The real test is the 'compression cliff' where NIAH fails.
2 beğeni

@AMerchantmoh
@GoogleDeepMind @TheAhmadOsman Check this out big man.
2 beğeni

@guilhermeotina
@GoogleDeepMind interesting that the compression ratio varies with context length (10x at 8K vs 11.2x at 32K). pure quantization would be constant. suggests eviction, structural pruning, or a hybrid approach. what breaks first when you push beyond 32K?
2 beğeni

@skynetislov3
@GoogleDeepMind Pretty legit. But cool kids are processing in the compressed space, too. I could teach you, but i'd have to charge. 🔔🥛
2 beğeni

@Elon_Orbit
@GoogleDeepMind Need the repo. 10x KV compression at zero loss? Big claim. DeepMind didn't ship turboquant overnight. Show the work.
2 beğeni













