i just beat Google DeepMind's turboquant
@krishgarg1,8 тыс. подписчиков
157 343 просмотра
35 комментариев
Комментарии из оригинального поста в X (Twitter)

@krishgarg
github:
35 отметок «Нравится»

@pupposandro
@AIBuddyK @GoogleDeepMind Cool work! but nobody is using Llama-3.1-8b. Would love to see this working with Qwen3.6 MoE or 27b
19 отметок «Нравится»

@no_stp_on_snek
Interesting direction. The asymmetry is the important part here, not the headline compression ratio. K and V behave fundamentally differently under attention, and treating them identically leaves performance on the table. The memory caveat matters too: a compressed cache is only half the story if decode still needs a second working copy. If you store a tiny cache but also build dense scratch/reconstruction buffers to actually use it, the real RAM bill is closer to “compressed cache + working cache,” not just the advertised stored-cache size. That is where these systems get interesting: decode-time distribution shift, materialization overhead, transient workspace growth, and attention-weighted error behavior rather than raw reconstruction MSE. Related: * Asymmetric K/V Cache Compression: * Sparse V Dequantization: * Why MSE Fails for KV Quantization:
18 отметок «Нравится»

@will_ea
@GoogleDeepMind This needs a KVTC baseline
13 отметок «Нравится»

@sakurayukiai
@GoogleDeepMind NIAH is basically a solved metric at this point. Run this through SWE-bench and let's see if the attention heads actually survive a 10x squeeze.
10 отметок «Нравится»

@xbin12345
@GoogleDeepMind I explored this but it only works with llama? Can you get it to work with Qwen3.6 Moe because then its a Game changer!
7 отметок «Нравится»

@ancrth
@GoogleDeepMind The benchmark I'd want to see: does Shard hold quality across 50-turn agent loops, not just single-shot evals? KV bloat is the actual bottleneck once tool calls + retries start stacking.
6 отметок «Нравится»

@dhruvmt2
@GoogleDeepMind Yep, low rank tensors are everywhere. The tricky part has always been, as you found, the decomp kills throughput
5 отметок «Нравится»

@AshleyH1656
Love what you're doing, and really enjoyed the well written write up. One thought - Turboquant is data-oblivious - Shard fits a per-layer PCA basis + DP bit allocation on prefill activations. Have you compared it to other data-dependent baselines in same regime like KVTC, AsymKV etc?
4 отметки «Нравится»

@iamthemoat
@GoogleDeepMind
3 отметки «Нравится»

@geekfoxcharlie
@GoogleDeepMind 10x with NIAH 1.000 is wild. Curious how Shard handles attention head heterogeneity — flat ratio across all heads feels suboptimal. Real agent workflows with tool calls stress KV patterns differently than LongBench prose. Would love multi-turn agentic benchmarks.
3 отметки «Нравится»

@shamsharoonn
@GoogleDeepMind ur tuff
2 отметки «Нравится»

@utosrad
@GoogleDeepMind genuinely the tuffest
2 отметки «Нравится»

@SOHAMPATRA18
@GoogleDeepMind @grok how he have done this , process , and got this accurately
2 отметки «Нравится»

@AgiWhen
@GoogleDeepMind @grok how efficient is this in respect to deepseek's context compression? on a scale of 1-10, two decimal digits allowed
2 отметки «Нравится»

@vibecoder_dc
@GoogleDeepMind 10x compression, zero loss? Every compression paper says this on its own eval. The real test is the 'compression cliff' where NIAH fails.
2 отметки «Нравится»

@AMerchantmoh
@GoogleDeepMind @TheAhmadOsman Check this out big man.
2 отметки «Нравится»

@guilhermeotina
@GoogleDeepMind interesting that the compression ratio varies with context length (10x at 8K vs 11.2x at 32K). pure quantization would be constant. suggests eviction, structural pruning, or a hybrid approach. what breaks first when you push beyond 32K?
2 отметки «Нравится»

@skynetislov3
@GoogleDeepMind Pretty legit. But cool kids are processing in the compressed space, too. I could teach you, but i'd have to charge. 🔔🥛
2 отметки «Нравится»

@Elon_Orbit
@GoogleDeepMind Need the repo. 10x KV compression at zero loss? Big claim. DeepMind didn't ship turboquant overnight. Show the work.
2 отметки «Нравится»













