DMax Aggressive Parallel Decoding for dLLMs paper:

AK

23,522 次观看 • 3 个月前

We are excited to introduce Mercury, the first commercial-grade... show more

Inception

1,915,058 次观看 • 1 年前

In the Mean While #Soundariya In Parallel Universe Be... show more

Joe Michael Praveen

100,519 次观看 • 1 年前

Can $300 RTX3060 be faster than $10,000 A100 for... show more

Hongyang Zhang

35,689 次观看 • 2 年前

DECODING FOR "ARIRANG" LOCKED RECORDINGS ON SPOTIFY STARTED.. (... show more

BTS Updates, News & Charts ⁷

116,299 次观看 • 4 个月前

Added context to my tiny diffusion model to enable... sequential generation of longer outputs! Currently the context is a quarter of the sequence length (seq_len=256, context_len=64). I have a theory that the less semantic-value-per-token, the worse the “curse of parallel decoding” is. With parallel decoding, we independently predict multiple tokens in one step. With the sentence “My poker hand was a ___ ___”, two valid predictions are “two pair” and “straight flush”. Because each token prediction is independent though, we can end up with a nonsensical output like “two flush”. This seems to be exacerbated with low semantic-value-per-token, as now you need more tokens to express the same concept. Instead of needing to independently predict two tokens, we might need to predict 10 instead (which is of course much harder). The model currently has noticeably worse output compared to nanogpt (similar size) and I believe this is a main reason. I’ll try adding confidence-aware parallel decoding (from NVIDIA’s Fast-dLLM paper) and other tricks and see how much they improve generation quality.show more