CS PhD @Stanford || Ask me about NVFP4 training https://t.co/GwsXouMWXg
Shorts
🥐 Hi everyone! FlashAttention's backward pass has the same zero-sum bug Jon Barron just fixed in softmax. With FP4 stochastic rounding it's wrong on every single draw. We fixed it in our ICLR'27 submission, CroissAttn and unlock e2e FP4 training.
