Another piece from the Sasha Rush conversation, this time on rewards for coding RL.
Ravid Shwartz Ziv
Visit on X@ziv_ravid12K followers
14,826 views
Another piece from the Sasha Rush conversation, this time on rewards for coding RL.
He said Cursor uses a mix. Some rewards look at the tool calls themselves, some only at the final output. Everything end-to-end, no process rewards guessing what happens in the middle.
I agree with him that the next push (and the current one) is going past verifiers. Coding is a good example where you get both. You have the 0/1 signals, but also a ton of soft ones. Does the diff read well? Did the agent burn context for no reason? Is the PR the right size? Did it rewrite a file it didn't need to touch? So many signals sitting there, and we need to figure out how to actually train on them.
This is also why I think the environment (and the data) is becoming the bottleneck, not the algorithm. GRPO works. PPO works. The hard part is building a sandbox that exposes enough signal to learn from. We're already seeing this, and it'll only accelerate: a whole category of companies selling environments is emerging, and they have a good reward to do so…
Comments
Comments from the original post on X (Twitter)
No comments yet
