Skip navigation
Another piece from the Sasha Rush conversation, this time on rewards for coding RL.
0:00 / 2:00

Another piece from the Sasha Rush conversation, this time on rewards for coding RL.

Ravid Shwartz Ziv

@ziv_ravid12K followers

Visit on X
102 likesDownloadOriginal post

14,826 views

Another piece from the Sasha Rush conversation, this time on rewards for coding RL. He said Cursor uses a mix. Some rewards look at the tool calls themselves, some only at the final output. Everything end-to-end, no process rewards guessing what happens in the middle. I agree with him that the next push (and the current one) is going past verifiers. Coding is a good example where you get both. You have the 0/1 signals, but also a ton of soft ones. Does the diff read well? Did the agent burn context for no reason? Is the PR the right size? Did it rewrite a file it didn't need to touch? So many signals sitting there, and we need to figure out how to actually train on them. This is also why I think the environment (and the data) is becoming the bottleneck, not the algorithm. GRPO works. PPO works. The hard part is building a sandbox that exposes enough signal to learn from. We're already seeing this, and it'll only accelerate: a whole category of companies selling environments is emerging, and they have a good reward to do so…

Comments

Comments from the original post on X (Twitter)

No comments yet