inspired by SDPO, i made continualcode -- a minimal claude code that learns from your corrections in real-time, built on tinker. when you deny a diff, the model uses your correction as context to teach itself, takes a gradient step on LoRA, and retries with updated weights. claude code... but it updates the model weights!show more

I got interested in trying to figure out how many bits per episode we coudl learn in RL, after some ablations on Tinker last November I was only able to track ~0.03 due to noise On-policy distillation is interesting because it gives dense per-token supervision (many learning signals per episode), instead of a single scalar reward. SDPO is especially interesting since you can treat rich feedback as privileged context: the model-with-feedback becomes its own teacher.

I benchmarked this idea in continualcode on LiveCodeBench with Qwen3-4B and Qwen3-30B. Both show an early jump in pass@1 and then quickly plateau. My current hypothesis (despite gpt 5.2 codex being used to give rich feedback when benchmarking) is that the per token advantages are too small and the teacher only disagrees with <30% of tokens. My intuition is that since we only use feedback on denied tools it slowly generates less and might need to use GRPO to increase signal. Albeit the runs I've done are relatively "short" and will try running for longer to see if I can try fixing more stability issues. Any feedback or advice would be greatly appreciated Thanks @sunfanyun @andrewnhwang for their advice and @johnschulman2 for the tinker cookbook this was built on, and @thinkymachines @clarejtbirch for tinker!

pip install continualcode and

thanks to @rahulgs for nanocode @willdepue @radbackwards and everyone who had some part in this as well

built off of nice work. there were some hurdles porting it onto Tinker but I think it was worth it

Very interesting and timely work, but I am curious what models did you use?

qwen3 4b! you can use any model here

i didnt find a difference in mode though i wasnt able to properly stabalize SDPO

That’s what I mean, the true continual learning needs to scale with more compute. But it is hard to achieve today.

This is so sick

Nice!

Very cool

@charles_irl

clean realtime lora step on user denies unlocks tight feedback for code iteration. without persistent traces across sessions though, adapted weights vanish on restart - back to square one. anchoring those updates makes the learning stick. 🇳🇴

Fire

Updating model weights via LoRA based on rejected diffs is a genius loop for 'continualcode'. It feels like the missing piece for truly personalized coding agents. Sent you a DM about a potential partnership - would love for you to take a look!"

Faszinierend! Echtzeit-Lernen via LoRA-Updates könnte ein Gamechanger sein. Wie verhält sich das bei komplexeren Codebases? Und wie stabil sind die Gradient-Updates ohne catastrophic forgetting? #AI

Very cool! Sounds like it stops wanting to call tools? I wonder if you could only “teach” and do gradient updates for tokens in the content of the tool call/diff and the reasoning, while leaving the actual tool call decision/tokens untrained? Or you could ensure the privileged teacher calls a tool

Your approach reminds me of how I've seen language models adapt to user feedback in real-time, but the idea of incorporating corrections as context is what makes continualcode stand out. By leveraging corrections in this way, you're effectively creating an iterative feedback loop that refines the model's understanding over time. It's a clever way to bridge the gap between human evaluation and AI-driven improvements.

fire

🔥🔥🔥

this is cool

@blakeandersonw based. does it handle edge cases where corrections might conflict with previous context?

LoRA weight updates from real-time coding mistakes? Now that's how you bake continual learning directly into the workflow. No more 'train once, cry forever' models.