正在加载视频...
视频加载失败
Performant reinforcement learning is a hard infrastructure and research problem. We’ve recently made on-demand multi-node clusters available and showed how to take advantageous of heterogeneous sampling compute. But there still exists the need for high-fidelity environments, clear and robust reward functions, and tuned training recipes that actually work. Most... show more
12,904 次观看 • 18 天前 •via X (Twitter)
9 条评论
Axiom Bio trains large open models to identify molecules from mass spectrometry data, which requires doing long-context RL. With our training library, they had full control to quickly conduct 64k-context DeepSeek V4.1 Flash runs on 64 B300s.
@AfterQuery curates long horizon data from domain experts for models to be trained on. Using our training library, they were able to kick off autoresearch for models like Kimi and DeepSeek at 256k context length on their existing datasets.
Read more at
love seeing this, post training is turning into a product instead of a research project
Can the training/eval split keep whole customers or workflows out, rather than random steps from the same traces? I’d want to see whether gains survive on an unseen workflow before treating higher reward as better production performance.
For a small team, “own your intelligence” still needs a break-even number: training + failed runs + data cleanup, spread over expected traffic. Cheaper inference can take a while to pay that back. Would love to see that alongside the reward curves.
@andrewdsouza modal's dojo trains llms for ai teams we know a team flagging reward hacking mid training
Long-context runs make debugging reward changes expensive. Saving the environment version, reward code and representative rollout traces with each checkpoint would let teams explain why a score moved, rather than only seeing that it did.
you list clear reward functions as one of the hard parts and the library ships tuned defaults. how does it help when a team's reward function is the thing that's wrong?
!["One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals. And you look at the evals and you go, 'Those are pretty hard evals.' But the economic impact seems to be dramatically behind. There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data. When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.' You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?' If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"](https://image.24vids.com/tw-1993450075616690474/amplify_video_thumb/1993449024045932544/img/W-1TjOXcXeiUbHaH.jpg)
![Catherine Austin Fitts argues that Donald Trump was installed by bankers to construct a digital control grid comprised of programmable money, digital ID, and a social credit system. She maintains that Trump is expendable now that the infrastructure for this system is complete. "So let's look at the control grid. There are three general baskets of what you need to do. The first is programmable money. The second is digital ID... And then the third is you need the hardware and the software infrastructure to do the social credit system." "And so you need all the data centres... But you also need a surveillance infrastructure and enforcement infrastructure to back it." "So how do you persuade conservatives... to embrace building that infrastructure? The way you do it is you say, oh, we have election fraud, or oh, we have an immigration problem. And so we need a digital ID to identify who everybody is so we can't have fraud." "If you look at everything [Trump has] done to build the control grid, or to put the conditions in place to do it, they now have what they need." "Trump is basically expendable now."](https://image.24vids.com/tw-2044699385519386898/amplify_video_thumb/2044699307027185664/img/hyVp_n0t-nVrPCBk.jpg)
