正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Introducing INTELLECT-3: Scaling RL to a 100B+ MoE model on our end-to-end stack Achieving state-of-the-art performance for its size across math, code and reasoning Built using the same tools we put in your hands, from environments & evals, RL frameworks, sandboxes & more

Prime Intellect

84,026 subscribers

1,149,660 次观看 • 9 个月前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

🆕Daytona’s Agent-Native Compute: 60ms sandboxes, 50K startups in 75 sec, 850K daily runs, RL/evals, CLI > MCP, & the end of localhost Daytona CEO Ivan Burazin explains why AI agents need composable computers, how Daytona pivoted from human dev environments to agent sandboxes, why bare metal and stateful snapshots matter, how RL workloads went from 0% to ~50% of usage, why Kubernetes breaks down at agent scale, and why the AI cloud may look more like Stripe than AWS.

🆕Daytona’s Agent-Native Compute: 60ms sandboxes, 50K startups in 75 sec, 850K daily runs, RL/evals, CLI > MCP, & the end of localhost Daytona CEO Ivan Burazin explains why AI agents need composable computers, how Daytona pivoted from human dev environments to agent sandboxes, why bare metal and stateful snapshots matter, how RL workloads went from 0% to ~50% of usage, why Kubernetes breaks down at agent scale, and why the AI cloud may look more like Stripe than AWS.

Latent.Space

12,972 次观看 • 3 个月前

Hiring RL Engineer! Started off as a curious project at Lossfunk to push the boundaries of LLMs in social reasoning - we are now building RL environments, data, and benchmarks to simulate more real-world scenarios. If you want to train SoTA RL models over multi-GPUs (H200s/B200s) to unlock next AI frontier, this is for you.

Hiring RL Engineer! Started off as a curious project at Lossfunk to push the boundaries of LLMs in social reasoning - we are now building RL environments, data, and benchmarks to simulate more real-world scenarios. If you want to train SoTA RL models over multi-GPUs (H200s/B200s) to unlock next AI frontier, this is for you.

Satpal Singh Rathore

46,010 次观看 • 1 年前

Check out our #ICRA2024 paper "Actor-Critic Model Predictive Control." Model-free #reinforcementlearning (RL) is known for its strong task performance and flexibility in optimizing general reward formulations. On the other hand, #ModelPredictiveControl (MPC) benefits from robustness and online replanning capabilities. We combine both approaches by introducing a new framework called Actor-Critic Model Predictive Control. The key idea is to embed a differentiable MPC within an Actor-Critic RL framework. The proposed approach leverages the short-term predictive optimization capabilities of MPC with the exploratory and end-to-end training properties of RL. The resulting policy effectively manages both short-term decisions through the MPC-based actor and long-term prediction via the critic network, unifying the benefits of both model-based control and end-to-end learning. We validate our method in simulation and the real world with a quadcopter across various high-level tasks. We show that the proposed architecture can achieve real-time control performance, learn complex behaviors via trial and error, and retain the predictive properties of the MPC to better handle out-of-distribution behavior. Paper: Full Video with more details: Kudos to Ángel Romero, Yunlong Song IEEE ICRA University of Zurich UZH Science UZH Space Hub Aerial Core AUTOASSESS European Research Council (ERC)

Check out our #ICRA2024 paper "Actor-Critic Model Predictive Control." Model-free #reinforcementlearning (RL) is known for its strong task performance and flexibility in optimizing general reward formulations. On the other hand, #ModelPredictiveControl (MPC) benefits from robustness and online replanning capabilities. We combine both approaches by introducing a new framework called Actor-Critic Model Predictive Control. The key idea is to embed a differentiable MPC within an Actor-Critic RL framework. The proposed approach leverages the short-term predictive optimization capabilities of MPC with the exploratory and end-to-end training properties of RL. The resulting policy effectively manages both short-term decisions through the MPC-based actor and long-term prediction via the critic network, unifying the benefits of both model-based control and end-to-end learning. We validate our method in simulation and the real world with a quadcopter across various high-level tasks. We show that the proposed architecture can achieve real-time control performance, learn complex behaviors via trial and error, and retain the predictive properties of the MPC to better handle out-of-distribution behavior. Paper: Full Video with more details: Kudos to Ángel Romero, Yunlong Song IEEE ICRA University of Zurich UZH Science UZH Space Hub Aerial Core AUTOASSESS European Research Council (ERC)

Davide Scaramuzza

34,889 次观看 • 2 年前

OpenAI's Noam Brown says letting AI models think for 20 seconds got the same boost in performance as scaling up the model by 100,000x and training it for 100,000 times longer, and scaling model size and compute will end up being a footnote compared to scaling model thinking

OpenAI's Noam Brown says letting AI models think for 20 seconds got the same boost in performance as scaling up the model by 100,000x and training it for 100,000 times longer, and scaling model size and compute will end up being a footnote compared to scaling model thinking

Tsarathustra

273,500 次观看 • 1 年前

I hand-wrote a 500-LoC RL stack to make hacking on RL research much easier. Most RL stacks are either massive and unhackable, or duct-taped research scripts. I am open-sourcing Mithrl, a modular RLVR stack. Next items on my checklist: adding more complex environment examples, supporting multi-gpu + async RL, and QoL fixes. I might scrap external runtime dependencies (Huggingface PEFT + vLLM) and write purpose-built, simpler versions from scratch if I feel the need. If you want to experiment with RL and are looking to own sovereign tools, I’d love to get on call, understand your requirements and help integrate for free.

I hand-wrote a 500-LoC RL stack to make hacking on RL research much easier. Most RL stacks are either massive and unhackable, or duct-taped research scripts. I am open-sourcing Mithrl, a modular RLVR stack. Next items on my checklist: adding more complex environment examples, supporting multi-gpu + async RL, and QoL fixes. I might scrap external runtime dependencies (Huggingface PEFT + vLLM) and write purpose-built, simpler versions from scratch if I feel the need. If you want to experiment with RL and are looking to own sovereign tools, I’d love to get on call, understand your requirements and help integrate for free.

omkaar

17,218 次观看 • 5 个月前

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

Physical Intelligence

441,689 次观看 • 5 个月前

introducing AlphaLab -an autonomous system that runs quantitative research end-to-end give it a dataset + goal, it explores, reads papers, builds evals, and runs parallel experiments across a GPU cluster, learning&evolving from the results all details, code, and paper below

introducing AlphaLab -an autonomous system that runs quantitative research end-to-end give it a dataset + goal, it explores, reads papers, builds evals, and runs parallel experiments across a GPU cluster, learning&evolving from the results all details, code, and paper below

Brendan Hogan

178,070 次观看 • 5 个月前

This is the Claude Code moment for Reinforcement Learning. Every frontier lab knows the secret: post-training is where the magic happens. If you can eval a task, you can use RL to benchmax your model. But building and scaling the RL loop has remained a dark art reserved for the top 1% of AI Labs. Until today. Introducing Optimus — an autonomous RL Agent Harness, and the first of many we are about to ship Iacon Autonomics. We are removing the manual grind of RL and adding taste and relentlessness to the process. Just hand Optimus a codebase, an API key, and a goal. It spins up a fleet of sub-agents to run hundreds of parallel experiments. It learns from its own history, autonomously building an evolutionary tree of runs where every failure sets the next batch up to win. Train LLMs, agents, robots in sim, game npcs and much more. If you can score it, Optimus can train it. Stop doing manual RL. Let the agents cook.

This is the Claude Code moment for Reinforcement Learning. Every frontier lab knows the secret: post-training is where the magic happens. If you can eval a task, you can use RL to benchmax your model. But building and scaling the RL loop has remained a dark art reserved for the top 1% of AI Labs. Until today. Introducing Optimus — an autonomous RL Agent Harness, and the first of many we are about to ship Iacon Autonomics. We are removing the manual grind of RL and adding taste and relentlessness to the process. Just hand Optimus a codebase, an API key, and a goal. It spins up a fleet of sub-agents to run hundreds of parallel experiments. It learns from its own history, autonomously building an evolutionary tree of runs where every failure sets the next batch up to win. Train LLMs, agents, robots in sim, game npcs and much more. If you can score it, Optimus can train it. Stop doing manual RL. Let the agents cook.

Dev Shah

13,309 次观看 • 15 天前

Today, we’re introducing KinetIQ, our own AI framework for end-to-end orchestration of humanoid robot fleets. One system, multiple robot embodiments. Industrial, service and home environments coordinated in real time. The framework consists of 4 cognitive layers: from high-level task allocation and workflow optimisation down to VLA-based task execution and RL-trained whole-body control. Watch how KinetIQ runs both our wheeled and bipedal robots. Read more on our blog:

Today, we’re introducing KinetIQ, our own AI framework for end-to-end orchestration of humanoid robot fleets. One system, multiple robot embodiments. Industrial, service and home environments coordinated in real time. The framework consists of 4 cognitive layers: from high-level task allocation and workflow optimisation down to VLA-based task execution and RL-trained whole-body control. Watch how KinetIQ runs both our wheeled and bipedal robots. Read more on our blog:

Humanoid

23,962 次观看 • 6 个月前

.Polymath is training world generation models to automate the creation of RL environments. Traditionally, RL environment generation has been bottlenecked by human data. Superintelligence will never be achieved by human data alone. Polymath is building the core technology to enable automated environment generation using far less human effort than traditionally required, and eventually none. This allows for more complex and realistic worlds, and higher quality, scale, and diversity of tasks. This will be essential to unlock RL scaling. The end goal is to create large-scale, long-horizon environments from a text description alone. This will enable the creation of worlds of arbitrary complexity and scale, which is foundational for training & evaluating autonomous, superintelligent AI agents. Congrats on the launch, Dylan Ma and Naren Yenuganti!

.Polymath is training world generation models to automate the creation of RL environments. Traditionally, RL environment generation has been bottlenecked by human data. Superintelligence will never be achieved by human data alone. Polymath is building the core technology to enable automated environment generation using far less human effort than traditionally required, and eventually none. This allows for more complex and realistic worlds, and higher quality, scale, and diversity of tasks. This will be essential to unlock RL scaling. The end goal is to create large-scale, long-horizon environments from a text description alone. This will enable the creation of worlds of arbitrary complexity and scale, which is foundational for training & evaluating autonomous, superintelligent AI agents. Congrats on the launch, Dylan Ma and Naren Yenuganti!

Y Combinator

44,192 次观看 • 6 个月前

Check out our latest work, "Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight," published in the IEEE Transactions on Robotics, where we reconcile #OptimalControl and #ReinforcementLearning, achieving the same super-human performance, but with superior generalizability, as our previous model-free deep RL! Code released! PDF: Code: Full Video: Model-free #ReinforcementLearning (RL) is known for its strong task performance and flexibility in optimizing general reward formulations. On the other hand, #ModelPredictiveControl (MPC) provides robustness, constraint handling, and powerful online replanning capabilities. In this work, we extend our previous AC-MPC paper (Romero, ICRA'24) by taking a deeper look at how both approaches can be unified. We introduce and extend Actor-Critic Model Predictive Control (AC-MPC), a framework that embeds a differentiable MPC inside an Actor-Critic RL architecture. This integration allows the MPC-based actor to perform short-term predictive optimization, while the critic facilitates long-horizon learning and exploration. We conduct a comprehensive study that highlights AC-MPC’s key advantages: - Better out-of-distribution generalization, both against unknown disturbances and changes in the quadrotor dynamics - Improved sample efficiency - A novel empirical analysis uncovering a relationship between the critic’s value function and the MPC cost function, providing deeper insight into their interplay. We validate our method in simulation and the real world on a quadcopter flying at superhuman speeds of up to 21 m/s, matching state-of-the-art model-free RL performance, and retaining the predictive structure of MPC for more reliable out-of-distribution behavior. Reference: Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight IEEE Transactions on Robotics (T-RO), 2025 PDF: Full Video: Code: Kudos to Ángel Romero, Elie Aljalbout, Yunlong Song! University of Zurich UZH Science UZH Space Hub AUTOASSESS European Research Council (ERC) UZHai

Check out our latest work, "Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight," published in the IEEE Transactions on Robotics, where we reconcile #OptimalControl and #ReinforcementLearning, achieving the same super-human performance, but with superior generalizability, as our previous model-free deep RL! Code released! PDF: Code: Full Video: Model-free #ReinforcementLearning (RL) is known for its strong task performance and flexibility in optimizing general reward formulations. On the other hand, #ModelPredictiveControl (MPC) provides robustness, constraint handling, and powerful online replanning capabilities. In this work, we extend our previous AC-MPC paper (Romero, ICRA'24) by taking a deeper look at how both approaches can be unified. We introduce and extend Actor-Critic Model Predictive Control (AC-MPC), a framework that embeds a differentiable MPC inside an Actor-Critic RL architecture. This integration allows the MPC-based actor to perform short-term predictive optimization, while the critic facilitates long-horizon learning and exploration. We conduct a comprehensive study that highlights AC-MPC’s key advantages: - Better out-of-distribution generalization, both against unknown disturbances and changes in the quadrotor dynamics - Improved sample efficiency - A novel empirical analysis uncovering a relationship between the critic’s value function and the MPC cost function, providing deeper insight into their interplay. We validate our method in simulation and the real world on a quadcopter flying at superhuman speeds of up to 21 m/s, matching state-of-the-art model-free RL performance, and retaining the predictive structure of MPC for more reliable out-of-distribution behavior. Reference: Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight IEEE Transactions on Robotics (T-RO), 2025 PDF: Full Video: Code: Kudos to Ángel Romero, Elie Aljalbout, Yunlong Song! University of Zurich UZH Science UZH Space Hub AUTOASSESS European Research Council (ERC) UZHai

Davide Scaramuzza

27,279 次观看 • 7 个月前

Not every AI interaction is best served with a large frontier model. There is a long tail of trillion-token use cases, from tagging to search, best served by a sub-10b-parameter model that runs in milliseconds and costs many orders of magnitude less than the frontier. We built Freesolo Flash to make training loops like SFT and RL easy and end-to-end completable through your coding agent.

Not every AI interaction is best served with a large frontier model. There is a long tail of trillion-token use cases, from tagging to search, best served by a sub-10b-parameter model that runs in milliseconds and costs many orders of magnitude less than the frontier. We built Freesolo Flash to make training loops like SFT and RL easy and end-to-end completable through your coding agent.

Freesolo

28,543 次观看 • 1 个月前

"One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals. And you look at the evals and you go, 'Those are pretty hard evals.' But the economic impact seems to be dramatically behind. There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data. When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.' You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?' If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"

"One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals. And you look at the evals and you go, 'Those are pretty hard evals.' But the economic impact seems to be dramatically behind. There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data. When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.' You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?' If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"

Dwarkesh Patel

502,484 次观看 • 9 个月前

Our 6ft humanoid HumanPlus at Stanford can autonomously put on a Nike skateboard shoe, tie shoelaces, stand up and walk. Using two transformers & dual RGB vision, it integrates two recipes of general robotics end-to-end: - imitating humans in real world - large-scale RL in sim

Our 6ft humanoid HumanPlus at Stanford can autonomously put on a Nike skateboard shoe, tie shoelaces, stand up and walk. Using two transformers & dual RGB vision, it integrates two recipes of general robotics end-to-end: - imitating humans in real world - large-scale RL in sim

Zipeng Fu

246,224 次观看 • 2 年前

Lecture 10 of my course! Nominally on regularization in RL, so I discuss the evolving role of the KL penalty in RL, but also a set of nice RL papers that explain what RL helps models generalize better than SFT -- with theory supporting it. When going through these, it's so interesting how seasonal problems in ML are. Lots of problems from controlling reward models overopt will rhyme as we try to control rubrics for agents. 00:00 Intro & the role of regularization 02:50 The KL penalty in RL 10:53 RL as a reverse KL loss 21:09 Why RL generalizes better than SFT 25:15 Other regularization tools Just a few videos left as I get to the end of the course. Thanks all, and keep sending questions. Spread the word if you have a second.

Lecture 10 of my course! Nominally on regularization in RL, so I discuss the evolving role of the KL penalty in RL, but also a set of nice RL papers that explain what RL helps models generalize better than SFT -- with theory supporting it. When going through these, it's so interesting how seasonal problems in ML are. Lots of problems from controlling reward models overopt will rhyme as we try to control rubrics for agents. 00:00 Intro & the role of regularization 02:50 The KL penalty in RL 10:53 RL as a reverse KL loss 21:09 Why RL generalizes better than SFT 25:15 Other regularization tools Just a few videos left as I get to the end of the course. Thanks all, and keep sending questions. Spread the word if you have a second.

Nathan Lambert

89,404 次观看 • 1 个月前

Introducing Tahoe-x1 (Tx1) by Tahoe. A 3-billion-parameter, single-cell foundation model that learns unified representations of genes, cells, and drugs, achieving state-of-the-art performance across cancer-relevant cell biology benchmarks, open-sourced on Hugging Face. 🧵

Introducing Tahoe-x1 (Tx1) by Tahoe. A 3-billion-parameter, single-cell foundation model that learns unified representations of genes, cells, and drugs, achieving state-of-the-art performance across cancer-relevant cell biology benchmarks, open-sourced on Hugging Face. 🧵

Nima Alidoust

172,035 次观看 • 10 个月前

DPO Debate: Is RL needed for RLHF? All things as we cannot settle if DPO or RL is better. At least it is a good exercise. 1. Derivations in the DPO paper. Hint, the authors are good at math 2. cDPO, IPO, and related equations 3. Speculation on potential oddities of DPO vs RL 4. Reminders on the state of open RLHF tldr: we have more limitations with data and tooling and evaluation than optimizer choice Slides: Recent blog post of mine on DPO (more next Wed.): DPO Paper: On youtube:

DPO Debate: Is RL needed for RLHF? All things as we cannot settle if DPO or RL is better. At least it is a good exercise. 1. Derivations in the DPO paper. Hint, the authors are good at math 2. cDPO, IPO, and related equations 3. Speculation on potential oddities of DPO vs RL 4. Reminders on the state of open RLHF tldr: we have more limitations with data and tooling and evaluation than optimizer choice Slides: Recent blog post of mine on DPO (more next Wed.): DPO Paper: On youtube:

Nathan Lambert

100,027 次观看 • 2 年前

New release from Meta FAIR — Meta Motivo is a first-of-its-kind behavioral foundation model for controlling virtual physics-based humanoid agents for a wide range of complex whole-body tasks. The model is capable of expressing human-like behaviors and achieves performance competitive with task-specific methods and outperforms state-of-the-art unsupervised RL and model-based baselines. Try the demo ➡️ Get the model and code ➡️ We’re excited about how this research could pave the way for fully embodied agents, leading to more lifelike NPCs, democratization of character animation and new types of immersive experiences.

New release from Meta FAIR — Meta Motivo is a first-of-its-kind behavioral foundation model for controlling virtual physics-based humanoid agents for a wide range of complex whole-body tasks. The model is capable of expressing human-like behaviors and achieves performance competitive with task-specific methods and outperforms state-of-the-art unsupervised RL and model-based baselines. Try the demo ➡️ Get the model and code ➡️ We’re excited about how this research could pave the way for fully embodied agents, leading to more lifelike NPCs, democratization of character animation and new types of immersive experiences.

AI at Meta

129,253 次观看 • 1 年前

The DeepSeek-R1 paper is a gem! Highly encourage everyone to read it. It's clear that LLM reasoning capabilities can be learned in different ways. RL, if applied correctly and at scale, can lead to some really powerful and interesting scaling and emergent properties. There is more to RL than meets the eye! Here is my breakdown of the paper along with a few tests: The multi-state training might not make sense initially but they provide clues on optimizations that we can continue to tap into. Data quality is still very important for enhancing the usability of the LLM. Unlike other reasoning LLMs, DeepSeek-R1's training recipe and weights are open so we can build on top of it. This opens up exciting research opportunities. About the attached clip: the previous preview model wasn't able to solve this task. DeepSeek-R1 can solve this and many other tasks that o1 can solve. It's a very good model for coding and math.

The DeepSeek-R1 paper is a gem! Highly encourage everyone to read it. It's clear that LLM reasoning capabilities can be learned in different ways. RL, if applied correctly and at scale, can lead to some really powerful and interesting scaling and emergent properties. There is more to RL than meets the eye! Here is my breakdown of the paper along with a few tests: The multi-state training might not make sense initially but they provide clues on optimizations that we can continue to tap into. Data quality is still very important for enhancing the usability of the LLM. Unlike other reasoning LLMs, DeepSeek-R1's training recipe and weights are open so we can build on top of it. This opens up exciting research opportunities. About the attached clip: the previous preview model wasn't able to solve this task. DeepSeek-R1 can solve this and many other tasks that o1 can solve. It's a very good model for coding and math.

elvis

140,692 次观看 • 1 年前

"Our approach is to build products and conduct research that are in service of accelerated AI deployments. Our platform team builds tools and context primitives that enable faster deployment. Our research team builds frontier systems, including a state-of-the-art RL stack. We then take that research and product and forward-deploy with our customers to help deliver real value." Thanks Founders You Should Know for having us. Open roles at:

"Our approach is to build products and conduct research that are in service of accelerated AI deployments. Our platform team builds tools and context primitives that enable faster deployment. Our research team builds frontier systems, including a state-of-the-art RL stack. We then take that research and product and forward-deploy with our customers to help deliver real value." Thanks Founders You Should Know for having us. Open roles at:

Applied Compute

45,197 次观看 • 5 个月前