Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

We are moving from model controller to RL for locomotion. The policy is starting to look quite robust. Even more convinced now that legs are easier than wheels, given the software complexity has shrunk tremendously with RL.

Sankaet

14,482 subscribers

133,130 Aufrufe • vor 10 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

enjoyed hosting CNET at Foundation Robotics SF office. We discuss defense usecases, teleop, RL locomotion and more

enjoyed hosting CNET at Foundation Robotics SF office. We discuss defense usecases, teleop, RL locomotion and more

Sankaet

113,210 Aufrufe • vor 9 Monaten

You'd think the race to AGI would mean training the biggest possible model. But parameter scaling had stalled for a long time after GPT-4's trillion+ parameters, and only now are models getting bigger again. What gives? Partially it’s RL scaling, as Dylan Patel explains. A 5T parameter model takes 5x longer to generate RL rollouts than a 1T model. Even if the bigger model is 2x more sample-efficient, the smaller model finishes RL faster, gets deployed to research sooner, and starts helping build the next model before the big one is even done training.

You'd think the race to AGI would mean training the biggest possible model. But parameter scaling had stalled for a long time after GPT-4's trillion+ parameters, and only now are models getting bigger again. What gives? Partially it’s RL scaling, as Dylan Patel explains. A 5T parameter model takes 5x longer to generate RL rollouts than a 1T model. Even if the bigger model is 2x more sample-efficient, the smaller model finishes RL faster, gets deployed to research sooner, and starts helping build the next model before the big one is even done training.

Dwarkesh Patel

65,123 Aufrufe • vor 3 Monaten

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

Physical Intelligence

437,001 Aufrufe • vor 4 Monaten

Behavioral Foundation Models (BFMs) trained with RL are secretly more powerful than we think. BFM’s directly output a policy believed to be near-optimal given any reward function. Our new work shows that they can actually do much better:

Behavioral Foundation Models (BFMs) trained with RL are secretly more powerful than we think. BFM’s directly output a policy believed to be near-optimal given any reward function. Our new work shows that they can actually do much better:

Harshit Sikchi

44,182 Aufrufe • vor 1 Jahr

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

Chris Paxton

33,766 Aufrufe • vor 1 Monat

Open Duck Mini is standing up using a policy learned with RL in simulation! We are still pushing locomotion, so if you have some knowledge of Isaac Gym and want to help feel free to join us! Thomas Wolf

Open Duck Mini is standing up using a policy learned with RL in simulation! We are still pushing locomotion, so if you have some knowledge of Isaac Gym and want to help feel free to join us! Thomas Wolf

Antoine Pirrone

66,084 Aufrufe • vor 1 Jahr

Hiring RL Engineer! Started off as a curious project at Lossfunk to push the boundaries of LLMs in social reasoning - we are now building RL environments, data, and benchmarks to simulate more real-world scenarios. If you want to train SoTA RL models over multi-GPUs (H200s/B200s) to unlock next AI frontier, this is for you.

Hiring RL Engineer! Started off as a curious project at Lossfunk to push the boundaries of LLMs in social reasoning - we are now building RL environments, data, and benchmarks to simulate more real-world scenarios. If you want to train SoTA RL models over multi-GPUs (H200s/B200s) to unlock next AI frontier, this is for you.

Satpal Singh Rathore

46,010 Aufrufe • vor 11 Monaten

We are slowly leaving the “AI chatbot” era. agents are starting to control actual software. watching Mercury Agent interact with Spotify natively from inside the workflow feels way more important than people realize right now.

We are slowly leaving the “AI chatbot” era. agents are starting to control actual software. watching Mercury Agent interact with Spotify natively from inside the workflow feels way more important than people realize right now.

GitHub Projects Community

27,280 Aufrufe • vor 2 Monaten

Is It All Starting To Make Sense Now? Lies Are Easier To Believe Than The Truth

Is It All Starting To Make Sense Now? Lies Are Easier To Believe Than The Truth

{Matt} $XRPatriot

364,466 Aufrufe • vor 2 Jahren

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

TBPN

76,696 Aufrufe • vor 1 Jahr

"One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals. And you look at the evals and you go, 'Those are pretty hard evals.' But the economic impact seems to be dramatically behind. There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data. When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.' You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?' If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"

"One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals. And you look at the evals and you go, 'Those are pretty hard evals.' But the economic impact seems to be dramatically behind. There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data. When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.' You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?' If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"

Dwarkesh Patel

502,249 Aufrufe • vor 8 Monaten

I hand-wrote a 500-LoC RL stack to make hacking on RL research much easier. Most RL stacks are either massive and unhackable, or duct-taped research scripts. I am open-sourcing Mithrl, a modular RLVR stack. Next items on my checklist: adding more complex environment examples, supporting multi-gpu + async RL, and QoL fixes. I might scrap external runtime dependencies (Huggingface PEFT + vLLM) and write purpose-built, simpler versions from scratch if I feel the need. If you want to experiment with RL and are looking to own sovereign tools, I’d love to get on call, understand your requirements and help integrate for free.

I hand-wrote a 500-LoC RL stack to make hacking on RL research much easier. Most RL stacks are either massive and unhackable, or duct-taped research scripts. I am open-sourcing Mithrl, a modular RLVR stack. Next items on my checklist: adding more complex environment examples, supporting multi-gpu + async RL, and QoL fixes. I might scrap external runtime dependencies (Huggingface PEFT + vLLM) and write purpose-built, simpler versions from scratch if I feel the need. If you want to experiment with RL and are looking to own sovereign tools, I’d love to get on call, understand your requirements and help integrate for free.

omkaar

17,218 Aufrufe • vor 4 Monaten

Introducing INTELLECT-3: Scaling RL to a 100B+ MoE model on our end-to-end stack Achieving state-of-the-art performance for its size across math, code and reasoning Built using the same tools we put in your hands, from environments & evals, RL frameworks, sandboxes & more

Introducing INTELLECT-3: Scaling RL to a 100B+ MoE model on our end-to-end stack Achieving state-of-the-art performance for its size across math, code and reasoning Built using the same tools we put in your hands, from environments & evals, RL frameworks, sandboxes & more

Prime Intellect

1,142,730 Aufrufe • vor 8 Monaten

Like LLMs, autoregressive transformers for action tokens need a reasoning layer to reduce hallucinations and boost reliability. Grounding this layer in the physics of the action space using DVBFs makes for scalable, task-agnostic training—far simpler than creating RL reward functions for each task. Learn more about our novel approach in the video below 👇🏼

Like LLMs, autoregressive transformers for action tokens need a reasoning layer to reduce hallucinations and boost reliability. Grounding this layer in the physics of the action space using DVBFs makes for scalable, task-agnostic training—far simpler than creating RL reward functions for each task. Learn more about our novel approach in the video below 👇🏼

Sankaet

52,565 Aufrufe • vor 1 Jahr

This is a 20-minute unedited on/off-gantry run from today. What's special about it is that no gains were tuned on the robot before we started running the policy. This is the DVBF-based motion policy we've been working on, which is turning out to be quite special. The goal of using DVBF is to ensure policies can be deployed on many robots at once without manual gain adjustment before the policy runs reliably on each robot.

This is a 20-minute unedited on/off-gantry run from today. What's special about it is that no gains were tuned on the robot before we started running the policy. This is the DVBF-based motion policy we've been working on, which is turning out to be quite special. The goal of using DVBF is to ensure policies can be deployed on many robots at once without manual gain adjustment before the policy runs reliably on each robot.

Sankaet

36,570 Aufrufe • vor 8 Monaten

Before humanoid robots are ready for homes, they need to pass the "Home Alone test," according to Standard Bots CEO Evan Beard. "We think maybe humanoids are right for the home. But then you have the safety issue. When it's going up the stairs, you need two five-year-olds to be able to jump on the back of it, pull it, or try to trip it. It's the Home Alone test." "You see RL robots and it looks so impressive. I think the average person thinks 'Oh yeah, robotics is solved.' But to beat the Home Alone test, RL is not quite there and it's hard to simulate all the things that an eight-year-old could do to it. So you can't even do the RL right now. It's an unsolved problem... I think it's going to be years away from being solved in the home."

Before humanoid robots are ready for homes, they need to pass the "Home Alone test," according to Standard Bots CEO Evan Beard. "We think maybe humanoids are right for the home. But then you have the safety issue. When it's going up the stairs, you need two five-year-olds to be able to jump on the back of it, pull it, or try to trip it. It's the Home Alone test." "You see RL robots and it looks so impressive. I think the average person thinks 'Oh yeah, robotics is solved.' But to beat the Home Alone test, RL is not quite there and it's hard to simulate all the things that an eight-year-old could do to it. So you can't even do the RL right now. It's an unsolved problem... I think it's going to be years away from being solved in the home."

TBPN

32,203 Aufrufe • vor 1 Monat

I always dreamed of AGI as a wise advisor for humanity. Although LLMs are great for coding & knowledge work, I wouldn’t trust them to give me advice on my career, business strategy, or policy preferences. How can we build AI systems optimized for wisdom? At Mantic we believe the unlock is prediction: predicting world events as accurately as possible, and hill-climbing this single metric. Today we share some recent progress on the Thinking Machines website, having found Tinker a great platform for our RL experiments. TL;DR: We RL-tune gpt-oss-120b to become a better forecaster than any other model. Having good scaffolding is a prerequisite. A fun result: our tuned model + Grok are decorrelated from the other best models, and so are the most indispensable when picking a team.

I always dreamed of AGI as a wise advisor for humanity. Although LLMs are great for coding & knowledge work, I wouldn’t trust them to give me advice on my career, business strategy, or policy preferences. How can we build AI systems optimized for wisdom? At Mantic we believe the unlock is prediction: predicting world events as accurately as possible, and hill-climbing this single metric. Today we share some recent progress on the Thinking Machines website, having found Tinker a great platform for our RL experiments. TL;DR: We RL-tune gpt-oss-120b to become a better forecaster than any other model. Having good scaffolding is a prerequisite. A fun result: our tuned model + Grok are decorrelated from the other best models, and so are the most indispensable when picking a team.

Toby Shevlane

177,741 Aufrufe • vor 4 Monaten

It seems the buzz from August's Alaska summit has faded. Russian Deputy Foreign Minister Sergey Ryabkov has compared relations with the US to “a building that has cracked and is now collapsing to its foundations.” “There are no factory settings left since those relations were built in another era. We have a structure that has gone to cracks; it’s collapsing, and the Americans are to blame. The cracks have reached the foundation. It’s easier to destroy than to build. We are ready to build, but we see no movement from their side.”

It seems the buzz from August's Alaska summit has faded. Russian Deputy Foreign Minister Sergey Ryabkov has compared relations with the US to “a building that has cracked and is now collapsing to its foundations.” “There are no factory settings left since those relations were built in another era. We have a structure that has gone to cracks; it’s collapsing, and the Americans are to blame. The cracks have reached the foundation. It’s easier to destroy than to build. We are ready to build, but we see no movement from their side.”

Brian McDonald

13,908 Aufrufe • vor 9 Monaten

We used RL to train models that create curated context from long documents for downstream use by agents. The models sometimes learn to invent their own abbreviations and shorthand. Optimizing with RL for downstream use produces very different artifacts from ordinary summaries: shorter, denser, creatively concise. We call these neural cheat-sheets.

We used RL to train models that create curated context from long documents for downstream use by agents. The models sometimes learn to invent their own abbreviations and shorthand. Optimizing with RL for downstream use produces very different artifacts from ordinary summaries: shorter, denser, creatively concise. We call these neural cheat-sheets.

Applied Compute

36,155 Aufrufe • vor 29 Tagen