Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Mondo's Beni ( may look cute, but there is a serious learning stack inside. It can perform flips, navigate autonomously, and track moving subjects. The secret is reinforcement learning: Beni practices at massive scale in simulation, optimizing its controller through millions of virtual trials before entering the real world....

ShuoYangAIR's profile picture

Shuo Yang

5,468 subscribers

31,539 görüntüleme • 3 ay önce •via X (Twitter)

Bilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

24 Yorum

Shuo Yang profil fotoğrafı
Shuo Yang3 ay önce

1/ @nvidia IsaacLab and @GoogleDeepMind MuJoCo are core parts of our training and validation pipeline. We train controllers across large numbers of simulated environments, then validate them in a separate physics engine.

Shuo Yang profil fotoğrafı
Shuo Yang3 ay önce

2/ Modern legged-robot reinforcement learning builds on years of excellent research. The work of @rdn_nikita , David Hoeller, and their collaborators has been particularly influential. For Beni’s flipping motions, we also drew inspiration from @xbpeng4’s classic DeepMimic paper. Beni has been reading the literature—and practicing its backflips.

Shuo Yang profil fotoğrafı
Shuo Yang3 ay önce

3/ Some behaviors need an additional layer of sim-to-real polish. For these, we use ideas from Regularized Online Adaptation, or ROA, developed by my good friends @zipengfu and @xuxin_cheng (and @deepakpathak) :

Shuo Yang profil fotoğrafı
Shuo Yang3 ay önce

4/ Along side all the neural networks, Beni also uses some good old-fashioned SLAM—in several creative ways. One particularly valuable tool is GTSAM, created by @fdellaert and contributors.

Shuo Yang profil fotoğrafı
Shuo Yang3 ay önce

5/ Beni’s target-tracking system also contains several algorithmic innovations developed by our own team. These improvements allow Beni to follow a subject smoothly while coordinating perception, locomotion, and camera motion. We may publish more technical details in the future.

Shuo Yang profil fotoğrafı
Shuo Yang3 ay önce

Sometimes the best robotics system is not “AI versus classical robotics.” It is AI standing on top of decades of great robotics research. Small robot. Serious algorithms. 🤖

Carlos DP 🤖🇺🇸 profil fotoğrafı
Carlos DP 🤖🇺🇸3 ay önce

is there a developer mode on Beni? like, could he be used for people learning to do RL on real robots in a fun way?

Deepak Pathak profil fotoğrafı
Deepak Pathak3 ay önce

Great work @Shuo — love to see our ROA algorithm in action. This robot is so cool!

JVM profil fotoğrafı
JVM3 ay önce

and you want $549 for this?!

cr_ss_ profil fotoğrafı
cr_ss_2 ay önce

Thats a really cool toy!

HGONG | Founder & Researcher profil fotoğrafı
HGONG | Founder & Researcher3 ay önce

咦？你是RM的杨硕吗？

Ankur Deka profil fotoğrafı
Ankur Deka3 ay önce

Great to see Beni’s acrobatics @ShuoYangAIR!

Keith Botha profil fotoğrafı
Keith Botha3 ay önce

We love to see it ! Really good partnership opportunity with @UFBots

Jrmginer profil fotoğrafı
Jrmginer3 ay önce

that’s cool！

Swiftlined profil fotoğrafı
Swiftlined3 ay önce

Solid stuff

tobitege profil fotoğrafı
tobitege3 ay önce

Now I want Beni to kick back 🤣 Love that lil guy, even though I don't have a use for him!

Will Kriski profil fotoğrafı
Will Kriski2 ay önce

can we train it ourselves similar to microduck and use a simulator? @pollenrobotics

theobot profil fotoğrafı
theobot3 ay önce

this is awesome

Sadikk profil fotoğrafı
Sadikk2 ay önce

This is great for cat parents.. Best toy for cats to ay with.. Consider cheaper version.

Tanay Jaipuria profil fotoğrafı
Tanay Jaipuria3 ay önce

have you experimented with world models for simulation?

Black Eyed Hat Man profil fotoğrafı
Black Eyed Hat Man3 ay önce

😈 cute! But was the kick necessary?! 😰

T. profil fotoğrafı
T.3 ay önce

a cutie, how hes following the skater 🥰

AI Apps API profil fotoğrafı
AI Apps API3 ay önce

The simulation scale is the impressive part, and it is also where the hard problem hides. Training at that volume gets you a policy that is optimal for your simulator, including its contact model, its friction constants and its integration step. The gap to hardware is not noise, it is systematic, and domain randomization tends to paper over it rather than measure it. The number I always want to see alongside these is how much the policy degrades on the real robot before any fine tuning. That one figure tells you how good the physics model actually was.

Hoon_Sky profil fotoğrafı
Hoon_Sky3 ay önce

Probably cannot last long on batteries

Benzer Videolar

The RL era of robotics is here, and simulation is how to scale it. An early look at our research on a achieving a better scaling paradigm for robot learning with simulation 👇
1:59

The RL era of robotics is here, and simulation is how to scale it. An early look at our research on a achieving a better scaling paradigm for robot learning with simulation 👇

Moonlake

472,603 görüntüleme • 4 gün önce

This is a demonstration video of in-hand object rotation with new Torobo Hand. By leveraging large-scale parallel reinforcement learning in NVIDIA Isaac Sim, the learned policy can perform object manipulation in MuJoCo and on the real robot without any additional training.
1:09

This is a demonstration video of in-hand object rotation with new Torobo Hand. By leveraging large-scale parallel reinforcement learning in NVIDIA Isaac Sim, the learned policy can perform object manipulation in MuJoCo and on the real robot without any additional training.

東京ロボティクス株式会社

10,951 görüntüleme • 1 yıl önce

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->
1:15

The best way to get robust, high-quality robot performance is through reinforcement learning; but RL in either the real world or a traditional simulation has lots of limitations. Instead, Jiazhi Yang in RISE does RL in a compositional world model. Learn more ->

Chris Paxton

34,102 görüntüleme • 5 ay önce

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.
0:58

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 görüntüleme • 1 yıl önce

Today we present Morpheus, a persistent enterprise simulation platform designed to make Continual Learning a reality. Morpheus is the world’s first real world Reinforcement Learning environment. Every Reinforcement Learning environment operates in the game world. Benchmarks like Atari, OpenAI Gym, MuJoCo, and Procgen are all small, game-like worlds that reset every few minutes. But the real world never resets. A business keeps running and evolving everyday. We tested how frontier LLMs would perform in realistic and dynamic business environments 🧬on Morpheus. The main conclusion was that LLMs are not continual learners. 🧵Here’s how we did it and what we learned:
1:18

Today we present Morpheus, a persistent enterprise simulation platform designed to make Continual Learning a reality. Morpheus is the world’s first real world Reinforcement Learning environment. Every Reinforcement Learning environment operates in the game world. Benchmarks like Atari, OpenAI Gym, MuJoCo, and Procgen are all small, game-like worlds that reset every few minutes. But the real world never resets. A business keeps running and evolving everyday. We tested how frontier LLMs would perform in realistic and dynamic business environments 🧬on Morpheus. The main conclusion was that LLMs are not continual learners. 🧵Here’s how we did it and what we learned:

Skyfall AI

270,028 görüntüleme • 3 ay önce

Just months after its debut, Atlas is proving why it is the world’s most capable and dynamic humanoid robot, ready for real work. Lifting a mini-fridge is a feat of strength, but the true breakthrough is in the underlying reinforcement learning and controls systems.
5:34

Just months after its debut, Atlas is proving why it is the world’s most capable and dynamic humanoid robot, ready for real work. Lifting a mini-fridge is a feat of strength, but the true breakthrough is in the underlying reinforcement learning and controls systems.

Boston Dynamics

41,726 görüntüleme • 5 ay önce

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,867 görüntüleme • 5 ay önce

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!
1:11:51

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

18,813 görüntüleme • 9 ay önce

Bittensor is the main thing I’m watching at Proof of Talk. The subnet conversation is moving from CT speculation into real institutional interest now. Decentralized intelligence, open markets for machine learning, subnet economies, this is the part of crypto that actually feels like it can matter at scale. Most people still underestimate how big that shift is. Feels like $TAO is entering its serious capital era.
1:35

Bittensor is the main thing I’m watching at Proof of Talk. The subnet conversation is moving from CT speculation into real institutional interest now. Decentralized intelligence, open markets for machine learning, subnet economies, this is the part of crypto that actually feels like it can matter at scale. Most people still underestimate how big that shift is. Feels like $TAO is entering its serious capital era.

nordin.eth

11,217 görüntüleme • 5 ay önce

See Spot perform dynamic whole-body manipulation. Using a combination of reinforcement learning (RL) and sampling-based control, the robot is able to autonomously drag, roll, and stack tires weighing 15 kg (33 lb), well above its maximum arm lift capacity. Learn more about coordinating locomotion and manipulation processes:
0:27

See Spot perform dynamic whole-body manipulation. Using a combination of reinforcement learning (RL) and sampling-based control, the robot is able to autonomously drag, roll, and stack tires weighing 15 kg (33 lb), well above its maximum arm lift capacity. Learn more about coordinating locomotion and manipulation processes:

RAI Institute

87,753 görüntüleme • 1 yıl önce

Reinforcement learning is used to speed the production of behavior for the Boston Dynamics Atlas humanoid robot. At the heart of the learning process is a physics-based simulator that generates training data for a variety of maneuvers.
1:10

Reinforcement learning is used to speed the production of behavior for the Boston Dynamics Atlas humanoid robot. At the heart of the learning process is a physics-based simulator that generates training data for a variety of maneuvers.

RAI Institute

76,796 görüntüleme • 1 yıl önce

notes on my reinforcement learning for rocket league side quest so far: i’m in awe that — we live in a time where i can train a reinforcement learning model that is learning to play rocket league and at the same time, test community made rocket league models. where, both processes can run in parallel & locally on my hardware. in the game window — i’m playing against a reinforcement learning ppo model that’s running locally. and in the vscode window — i’m training my own reinforcement learning model (just a dummy example script for now). also, the rocket league botting ecosystem has incredible tooling where you can test out different reinforcement learning ideas against other algorithms + it has great documentation for linux and windows. it’s honestly a beautiful hidden gem of a rabbit hole :)
0:24

notes on my reinforcement learning for rocket league side quest so far: i’m in awe that — we live in a time where i can train a reinforcement learning model that is learning to play rocket league and at the same time, test community made rocket league models. where, both processes can run in parallel & locally on my hardware. in the game window — i’m playing against a reinforcement learning ppo model that’s running locally. and in the vscode window — i’m training my own reinforcement learning model (just a dummy example script for now). also, the rocket league botting ecosystem has incredible tooling where you can test out different reinforcement learning ideas against other algorithms + it has great documentation for linux and windows. it’s honestly a beautiful hidden gem of a rabbit hole :)

naklecha

46,795 görüntüleme • 1 yıl önce

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.
1:22

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re moving from robots that follow programmed rules to machines that learn strategies in simulation and execute them in the physical world.

SciTech Era

1,585,680 görüntüleme • 5 ay önce

Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.
0:34

Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma. To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference.

Carlo Sferrazza

220,153 görüntüleme • 11 ay önce

These are not CGI. Reinforcement learning is so back. When operating on strings, it gives us o3. When operating on physical motors, it gives us a perfect humanoid backflip and a robot creature that out-maneuvers almost every animal on earth. RL is one of the only learning algorithms that can master both the world of bits and the world of atoms. Give me a reward function, and I shall move the world. 2025, Year of RL.
0:12

These are not CGI. Reinforcement learning is so back. When operating on strings, it gives us o3. When operating on physical motors, it gives us a perfect humanoid backflip and a robot creature that out-maneuvers almost every animal on earth. RL is one of the only learning algorithms that can master both the world of bits and the world of atoms. Give me a reward function, and I shall move the world. 2025, Year of RL.

Jim Fan

357,071 görüntüleme • 1 yıl önce

Mark Zuckerberg and Priscilla Chan say virtual cells could change how science takes risk, letting researchers test bold ideas in simulation before entering the wet lab. “The promise of being able to do virtual biology… is you can actually take on riskier ideas.” “If you had a virtual cell model… you could actually then start testing it and tinkering on the computational side… and actually see if there is promise doing the experiments in silico before you make the time and money investment in the wet lab.”
1:15

Mark Zuckerberg and Priscilla Chan say virtual cells could change how science takes risk, letting researchers test bold ideas in simulation before entering the wet lab. “The promise of being able to do virtual biology… is you can actually take on riskier ideas.” “If you had a virtual cell model… you could actually then start testing it and tinkering on the computational side… and actually see if there is promise doing the experiments in silico before you make the time and money investment in the wet lab.”

a16z

44,009 görüntüleme • 1 yıl önce

Former Meta Chief AI Scientist Yann LeCun on the three paradigms of machine learning — and why the third is what made ChatGPT possible: Here's each one, and where it breaks. First, supervised learning. You tell the machine the answer. "You show it a picture, let's say of a table, and you tell it this is a table. So it's supervised because you tell it what the correct answer is." Get it wrong, and the machine rewrites itself: "The system computes its output, and if it says something else than table, then it's going to adjust its parameters, its internal structure, so that the output it produces gets closer to the output you want." Repeat at scale and something more than memorisation appears: "Eventually the system will find a way to recognize every image you trained it on, but also images it's never seen that are similar to the one you train it on. This is called a generalization ability." The limit: a human has to supply every single answer. That doesn't scale to the size of the internet. Second, reinforcement learning. You don't give the answer, only a verdict. "You don't tell the system what the correct answer is. You only tell it whether the answer it produced was good or bad." Learning to ride a bike, essentially: "You try to ride a bike and you don't know how to ride the bike and after a while you fall. So you know you did something bad and so you change your strategy a little bit. And eventually you learn how to ride a bike." For years the field assumed this was the closest thing to how animals actually learn. Yann LeCun's verdict: "Now it turns out reinforcement learning is extremely inefficient." It dominates wherever failure is free: "It works really well if you want to train a system to play chess or play go or poker, because you can have the system play millions and millions of games against itself and basically fine-tune itself. But it doesn't really work in the real world." The limit, in one image: "If you want to train a car to drive itself, you're not going to do it with reinforcement learning. It's going to crash thousands of times." On robotics he's careful rather than dismissive: "Reinforcement learning can be part of the solution, but it's not the complete answer. It's not sufficient." Third, self-supervised learning. You tell the machine nothing at all. "And this is what has enabled the recent progress in natural language understanding and chatbots." The strange part is that you stop asking for a task: "You don't train the system to accomplish any particular task. You just train it to basically capture the structure..." The method is deliberate sabotage: "You take a piece of text, you corrupt it in some way, by for example removing some words, and then you train a big neural net to predict the words that are missing." And one narrow version of that trick runs every chatbot on Earth: "A special case of this is that you take a piece of text and the last word in that text is not visible, and so you train the system to predict the last word in that text — and this is the way large language models are trained on." So why did the third one win? Supervised learning needs a human. Reinforcement learning needs a crash. Self-supervised learning needs neither — because the missing word and the correct answer are the same thing. The data grades itself.
2:58

Former Meta Chief AI Scientist Yann LeCun on the three paradigms of machine learning — and why the third is what made ChatGPT possible: Here's each one, and where it breaks. First, supervised learning. You tell the machine the answer. "You show it a picture, let's say of a table, and you tell it this is a table. So it's supervised because you tell it what the correct answer is." Get it wrong, and the machine rewrites itself: "The system computes its output, and if it says something else than table, then it's going to adjust its parameters, its internal structure, so that the output it produces gets closer to the output you want." Repeat at scale and something more than memorisation appears: "Eventually the system will find a way to recognize every image you trained it on, but also images it's never seen that are similar to the one you train it on. This is called a generalization ability." The limit: a human has to supply every single answer. That doesn't scale to the size of the internet. Second, reinforcement learning. You don't give the answer, only a verdict. "You don't tell the system what the correct answer is. You only tell it whether the answer it produced was good or bad." Learning to ride a bike, essentially: "You try to ride a bike and you don't know how to ride the bike and after a while you fall. So you know you did something bad and so you change your strategy a little bit. And eventually you learn how to ride a bike." For years the field assumed this was the closest thing to how animals actually learn. Yann LeCun's verdict: "Now it turns out reinforcement learning is extremely inefficient." It dominates wherever failure is free: "It works really well if you want to train a system to play chess or play go or poker, because you can have the system play millions and millions of games against itself and basically fine-tune itself. But it doesn't really work in the real world." The limit, in one image: "If you want to train a car to drive itself, you're not going to do it with reinforcement learning. It's going to crash thousands of times." On robotics he's careful rather than dismissive: "Reinforcement learning can be part of the solution, but it's not the complete answer. It's not sufficient." Third, self-supervised learning. You tell the machine nothing at all. "And this is what has enabled the recent progress in natural language understanding and chatbots." The strange part is that you stop asking for a task: "You don't train the system to accomplish any particular task. You just train it to basically capture the structure..." The method is deliberate sabotage: "You take a piece of text, you corrupt it in some way, by for example removing some words, and then you train a big neural net to predict the words that are missing." And one narrow version of that trick runs every chatbot on Earth: "A special case of this is that you take a piece of text and the last word in that text is not visible, and so you train the system to predict the last word in that text — and this is the way large language models are trained on." So why did the third one win? Supervised learning needs a human. Reinforcement learning needs a crash. Self-supervised learning needs neither — because the missing word and the correct answer are the same thing. The data grades itself.

Big Brain AI

49,293 görüntüleme • 2 ay önce

New groundbreaking research in AI x Robotics! ⦿ The system can autonomously perform various mobile bimanual tasks, including cooking, cleaning, housekeeping, and operating an elevator. ⦿ The hardware costs only $32k. ⦿ Learning is achieved through human demonstration data and co-training.
2:32

New groundbreaking research in AI x Robotics! ⦿ The system can autonomously perform various mobile bimanual tasks, including cooking, cleaning, housekeeping, and operating an elevator. ⦿ The hardware costs only $32k. ⦿ Learning is achieved through human demonstration data and co-training.

The Humanoid Hub

10,835 görüntüleme • 2 yıl önce

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT
4:34

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

MasterBOT

51,189 görüntüleme • 1 yıl önce

Noble Machines' Moby robot hauling a 50 lb crate. The whole-body control (WBC) is the physical foundation of the learning stack. It absorbs payload variation, contact response, and center-of-mass shifts in real time at the control layer and enabled cleaner data-focused learning
1:37

Noble Machines' Moby robot hauling a 50 lb crate. The whole-body control (WBC) is the physical foundation of the learning stack. It absorbs payload variation, contact response, and center-of-mass shifts in real time at the control layer and enabled cleaner data-focused learning

The Humanoid Hub

17,597 görüntüleme • 5 ay önce