Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Wow. This is crazy. A developer trained an AI agent in simulation and deployed it onto a real robotic air hockey table using reinforcement learning. This robot can track the puck with millimeter-level accuracy and react in roughly 20 milliseconds, fast enough to challenge even skilled human players. We’re... show more

SciTech Era

22,926 subscribers

1,585,680 views • 4 months ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

33 Comments

wackfpv4 months ago

My toxic trait is believing I can beat it

Dmitry Grinberg4 months ago

This is the dumbest possible timeline. You can make a machine to play air hockey better than any human using a small eight bit micro controller. Throwing gigabytes of RAM and “ML” at it is peak idiocy.

Michael Beckerman4 months ago

Great. Now, put TWO of those on a table so we can watch them play each other!

SciTech Era4 months ago

more detail from GitHub report: for yt video explanation:

SciTech Era4 months ago

Here's the system overview and a video of an agent playing against itself in simulation.

Hotep Jesus4 months ago

Burn it now before it’s too late!!!

Again?4 months ago

For some reason, probably because I’m dense, this impresses me more than most robotics demonstrations.

Edward Smith4 months ago

A big part of playing this game is watching the arm of the opposing player. The brief instant of out stretch hand gesture is enough to prepare you for a directional movement. Removing that from the game gives the person a massive disadvantage. In addition removing the human interaction and enjoyment of playing a game with others.

Captain Aaron4 months ago

And kudos it doesn’t pick up the paddle and place it on top of the puck to stop it like every human asshat I’ve ever played.

Correcting the Record with Guillotines ☭4 months ago

I wouldn’t call this guy “skilled,” but is it impossible to score on this thing at all? What’s the point?

William Ruider4 months ago

Wow, I have to say, this is one of the most useful machine learning skills I've seen lately. I experiment with Cartesian robots myself, and I know how much work you've put into this project. Integrating the entire environment, not to mention real-time feedback, hundreds of hours of simulation, and the ability to recognise and respond to simulated and intended movements in real time, a book could be written about it. You've caught my attention, and I'm subscribing to your YouTube channel. Good luck with your future projects.💪💪💪

Rohin Hattiangadi4 months ago

The psychological barrier of seeing that silver piece of metal across the table certainly can’t help things!

DenisQuaalude4 months ago

Not crazy at all. Kinda cool, but fully expected. This ain’t nothing compare to what is coming.

A Horse with No Name4 months ago

Wow, and they also managed to reduce the pixelation from the previous generation!

FocalZero4 months ago

Wow, they beat us at chess, at Go, and now in arcades...

abe kazoo4 months ago

yeah, this kind of computer learning is gonna be a really big deal thirty years ago! I'm excited for the past

HeyGen4 months ago

20 milliseconds reaction time. You are not beating this thing. Ever.

X4 months ago

Nothing new -I played that like 45 years ago.

Rich Tanenbaum4 months ago

The bar is going to stop the puck no matter where it is: there are spots where it shoots when the puck is not against the part moving back and forth on the bar. And if its reaction time is fast it’s been given an unfair advantage.

arXiv Bangers4 months ago

Get fucked humans 😂😂 Banger

The Wayne Shaw4 months ago

Most people fear AI taking their job not realizing AI will just make their job much more brutal to perform…

Yejun Jang4 months ago

I'd like to see a game with the air hockey world champion, whoever that is.

UBC Engineering Physics4 months ago

This was developed by mutiple groups of students in the Engineering Physics Project Lab at University of British Columbia. You can find some more "fascinating" projects on our feed.

PB4 months ago

And they said all that token spend hasn't shown ROI yet.

Brent P4 months ago

I would fucken smoke that robot in air hockey

Dan 𝙏𝙖𝙭𝙖𝙩𝙞𝙤𝙣 𝙄𝙨 𝙏𝙝𝙚𝙛𝙩 Behrman Esq🍍4 months ago

Now make 2 play each other

Nameless investor #V34034 months ago

Wow, another AI breakthrough in ruining the game people used to enjoy. How about something that is actually useful and profitable?

Peter Durfee4 months ago

In related news, we can make a robot travel far faster than Usain Bolt so there's no reason to watch the Olympics any more? Fuck this artificial shit with no connection to human reality

Joshua Hartley4 months ago

I would buy this for $2500 easy. Probably more. Love Air Hockey but too competitive for everyone else. Would play this thing just to keep the reacts quick.

Windy4 months ago

I think you'd call that machine learning not AI

Thomas Cave4 months ago

20 milliseconds: Let a cat fight against him.

Chris Ferry4 months ago

It seems like it’s taking ripper shots from just before half court whereas the human can maybe barely reach that far. Cool project but it shouldn’t be able to reach further than wingspan of opponent for shots.

らく4 months ago

エアホッケーが一番上手い人は、日本のコメディアンのホンジャマカです。その人に勝てたら、すごいと認めます^_^

Related Videos

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.

The coolest part of Microduck might not be the robot. It's how it learns to move. This repo shows the full reinforcement learning pipeline behind Microduck. The robot learns in simulation using MuJoCo, PPO and GPU-based training, then the trained policy is exported to ONNX and deployed on the real robot. But the real challenge is sim-to-real. The simulation models things like actuator behavior, battery voltage, command delays, friction and even gear backlash. That helps make the learned policy work when it moves from simulation to the physical robot. The repo includes policies for walking, fall recovery, sitting, standing, kicking, ground picking and even roller skating. Train in simulation. Test the behavior. Export the policy. Run it on a real robot.

Praveen Kumar Verma

138,017 views • 2 months ago

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

TWIST: a real-time teleoperation system for humanoid robots to mimic whole-body motions. Reference motion data is generated by retargeting human motion-capture data to the robot. Then, the controller is trained in simulation using reinforcement learning and behavior cloning.

The Humanoid Hub

46,715 views • 1 year ago

Real → Sim → Robot — fast. With Gaussian splats, GRID brings physical assets into simulation tools like AirGen and NVIDIA Isaac Sim, enabling AI skills to be tested in sim and deployed to real robots in minutes. #Robotics #PhysicalAI #Simulation

Real → Sim → Robot — fast. With Gaussian splats, GRID brings physical assets into simulation tools like AirGen and NVIDIA Isaac Sim, enabling AI skills to be tested in sim and deployed to real robots in minutes. #Robotics #PhysicalAI #Simulation

General Robotics

12,257 views • 1 year ago

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

The future of robotics isn’t being built in factories, it’s being trained in simulation MasterBOT’s proprietary AI model is learning in real-time, powered by reinforcement and continuous feedback loops This is how machines learn to think $BOT

MasterBOT

51,189 views • 1 year ago

Researchers developed Hitter, a humanoid robot that can play table tennis in real-world settings. It was trained on the Unitree G1 robot to predict the ball’s path, choose the right shot, and react in less than a second. In tests, it achieved 106 consecutive shots with a human and even rallied with another humanoid robot.

Researchers developed Hitter, a humanoid robot that can play table tennis in real-world settings. It was trained on the Unitree G1 robot to predict the ball’s path, choose the right shot, and react in less than a second. In tests, it achieved 106 consecutive shots with a human and even rallied with another humanoid robot.

Space and Technology

25,736 views • 6 months ago

This is a neural network flying a drone at extremely high speed, beating human champions in FPV drone racing. - Reinforcement learning as a tool is so marvelously versatile. It's able to solve both fast, reactive tasks and slow, deliberate tasks (ChatGPT RLHF). - Trained in large-scale simulation, finetuned in real world - I believe this is the paradigm that will get us to generalist robot some day. Published on Nature's cover, "Champion-Level Drone Racing using Deep Reinforcement Learning." Authors: Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun & Davide Scaramuzza

This is a neural network flying a drone at extremely high speed, beating human champions in FPV drone racing. - Reinforcement learning as a tool is so marvelously versatile. It's able to solve both fast, reactive tasks and slow, deliberate tasks (ChatGPT RLHF). - Trained in large-scale simulation, finetuned in real world - I believe this is the paradigm that will get us to generalist robot some day. Published on Nature's cover, "Champion-Level Drone Racing using Deep Reinforcement Learning." Authors: Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun & Davide Scaramuzza

Jim Fan

598,930 views • 3 years ago

ELON: TESLA’S ROBOTS ARE MOVING FROM SIMULATIONS TO REALITY “For the robot, we’re going to need to build a lot of them and put them in an Optimus Academy so they can do self-play in reality and test different tasks. Tesla has quite a good, physics accurate, reality-generator, and we’re doing that for the robots. So you can do millions of robots in a simulated world, and then 10k in the real world to close the simulation to reality gap. Grok would orchestrate the behavior of the Optimus robots.” Source: Cheeky Pint Podcast, Elon Musk

ELON: TESLA’S ROBOTS ARE MOVING FROM SIMULATIONS TO REALITY “For the robot, we’re going to need to build a lot of them and put them in an Optimus Academy so they can do self-play in reality and test different tasks. Tesla has quite a good, physics accurate, reality-generator, and we’re doing that for the robots. So you can do millions of robots in a simulated world, and then 10k in the real world to close the simulation to reality gap. Grok would orchestrate the behavior of the Optimus robots.” Source: Cheeky Pint Podcast, Elon Musk

Mario Nawfal

19,289 views • 8 months ago

This robot learned to walk without ever actually walking in the real world Like in The Matrix (“I know Kung Fu”) it learned quickly just by SIMULATING walking. It practiced "in its head" Ok so what? Do you realize how suddenly our lil chatbots might go from blinking cursors to very real things? Some people assume that we’ll see robots in the real world slowly getting better over years or decades !BUT! if they can learn 10000x faster via simulation, they could they could blow our expectations out of the water. For example, an autonomous killer drone (like the ones already deployed in Ukraine) could get 1,000 years of aiming experience in 1 hour of human time. It would never miss. (More technically, this is a humanoid transformer that was trained with large-scale reinforcement learning in simulation. It was deployed to the real world zero-shot.)

This robot learned to walk without ever actually walking in the real world Like in The Matrix (“I know Kung Fu”) it learned quickly just by SIMULATING walking. It practiced "in its head" Ok so what? Do you realize how suddenly our lil chatbots might go from blinking cursors to very real things? Some people assume that we’ll see robots in the real world slowly getting better over years or decades !BUT! if they can learn 10000x faster via simulation, they could they could blow our expectations out of the water. For example, an autonomous killer drone (like the ones already deployed in Ukraine) could get 1,000 years of aiming experience in 1 hour of human time. It would never miss. (More technically, this is a humanoid transformer that was trained with large-scale reinforcement learning in simulation. It was deployed to the real world zero-shot.)

AI Notkilleveryoneism Memes ⏸️

129,047 views • 2 years ago

Physics simulation gives robots a place to learn before they enter the physical world. 🤖 Developers can model complex interactions, train and evaluate policies, and validate robot behavior in controlled virtual environments before deployment. Explore physics simulation for robotics 🔗

Physics simulation gives robots a place to learn before they enter the physical world. 🤖 Developers can model complex interactions, train and evaluate policies, and validate robot behavior in controlled virtual environments before deployment. Explore physics simulation for robotics 🔗

NVIDIA Robotics

28,742 views • 1 month ago

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

18,813 views • 9 months ago

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado

a16z

35,794 views • 3 months ago

Using our brain simulator, we’ve trained a reinforcement learning agent to maximize bits per second. Here is the RL policy converting brain data to cursor control in simulation:

Using our brain simulator, we’ve trained a reinforcement learning agent to maximize bits per second. Here is the RL policy converting brain data to cursor control in simulation:

Neuralink

18,410 views • 1 year ago

Teaching robots real dexterity has always been a challenge. But what if they could handle tools like a human? DexterityGen (DexGen) is a new system that helps robots use their hands better. It improves how they grip, move, and handle objects… from holding a pen to using a screwdriver. DexGen learns in simulation and refines its skills in the real world, making robotic hands much more useful. What makes DexGen special? ✅ Smarter movements that refine rough actions into precise skills ✅ Trained on a massive collection of dexterous tasks for better learning ✅ Better teleoperation that makes robotic hand control easier and safer ✅ Handles real-world challenges like small objects, tricky angles, and gravity This moves robots closer to real dexterity. It makes tool use more natural, improves stability, and brings robotic hands one step closer to human-level skill. Seen at Zhao-Heng Yin 🫶 Github: Paper:

Teaching robots real dexterity has always been a challenge. But what if they could handle tools like a human? DexterityGen (DexGen) is a new system that helps robots use their hands better. It improves how they grip, move, and handle objects… from holding a pen to using a screwdriver. DexGen learns in simulation and refines its skills in the real world, making robotic hands much more useful. What makes DexGen special? ✅ Smarter movements that refine rough actions into precise skills ✅ Trained on a massive collection of dexterous tasks for better learning ✅ Better teleoperation that makes robotic hand control easier and safer ✅ Handles real-world challenges like small objects, tricky angles, and gravity This moves robots closer to real dexterity. It makes tool use more natural, improves stability, and brings robotic hands one step closer to human-level skill. Seen at Zhao-Heng Yin 🫶 Github: Paper:

Ilir Aliu

51,526 views • 1 year ago

Experiments in progress. The one on the right has been learning for ~3 hours, the one in the middle for ~1 hour, and the one on the left just started a few minutes ago. The initial motivation for making the physical Atari was just to commit ourselves to a subset of algorithms that can make progress in this setup. This commitment rules out algorithms that require billions of samples to learn (or worse, require multiple environments running in parallel). Atari games are simple enough that we should be able to show learning on them in a short amount of time with no prior knowledge. Since then, I've realized that this setup is also a good way to compare different paradigms in robotics in a principled way. These paradigms are sim2real, learning from tele-operated data, and learning directly on the robots. So far, I have observed that getting sim2real to work reliably is hard. It requires tweaks that don't scale. Policies that can play perfectly in simulation fall apart because of latencies and the messiness of the real world. These aspects could be modeled to improve the simulation, but not without sinking significant human engineering hours. I have higher hopes for learning from tele-operated data, but that requires a human to learn the task first. These experiments are on my to-do list. I have to learn to play some of the games well through the robot. I’m half-decent at playing Pong and Ms Pacman now. Learning directly on robots is looking like the most promising approach. This approach takes away pesky distribution shifts and makes it possible to have algorithms that continually improve with more data and time without any human intervention. It feels great to let experiments run overnight and wake up to find improved policies. With learning on robots, I should, in principle, be able to go on a long vacation and come back to find better policies for complex tasks beyond Atari games. Whether that is possible with current learning algorithms is a different question.

Experiments in progress. The one on the right has been learning for ~3 hours, the one in the middle for ~1 hour, and the one on the left just started a few minutes ago. The initial motivation for making the physical Atari was just to commit ourselves to a subset of algorithms that can make progress in this setup. This commitment rules out algorithms that require billions of samples to learn (or worse, require multiple environments running in parallel). Atari games are simple enough that we should be able to show learning on them in a short amount of time with no prior knowledge. Since then, I've realized that this setup is also a good way to compare different paradigms in robotics in a principled way. These paradigms are sim2real, learning from tele-operated data, and learning directly on the robots. So far, I have observed that getting sim2real to work reliably is hard. It requires tweaks that don't scale. Policies that can play perfectly in simulation fall apart because of latencies and the messiness of the real world. These aspects could be modeled to improve the simulation, but not without sinking significant human engineering hours. I have higher hopes for learning from tele-operated data, but that requires a human to learn the task first. These experiments are on my to-do list. I have to learn to play some of the games well through the robot. I’m half-decent at playing Pong and Ms Pacman now. Learning directly on robots is looking like the most promising approach. This approach takes away pesky distribution shifts and makes it possible to have algorithms that continually improve with more data and time without any human intervention. It feels great to let experiments run overnight and wake up to find improved policies. With learning on robots, I should, in principle, be able to go on a long vacation and come back to find better policies for complex tasks beyond Atari games. Whether that is possible with current learning algorithms is a different question.

Khurram Javed

52,110 views • 10 months ago

World Labs co-founder Justin Johnson predicts a real-to-sim-to-real future where Atlas could train a robot for any new environment in minutes using just 5 photos from your phone: "One is this notion of fully in-context learning. Maybe I've got a robotics foundation model, then I can demonstrate the robot once how a task should be performed, and that's enough for the robot to figure out." "Another version is real to sim to real. Maybe I've got my pre-trained robotics foundation model, but I want to adapt it to this particular environment, like this studio, this table, moving this microphone around." "There's a version where you could come in here and take just a couple casual videos with your phone or a couple images with your phone, use Atlas to reconstruct the space, and now stage all kinds of robotics interactions in this studio space." "If you could lower the barrier to doing that, you could do it in five minutes to take five photos, upload it to Atlas, have it generate a simulation, maybe describe in natural language what kind of task you want the robot to do, have an agent go build that simulation for you, RL fine-tune your general purpose robotics foundation model, and now you've got a robot maybe in the span of a couple minutes that could come in and is perfectly adapted to this space." World Labs

World Labs co-founder Justin Johnson predicts a real-to-sim-to-real future where Atlas could train a robot for any new environment in minutes using just 5 photos from your phone: "One is this notion of fully in-context learning. Maybe I've got a robotics foundation model, then I can demonstrate the robot once how a task should be performed, and that's enough for the robot to figure out." "Another version is real to sim to real. Maybe I've got my pre-trained robotics foundation model, but I want to adapt it to this particular environment, like this studio, this table, moving this microphone around." "There's a version where you could come in here and take just a couple casual videos with your phone or a couple images with your phone, use Atlas to reconstruct the space, and now stage all kinds of robotics interactions in this studio space." "If you could lower the barrier to doing that, you could do it in five minutes to take five photos, upload it to Atlas, have it generate a simulation, maybe describe in natural language what kind of task you want the robot to do, have an agent go build that simulation for you, RL fine-tune your general purpose robotics foundation model, and now you've got a robot maybe in the span of a couple minutes that could come in and is perfectly adapted to this space." World Labs

MTS

25,204 views • 1 month ago

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,867 views • 4 months ago

Watch this robot dog learn to walk from scratch in real time! Our new method, APRL, dynamically adjusts exploration constraints to enable fast and performant RL directly in the real world. APRL can also adapt to changes in the terrain. No simulation, no demos. A thread 👇

Watch this robot dog learn to walk from scratch in real time! Our new method, APRL, dynamically adjusts exploration constraints to enable fast and performant RL directly in the real world. APRL can also adapt to changes in the terrain. No simulation, no demos. A thread 👇

Sergey Levine

105,579 views • 3 years ago