Загрузка видео...

Не удалось загрузить видео

На главную

A video from a Pi robot deployed at Dandelion Chocolate, fully autonomous w/ no interventions. 🤖 Deploying robots has taught us surprising lessons about the gap between proof-of-concept (i.e. building one box) and real-world utility (productively building boxes for hours). I expected that the hard part is building the...

chelseabfinn's profile picture

Chelsea Finn

103,942 subscribers

104,016 просмотров • 18 дней назад •via X (Twitter)

Наука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 12

Фото профиля ayush
ayush18 дней назад

@physical_int oh so thats what this guy was up to

Фото профиля Will Bright
Will Bright18 дней назад

@physical_int @ASong408 What'd I tell you about CPG?

Фото профиля ethereagle · building
ethereagle · building18 дней назад

@physical_int the clip is zero-intervention, which is the easy number. the one that decides is what failed at minute 40 that minute 2 never saw. was it grasp retry pileup, or perception drift on the chocolate?

Фото профиля tab
tab18 дней назад

@physical_int Seeing it in a real setting like this, applications keep popping into my head.

Фото профиля Jakie PLA
Jakie PLA18 дней назад

@physical_int I could not help watching this on repeat. THE STACK is the hard part. Hours with zero babysitting beats one perfect pick :)

Фото профиля Senthilnathan K
Senthilnathan K18 дней назад

@physical_int The delayed collapse is the part demos hide, a placement can look fine and still be a bad policy because the cost only shows up ten boxes later. It’s more of a leftover-state problem than a dexterity gap.

Фото профиля Arthur Petron
Arthur Petron18 дней назад

@physical_int Awesome.

Фото профиля Porvesh Balasubramanian
Porvesh Balasubramanian18 дней назад

@physical_int Awesome!

Фото профиля Alex Kennberg
Alex Kennberg18 дней назад

@ZeYanjie @physical_int Dandelion Chocolate boxes are nice.

Фото профиля Paige
Paige18 дней назад

@physical_int Would be cool to see fine tuned Pi models in the wild. What other sites are you interested in?

Фото профиля RealMan Robotics
RealMan Robotics18 дней назад

@physical_int This is the real benchmark for deployment: not whether a robot can complete the task once, but whether it can keep doing it reliably for hours with no intervention. That gap between demo and utility is where most of the hard work lives.

Фото профиля Leo Lu
Leo Lu18 дней назад

@physical_int Robotics stops being a demo and starts becoming a product.

Похожие видео

Excited to announce Tau Robotics (Tau Robotics). We are building a general AI for robots. We start by building millions of robot arms that learn in the real world. In the video, two robot arms are fully autonomous and controlled by a single neural network conditioned on different language instructions (four axes and five axes robot arms). The other two arms are teleoperated. The entire hardware cost in the video is about $1400. The video is at 1.5x speed.
0:39

Excited to announce Tau Robotics (Tau Robotics). We are building a general AI for robots. We start by building millions of robot arms that learn in the real world. In the video, two robot arms are fully autonomous and controlled by a single neural network conditioned on different language instructions (four axes and five axes robot arms). The other two arms are teleoperated. The entire hardware cost in the video is about $1400. The video is at 1.5x speed.

Alexander Koch

441,716 просмотров • 2 лет назад

.Jason Ma on how Dyna-2 learned to untwist a real water bottle cap out of the box from just 13 minutes of dexterous hand data because the robot hand looks like a human's: "We got a new pair of dexterous hand. Most of our robots in the office have a gripper-like hand, so the gap between that versus human is actually quite large." "We thought, if we have a robot that has hands that look like human, wouldn't the transfer be greater? Right before the release, they were able to collect just 13 minutes of data." "They collected only 13 minutes of data and then post-trained the best Dyna model that trained on one million hours of data. The model is just able to, out of the box, start opening bottle caps on real water bottles." "This is very surprising because typically post-training a robot would take at least several hours of data. For a dexterous hand that's much more complex than a gripper, it would actually be much harder to train. But the experimental results we saw was the opposite. Perhaps because the hand looks more like the human, the transfer was easier." Dyna Robotics
1:16

.Jason Ma on how Dyna-2 learned to untwist a real water bottle cap out of the box from just 13 minutes of dexterous hand data because the robot hand looks like a human's: "We got a new pair of dexterous hand. Most of our robots in the office have a gripper-like hand, so the gap between that versus human is actually quite large." "We thought, if we have a robot that has hands that look like human, wouldn't the transfer be greater? Right before the release, they were able to collect just 13 minutes of data." "They collected only 13 minutes of data and then post-trained the best Dyna model that trained on one million hours of data. The model is just able to, out of the box, start opening bottle caps on real water bottles." "This is very surprising because typically post-training a robot would take at least several hours of data. For a dexterous hand that's much more complex than a gripper, it would actually be much harder to train. But the experimental results we saw was the opposite. Perhaps because the hand looks more like the human, the transfer was easier." Dyna Robotics

MTS

13,743 просмотров • 1 месяц назад

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics
1:14

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

MTS

17,740 просмотров • 1 месяц назад

Modern AI is confined to the digital world. At Skild AI, we are building towards AGI for the real world, unconstrained by robot type or task — a single, omni-bodied brain. Today, we are sharing our journey, starting with early milestones, with more to come in the weeks ahead. Our Mission: Artificial General Intelligence grounded in the physical world. We believe AGI that can truly understand and reason in the real world can only be built through grounding in the physical world. Our Vision: Any robot, Any task, One brain. We tackle robotics in its full generality – building a continually improving, omni-bodied brain that can control any hardware for any task. Who are we? A passionate group of scientists & engineers driven by our shared vision. We have been researching AI and robotics for more than a decade. Our team includes pioneers of self-supervised learning, curiosity-driven exploration, end-to-end sim2real for visual locomotion, dexterous manipulation, learning from human videos, robot parkour, and many more. Many of these works have won awards at top-tier AI and Robotics conferences. Our team has also built production-ready systems at Anduril, Tesla, Nvidia, Meta, Kitty Hawk, Google, Everyday Robotics, and Amazon. Join us in our mission to build the robot brains of tomorrow.
2:48

Modern AI is confined to the digital world. At Skild AI, we are building towards AGI for the real world, unconstrained by robot type or task — a single, omni-bodied brain. Today, we are sharing our journey, starting with early milestones, with more to come in the weeks ahead. Our Mission: Artificial General Intelligence grounded in the physical world. We believe AGI that can truly understand and reason in the real world can only be built through grounding in the physical world. Our Vision: Any robot, Any task, One brain. We tackle robotics in its full generality – building a continually improving, omni-bodied brain that can control any hardware for any task. Who are we? A passionate group of scientists & engineers driven by our shared vision. We have been researching AI and robotics for more than a decade. Our team includes pioneers of self-supervised learning, curiosity-driven exploration, end-to-end sim2real for visual locomotion, dexterous manipulation, learning from human videos, robot parkour, and many more. Many of these works have won awards at top-tier AI and Robotics conferences. Our team has also built production-ready systems at Anduril, Tesla, Nvidia, Meta, Kitty Hawk, Google, Everyday Robotics, and Amazon. Join us in our mission to build the robot brains of tomorrow.

Skild AI

386,301 просмотров • 1 год назад

Today we're incredibly excited to announce Dreamscale Labs (YC F26) (YC F26)! We're an applied research company putting robot brains in the cloud. Robot AI models are growing larger and on-board compute is getting increasingly inaccessible due to cost. We're taking a contrarian bet that the future is moving inference off the robot and into the cloud, while still meeting real-time deadlines required by robot motion. We're building the real-time inference infrastructure that enables frontier physical AI models to run smoothly on your robot. We optimize for low latency across the full stack: from the model, down to the GPU kernels, and over the network. Our mission is to unlock a future of safe and intelligent robots doing meaningful work, and we think this is only possible by letting you deploy and run robot models in the cloud. Reach out to us at founders@dreamscalelabs.com or visit our website at to learn more. Kevin L Antoine Nguyen Eric R
1:50

Today we're incredibly excited to announce Dreamscale Labs (YC F26) (YC F26)! We're an applied research company putting robot brains in the cloud. Robot AI models are growing larger and on-board compute is getting increasingly inaccessible due to cost. We're taking a contrarian bet that the future is moving inference off the robot and into the cloud, while still meeting real-time deadlines required by robot motion. We're building the real-time inference infrastructure that enables frontier physical AI models to run smoothly on your robot. We optimize for low latency across the full stack: from the model, down to the GPU kernels, and over the network. Our mission is to unlock a future of safe and intelligent robots doing meaningful work, and we think this is only possible by letting you deploy and run robot models in the cloud. Reach out to us at [email protected] or visit our website at to learn more. Kevin L Antoine Nguyen Eric R

Chris Yoo

48,099 просмотров • 15 дней назад

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot generalization on a wide range of tasks. Humanoid company 1X has a solution: world models. 1X Director of Evaluations Daniel Ho joins us on RoboPapers to talk about: - why world models are the future for scaling robot learning - how to use world models for robot control - what world models unlock for evaluating robot model performance - how we can hill-climb from here to general purpose robots Watch Episode #61 of RoboPapers, with Michael Cho - Rbt/Acc and Chris Paxton, now!
54:52

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot generalization on a wide range of tasks. Humanoid company 1X has a solution: world models. 1X Director of Evaluations Daniel Ho joins us on RoboPapers to talk about: - why world models are the future for scaling robot learning - how to use world models for robot control - what world models unlock for evaluating robot model performance - how we can hill-climb from here to general purpose robots Watch Episode #61 of RoboPapers, with Michael Cho - Rbt/Acc and Chris Paxton, now!

RoboPapers

27,945 просмотров • 8 месяцев назад

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:
1:17

We trained a humanoid with 22-DoF dexterous hands to assemble model cars, operate syringes, sort poker cards, fold/roll shirts, all learned primarily from 20,000+ hours of egocentric human video with no robot in the loop. Humans are the most scalable embodiment on the planet. We discovered a near-perfect log-linear scaling law (R² = 0.998) between human video volume and action prediction loss, and this loss directly predicts real-robot success rate. Humanoid robots will be the end game, because they are the practical form factor with minimal embodiment gap from humans. Call it the Bitter Lesson of robot hardware: the kinematic similarity lets us simply retarget human finger motion onto dexterous robot hand joints. No learned embeddings, no fancy transfer algorithms needed. Relative wrist motion + retargeted 22-DoF finger actions serve as a unified action space that carries through from pre-training to robot execution. Our recipe is called "EgoScale": - Pre-train GR00T N1.5 on 20K hours of human video, mid-train with only 4 hours (!) of robot play data with Sharpa hands. 54% gains over training from scratch across 5 highly dexterous tasks. - Most surprising result: a *single* teleop demo is sufficient to learn a never-before-seen task. Our recipe enables extreme data efficiency. - Although we pre-train in 22-DoF hand joint space, the policy transfers to a Unitree G1 with 7-DoF tri-finger hands. 30%+ gains over training on G1 data alone. The scalable path to robot dexterity was never more robots. It was always us. Deep dives in thread:

Jim Fan

301,632 просмотров • 7 месяцев назад

JUST IN: Dyna Robotics just published one of the most important research papers in robotics this year. It could fundamentally change how robot foundation models are trained. A scaling law that transfers from human video to robot performance. Dyna-2 is out and it's 🔥 Here's what that means in plain terms. Dyna-2 was pre-trained on ONE MILLION hours of egocentric human video, 170 years of continuous human experience, cooking, folding, assembling, cleaning. And as that human data scaled, robot performance improved. Predictably. Monotonically. Across 39 tasks on two different robot embodiments the model had never seen. → 1,000 hours pre-training → 20% normalised task performance → 10,000 hours → 28% → 100,000 hours → 45% → 1,000,000 hours → 53% Human video exists at effectively unlimited scale. Every cook, every factory worker, every craftsperson wearing a camera is generating training data for future robots. But the finding that stunned even the researchers, world modeling is what makes the transfer work. A model trained to predict future video AND actions massively outperforms one trained on actions alone. Video is the new scaling axis for robotics. One more jaw-dropping data point. 13 minutes of teleoperation data was enough to fine-tune Dyna-2 to open a bottle cap using two five-fingered robot hands. The robots are coming, and they're learning from us directly :D Read more here: Congrats Jason Ma and team! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
1:35

JUST IN: Dyna Robotics just published one of the most important research papers in robotics this year. It could fundamentally change how robot foundation models are trained. A scaling law that transfers from human video to robot performance. Dyna-2 is out and it's 🔥 Here's what that means in plain terms. Dyna-2 was pre-trained on ONE MILLION hours of egocentric human video, 170 years of continuous human experience, cooking, folding, assembling, cleaning. And as that human data scaled, robot performance improved. Predictably. Monotonically. Across 39 tasks on two different robot embodiments the model had never seen. → 1,000 hours pre-training → 20% normalised task performance → 10,000 hours → 28% → 100,000 hours → 45% → 1,000,000 hours → 53% Human video exists at effectively unlimited scale. Every cook, every factory worker, every craftsperson wearing a camera is generating training data for future robots. But the finding that stunned even the researchers, world modeling is what makes the transfer work. A model trained to predict future video AND actions massively outperforms one trained on actions alone. Video is the new scaling axis for robotics. One more jaw-dropping data point. 13 minutes of teleoperation data was enough to fine-tune Dyna-2 to open a bottle cap using two five-fingered robot hands. The robots are coming, and they're learning from us directly :D Read more here: Congrats Jason Ma and team! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

23,681 просмотров • 1 месяц назад

What if a dexterous robot could learn a 20+ step chemistry experiment without a single on-robot training demo? Meet TwinDEX: a pair of co-designed, three-finger, nine-DoF dexterous manipulation interface: one wearable for data collection, one for robot deployment. The twinned design shares identical kinematics, contact surfaces, visual appearance, and sensors across collection and deployment — keeping observations and actions aligned end to end. Trained from scratch on only a few hundred wearable demonstrations - with zero on-robot training or intervention data - TwinDEX completed a standardized chemistry experiment involving tool switches, fine force control, and bimanual coordination. Robot-free data showed comparable learning efficiency on the multi-task evaluation, TwinDEX delivered 5.3 times effective throughput than on-robot teleoperation. TwinDEX demonstrates that high-quality robot-free data can fully substitute for on-robot teleoperation data on challenging dexterous tasks — removing the dependency on real-robot hardware that has been the central bottleneck to scaling dexterous manipulation data. This was the proof-it phase. Now comes scale: what emerges at tens of thousands, or millions, of episodes? Watch the demo and read the technical blog: #TwinDEX #Robotics #EmbodiedAI #DexterousManipulation
4:31

What if a dexterous robot could learn a 20+ step chemistry experiment without a single on-robot training demo? Meet TwinDEX: a pair of co-designed, three-finger, nine-DoF dexterous manipulation interface: one wearable for data collection, one for robot deployment. The twinned design shares identical kinematics, contact surfaces, visual appearance, and sensors across collection and deployment — keeping observations and actions aligned end to end. Trained from scratch on only a few hundred wearable demonstrations - with zero on-robot training or intervention data - TwinDEX completed a standardized chemistry experiment involving tool switches, fine force control, and bimanual coordination. Robot-free data showed comparable learning efficiency on the multi-task evaluation, TwinDEX delivered 5.3 times effective throughput than on-robot teleoperation. TwinDEX demonstrates that high-quality robot-free data can fully substitute for on-robot teleoperation data on challenging dexterous tasks — removing the dependency on real-robot hardware that has been the central bottleneck to scaling dexterous manipulation data. This was the proof-it phase. Now comes scale: what emerges at tens of thousands, or millions, of episodes? Watch the demo and read the technical blog: #TwinDEX #Robotics #EmbodiedAI #DexterousManipulation

X Square Robot

1,346,282 просмотров • 1 месяц назад

Atlas is huge for real-to-sim. Scanning spaces like this usually requires tons of equipment, but Atlas can reconstruct the environment from just a couple pictures. We then simulate robots navigating through the generated world. Atlas generates what the robot would see from the perspective of its cameras. Even depth. It generates the environment AND the training data from it...
0:55

Atlas is huge for real-to-sim. Scanning spaces like this usually requires tons of equipment, but Atlas can reconstruct the environment from just a couple pictures. We then simulate robots navigating through the generated world. Atlas generates what the robot would see from the perspective of its cameras. Even depth. It generates the environment AND the training data from it...

David Pantera

13,704 просмотров • 29 дней назад

A jaw-on-the-floor moment for the whole team was when we had taught the robot to pick up a baggie and shake a usb-brick out with the left hand. During one rollout, it decided to do it with the right hand—and we all stopped. We were stunned, because no one taught it to do that. This was one of the first models with our pretrained base, and it was the moment we realized there was something here that was very different from what any of us had ever built before. We now know this to be part of the improvisational intelligence that emerges with large-scale pretraining on real manipulation data, and it continues to be one of the things that we believe to be worth pushing forward as capability. Our CTO and cofounder Andy Barry joined Association for Advancing Automation to share more about these moments, and how his time at Boston Dynamics and the Broad Institute drove him to focus on building the 🧠 for robots at Generalist. Full ep 👇
0:28

A jaw-on-the-floor moment for the whole team was when we had taught the robot to pick up a baggie and shake a usb-brick out with the left hand. During one rollout, it decided to do it with the right hand—and we all stopped. We were stunned, because no one taught it to do that. This was one of the first models with our pretrained base, and it was the moment we realized there was something here that was very different from what any of us had ever built before. We now know this to be part of the improvisational intelligence that emerges with large-scale pretraining on real manipulation data, and it continues to be one of the things that we believe to be worth pushing forward as capability. Our CTO and cofounder Andy Barry joined Association for Advancing Automation to share more about these moments, and how his time at Boston Dynamics and the Broad Institute drove him to focus on building the 🧠 for robots at Generalist. Full ep 👇

Generalist

11,117 просмотров • 3 месяцев назад

📢 Announcing one of the most exciting works from us this year on **scalable robot policy evaluation through real-to-sim transfer**, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is **orders of magnitude slower** than the development of language agents. We propose a new framework where simulation performance **strongly correlates** with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing **state-of-the-art correlation** between simulation and reality for deformable object manipulation. It provides a **scalable and reproducible evaluation engine** for robot learning. 🌐
3:04

📢 Announcing one of the most exciting works from us this year on **scalable robot policy evaluation through real-to-sim transfer**, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is **orders of magnitude slower** than the development of language agents. We propose a new framework where simulation performance **strongly correlates** with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing **state-of-the-art correlation** between simulation and reality for deformable object manipulation. It provides a **scalable and reproducible evaluation engine** for robot learning. 🌐

Yunzhu Li

39,960 просмотров • 11 месяцев назад

Foundation models are enough to solve robotics! Unfortunately, this is not true. We keep hearing that Vision-Language-Action (VLA) models struggle because of the gap between static training and the dynamic real world. A German startup (Sereact) just released a solution that bridges this gap perfectly. They are introducing a new paradigm called Interactive RL Policy Patching. It's a distributed framework that allows robots to self-learn from human corrections without needing full retraining. When a robot fails, a human operator provides a brief "patch" or demonstration. The system then uses online off-policy reinforcement learning to update the behavior instantly. This is powered by a massive foundation model trained on hundreds of millions of interactions from over 100 deployed robot stations. The best part is the distributed parameter synchronization... When one robot learns a fix, the update is published fleet-wide... meaning the entire swarm gets smarter from a single human intervention. They are already proving this on complex manipulation tasks like shoe unboxing and screw sorting, drastically reducing the data needed to handle edge cases. Real-world environments are unforgiving, and I love seeing systems that can actually adapt on the fly! 📍 More info:
0:38

Foundation models are enough to solve robotics! Unfortunately, this is not true. We keep hearing that Vision-Language-Action (VLA) models struggle because of the gap between static training and the dynamic real world. A German startup (Sereact) just released a solution that bridges this gap perfectly. They are introducing a new paradigm called Interactive RL Policy Patching. It's a distributed framework that allows robots to self-learn from human corrections without needing full retraining. When a robot fails, a human operator provides a brief "patch" or demonstration. The system then uses online off-policy reinforcement learning to update the behavior instantly. This is powered by a massive foundation model trained on hundreds of millions of interactions from over 100 deployed robot stations. The best part is the distributed parameter synchronization... When one robot learns a fix, the update is published fleet-wide... meaning the entire swarm gets smarter from a single human intervention. They are already proving this on complex manipulation tasks like shoe unboxing and screw sorting, drastically reducing the data needed to handle edge cases. Real-world environments are unforgiving, and I love seeing systems that can actually adapt on the fly! 📍 More info:

Ilir Aliu

18,210 просмотров • 8 месяцев назад

We RL'ed humanoid robots to Cristiano Ronaldo, LeBron James, and Kobe Byrant! These are neural nets running on real hardware at our GEAR lab. Most robot demos you see online speed videos up. We actually *slow them down* so you can enjoy the fluid motions. I'm excited to announce "ASAP", a "real2sim2real" model that masters extremely smooth and dynamic motions for humanoid whole body control. We pretrain the robot in simulation first, but there is a notorious "sim2real" gap: it's very difficult for hand-engineered physics equations to match real world dynamics. Our fix is simple: just deploy a pretrained policy on real hardware, collect data, and replay the motion in sim. The replay will obviously have many errors, but that gives a rich signal to compensate for the physics discrepancy. Use another neural net to learn the delta. Basically, we "patch up" a traditional physics engine, so that the robot can experience almost the real world at scale in GPUs. The future is hybrid simulation: combine the power of classical sim engines refined over decades and the uncanny ability of modern NNs to capture a messy world.
0:50

We RL'ed humanoid robots to Cristiano Ronaldo, LeBron James, and Kobe Byrant! These are neural nets running on real hardware at our GEAR lab. Most robot demos you see online speed videos up. We actually *slow them down* so you can enjoy the fluid motions. I'm excited to announce "ASAP", a "real2sim2real" model that masters extremely smooth and dynamic motions for humanoid whole body control. We pretrain the robot in simulation first, but there is a notorious "sim2real" gap: it's very difficult for hand-engineered physics equations to match real world dynamics. Our fix is simple: just deploy a pretrained policy on real hardware, collect data, and replay the motion in sim. The replay will obviously have many errors, but that gives a rich signal to compensate for the physics discrepancy. Use another neural net to learn the delta. Basically, we "patch up" a traditional physics engine, so that the robot can experience almost the real world at scale in GPUs. The future is hybrid simulation: combine the power of classical sim engines refined over decades and the uncanny ability of modern NNs to capture a messy world.

Jim Fan

567,404 просмотров • 1 год назад

SkildAI, which is developing a foundational AI model for robots, has released a new video of their humanoid robot walking over objects and exploring environments it had never seen before. The robot is using vision & adapts in real time. SkildAI: "From raw images and joint feedback, the model directly outputs low-level motor commands. This single neural network enables humanoid robots to seamlessly walk across flat ground, climb stairs, and step over obstacles without any planning, mapping, or manual switching between behaviors. For all of our testing in this video, we deployed the robot in each new environment without any prior planning or mapping. We took off the humanoid’s shoes so you can hear it.
2:49

SkildAI, which is developing a foundational AI model for robots, has released a new video of their humanoid robot walking over objects and exploring environments it had never seen before. The robot is using vision & adapts in real time. SkildAI: "From raw images and joint feedback, the model directly outputs low-level motor commands. This single neural network enables humanoid robots to seamlessly walk across flat ground, climb stairs, and step over obstacles without any planning, mapping, or manual switching between behaviors. For all of our testing in this video, we deployed the robot in each new environment without any prior planning or mapping. We took off the humanoid’s shoes so you can hear it.

Sawyer Merritt

69,381 просмотров • 1 год назад

We got Dagsen Steele Love 盧天子 a $53,000 humanoid robot from America’s first humanoid robot store, then challenged him to turn it into an autonomous being in just 12 hours. Thank you REK and Dimensional for your help! 00:00 Humanoids are everywhere, how hard are they to use? 00:57 Meeting a robot operating system developer 02:12 What can a Unitree humanoid robot do straight out of the box? 03:25 Visiting America’s first humanoid robot store 04:11 Unpacking the robot 04:50 Taking the robot for a run 05:22 Programming the robot 06:31 Calling in engineering experts 07:15 Speaking Chinese with a robot 07:27 The history of humanoid robots 08:40 A breakthrough 09:09 Robot vs obstacle course
10:55

We got Dagsen Steele Love 盧天子 a $53,000 humanoid robot from America’s first humanoid robot store, then challenged him to turn it into an autonomous being in just 12 hours. Thank you REK and Dimensional for your help! 00:00 Humanoids are everywhere, how hard are they to use? 00:57 Meeting a robot operating system developer 02:12 What can a Unitree humanoid robot do straight out of the box? 03:25 Visiting America’s first humanoid robot store 04:11 Unpacking the robot 04:50 Taking the robot for a run 05:22 Programming the robot 06:31 Calling in engineering experts 07:15 Speaking Chinese with a robot 07:27 The history of humanoid robots 08:40 A breakthrough 09:09 Robot vs obstacle course

Soon

24,892 просмотров • 4 месяцев назад

Robots from Humanoid passed real factory test at Siemens! 😮‍💨 Humanoid from the UK just completed a successful proof of concept with Siemens, deploying their HMND 01 wheeled humanoid robot in a real working factory. The task was simple but important: pick totes from a storage stack, transport them to a conveyor, and place them at the pickup point for human operators. Repeat until the stack is empty. The robot achieved 60 tote moves per hour, handled two different tote sizes, ran autonomously for more than 30 minutes at a time, stayed operational for over eight hours, and maintained over 90% success rate on pick and place tasks. This matters because it's not a demo video. It's a real deployment in Siemens' Electronics Factory in Erlangen, handling actual production work, measured against real operational metrics. The test ran in two phases. First, Humanoid built a physical twin in-house to test and optimize the system. Then they deployed on-site at Siemens for two weeks, where the robot operated in the real production environment. We can see a wheeled humanoid robot in action! So, now the ultimate question: wheels or legs? 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:26

Robots from Humanoid passed real factory test at Siemens! 😮‍💨 Humanoid from the UK just completed a successful proof of concept with Siemens, deploying their HMND 01 wheeled humanoid robot in a real working factory. The task was simple but important: pick totes from a storage stack, transport them to a conveyor, and place them at the pickup point for human operators. Repeat until the stack is empty. The robot achieved 60 tote moves per hour, handled two different tote sizes, ran autonomously for more than 30 minutes at a time, stayed operational for over eight hours, and maintained over 90% success rate on pick and place tasks. This matters because it's not a demo video. It's a real deployment in Siemens' Electronics Factory in Erlangen, handling actual production work, measured against real operational metrics. The test ran in two phases. First, Humanoid built a physical twin in-house to test and optimize the system. Then they deployed on-site at Siemens for two weeks, where the robot operated in the real production environment. We can see a wheeled humanoid robot in action! So, now the ultimate question: wheels or legs? 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

21,993 просмотров • 8 месяцев назад

Elon Musk: Tesla’s Mega Pods Could Transform AI Computing Everywhere We're also building out a mega pod design that has — because, for computers with x86, it's a very x86 computer with the Tesla AI computer. You know, it's got this visual Optimus in a box, a mega pod kind of like the Megapack packaging. So, we put a large number of four-plus x86 combos into a giant box, essentially. And these boxes can be placed anywhere in the country or outside the country. This allows us to scale AI compute using disaggregated electricity production. Again, there are lots of places all around the world, including Superchargers. I think we get something like seven gigawatts of power at our Superchargers. And we can place mega pods at many of these Superchargers and have distributed power for AI. So, I think that could be quite a big deal.
1:23

Elon Musk: Tesla’s Mega Pods Could Transform AI Computing Everywhere We're also building out a mega pod design that has — because, for computers with x86, it's a very x86 computer with the Tesla AI computer. You know, it's got this visual Optimus in a box, a mega pod kind of like the Megapack packaging. So, we put a large number of four-plus x86 combos into a giant box, essentially. And these boxes can be placed anywhere in the country or outside the country. This allows us to scale AI compute using disaggregated electricity production. Again, there are lots of places all around the world, including Superchargers. I think we get something like seven gigawatts of power at our Superchargers. And we can place mega pods at many of these Superchargers and have distributed power for AI. So, I think that could be quite a big deal.

Tesla Owners Silicon Valley

23,013 просмотров • 2 месяцев назад

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,772 просмотров • 3 месяцев назад