Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Robotics has a massive, silent bottleneck. It isn’t just data collection—it’s the brutal 1x speed of the physical world. Genesis AI Genesis AI just unveiled Genesis World 1.0, and they are attempting to turn the notorious Sim2Real gap into a pure compute problem. Evaluating a robotics foundation model across...

humanoidsdaily's profile picture

Humanoids daily

10,446 subscribers

17,228 Aufrufe • vor 1 Monat •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

That's sick! 🤯 Genesis AI simulates robots playing yo-yo! 🪀 Genesis AI just open-sourced Genesis World 1.0, and it might be one of the most important infrastructure releases in robotics this year. Robotics is still bottlenecked by the 1× speed of the physical world. Every model needs to be tested on real hardware, slowly, expensively, with limited coverage. Genesis World 1.0 from Genesis AI flips that equation: One hour in reality becomes 100 days in simulation. That turns a wall-clock bottleneck into a compute problem. And compute problems are solvable. The technical stack they rebuilt from scratch is serious: → GPU-accelerated cross-platform compiler via Quadrants, 10x faster launch time and up to 4.6x runtime vs the initial Genesis release → Penetration-free multi-physics contact solvers, the thing that makes simulation actually trustworthy → Unified rigid AND deformable physics in a single engine → Nyx, a high-performance path-traced rendering engine purpose-built for physical AI The sim-to-real gap has historically been the graveyard of robotics research. Policies that work beautifully in simulation fall apart on real hardware. Genesis World 1.0 is a direct attack on that problem. And it's fully open-source. The companies that master simulation infrastructure will train better robots faster than anyone else. Find it here: Genesis World 1.0: Quadrants: Nyx: Theophile Gervet, Zhou Xian congrats! 👏🏼 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:46

That's sick! 🤯 Genesis AI simulates robots playing yo-yo! 🪀 Genesis AI just open-sourced Genesis World 1.0, and it might be one of the most important infrastructure releases in robotics this year. Robotics is still bottlenecked by the 1× speed of the physical world. Every model needs to be tested on real hardware, slowly, expensively, with limited coverage. Genesis World 1.0 from Genesis AI flips that equation: One hour in reality becomes 100 days in simulation. That turns a wall-clock bottleneck into a compute problem. And compute problems are solvable. The technical stack they rebuilt from scratch is serious: → GPU-accelerated cross-platform compiler via Quadrants, 10x faster launch time and up to 4.6x runtime vs the initial Genesis release → Penetration-free multi-physics contact solvers, the thing that makes simulation actually trustworthy → Unified rigid AND deformable physics in a single engine → Nyx, a high-performance path-traced rendering engine purpose-built for physical AI The sim-to-real gap has historically been the graveyard of robotics research. Policies that work beautifully in simulation fall apart on real hardware. Genesis World 1.0 is a direct attack on that problem. And it's fully open-source. The companies that master simulation infrastructure will train better robots faster than anyone else. Find it here: Genesis World 1.0: Quadrants: Nyx: Theophile Gervet, Zhou Xian congrats! 👏🏼 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

36,767 Aufrufe • vor 1 Monat

We are back again :) After three weeks of quiet building. Introducing Genesis World 1.0, our latest simulation platform, the second release in our full-stack suite. Open-sourced. Robotics is still bottlenecked by the 1× speed of the physical world. Every model, checkpoint, and data recipe eventually needs to be tested on physical hardware, slowly, expensively, and with limited coverage. One hour in reality can become 100 days in simulation. That is how robotics model iteration moves from a wall-clock bottleneck to a compute problem. To make this work, simulation has to be both fast and trustworthy. Over the past year, we rebuilt the entire stack: a GPU-accelerated cross-platform compiler, penetration-free multi-physics contact solvers, unified rigid and deformable physics, and a photo-realistic renderer purpose-built for physical AI applications. We built Nyx, a high-performance path-traced rendering engine for robotics application. Genesis World 1.0 achieves near realtime performance with our latest development for penetration-free IPC solver, supporting various types of deformables beyond rigid bodies. It supports contact-rich, dexterous manipulation simulation across different embodiments: unitree, sharpa, wuji, genesis hand and various types of grippers. Under the hood is Quadrants, our effort in pushing forward cross-platform GPU-accelerated computation. Quadrants started as a fork of Taichi, and we rebuilt most of the critical parts for optimizing simulation workloads, giving 10x faster launch time and up to 4.6x runtime performance compared to the initial Genesis release. Together, they bring us to an unprecedentedly low sim-to-real gap, enabling zero-shot real-to-sim model evaluation and much faster iteration of GENE. All available today. Genesis World 1.0: Quadrants: Nyx:
2:51

We are back again :) After three weeks of quiet building. Introducing Genesis World 1.0, our latest simulation platform, the second release in our full-stack suite. Open-sourced. Robotics is still bottlenecked by the 1× speed of the physical world. Every model, checkpoint, and data recipe eventually needs to be tested on physical hardware, slowly, expensively, and with limited coverage. One hour in reality can become 100 days in simulation. That is how robotics model iteration moves from a wall-clock bottleneck to a compute problem. To make this work, simulation has to be both fast and trustworthy. Over the past year, we rebuilt the entire stack: a GPU-accelerated cross-platform compiler, penetration-free multi-physics contact solvers, unified rigid and deformable physics, and a photo-realistic renderer purpose-built for physical AI applications. We built Nyx, a high-performance path-traced rendering engine for robotics application. Genesis World 1.0 achieves near realtime performance with our latest development for penetration-free IPC solver, supporting various types of deformables beyond rigid bodies. It supports contact-rich, dexterous manipulation simulation across different embodiments: unitree, sharpa, wuji, genesis hand and various types of grippers. Under the hood is Quadrants, our effort in pushing forward cross-platform GPU-accelerated computation. Quadrants started as a fork of Taichi, and we rebuilt most of the critical parts for optimizing simulation workloads, giving 10x faster launch time and up to 4.6x runtime performance compared to the initial Genesis release. Together, they bring us to an unprecedentedly low sim-to-real gap, enabling zero-shot real-to-sim model evaluation and much faster iteration of GENE. All available today. Genesis World 1.0: Quadrants: Nyx:

Genesis AI

304,324 Aufrufe • vor 1 Monat

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n
3:26

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: The Genesis physics engine and simulation platform is fully open source at We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: Project webpage: Documentation: 1/n

Zhou Xian

3,816,374 Aufrufe • vor 1 Jahr

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.
1:29

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

The Humanoid Hub

16,591 Aufrufe • vor 5 Monaten

NEWS: Nvidia has just introduced "Cosmos," a world foundation model created to understand the physical world. The model can generate synthetic data to train robotics.
2:34

NEWS: Nvidia has just introduced "Cosmos," a world foundation model created to understand the physical world. The model can generate synthetic data to train robotics.

Sawyer Merritt

254,296 Aufrufe • vor 1 Jahr

A few months ago we set out to understand exactly how our models can be used to accelerate progress in robotics and other AI systems that don’t just represent the world, but are actually able to interact with it. We’ve been collaborating with leading companies in the space and we’ve developed an initial approach that we're now opening up to accelerate development across the industry. Our fifth announcement is GWM Robotics. GWM Robotics is a learned simulator that generates synthetic data for scalable robot training and policy evaluation, removing the bottlenecks of physical hardware.
1:14

A few months ago we set out to understand exactly how our models can be used to accelerate progress in robotics and other AI systems that don’t just represent the world, but are actually able to interact with it. We’ve been collaborating with leading companies in the space and we’ve developed an initial approach that we're now opening up to accelerate development across the industry. Our fifth announcement is GWM Robotics. GWM Robotics is a learned simulator that generates synthetic data for scalable robot training and policy evaluation, removing the bottlenecks of physical hardware.

Runway

35,040 Aufrufe • vor 6 Monaten

Testing robot policies on hardware is slow, expensive and hard to scale. World models offer a promising path to accelerating robot policy development. We're sharing new research from the Runway Robotics team, in which we simulated 8 robot policies inside our General World Model and found 0.95 correlation with real-world results. Those early results point to world model simulation as a practical substitute for hardware evaluation, comparing favorably to existing real-to-sim approaches. Learn more at the link below.
0:40

Testing robot policies on hardware is slow, expensive and hard to scale. World models offer a promising path to accelerating robot policy development. We're sharing new research from the Runway Robotics team, in which we simulated 8 robot policies inside our General World Model and found 0.95 correlation with real-world results. Those early results point to world model simulation as a practical substitute for hardware evaluation, comparing favorably to existing real-to-sim approaches. Learn more at the link below.

Runway

13,464 Aufrufe • vor 4 Monaten

🚨 BREAKING: ABB Robotics + NVIDIA close the sim-to-real gap with 99% accuracy! 👾 ABB Robotics is integrating NVIDIA Omniverse libraries into RobotStudio to deliver physical AI for industry, closing the gap from virtual training to real-world deployment with up to 99% accuracy. RobotStudio HyperReality, available second half of 2026, will fundamentally change how quickly manufacturers can scale production: reducing costs by up to 40%, accelerating time-to-market by 50%, and cutting setup and commissioning times by up to 80%. For decades, the deficit between simulation accuracy and real-world lighting, materials, and environments has limited manufacturers' ability to design advanced manufacturing processes in the virtual world. The only robot manufacturer with a virtual controller running the same firmware as the hardware, ensuring near-perfect correlation between simulation and real-world performance. The system uses physically accurate simulations and foundation models endlessly optimized with real-world data feedback. These models can train any number of ABB robots anywhere in the world with industrial-grade reliability. Foxconn is using RobotStudio HyperReality for consumer electronics assembly. Assembly robots are trained virtually using synthetic data to perfect multiple production processes across various scenarios, then moved to production lines with 99% accuracy. This eliminates physical training and tests, reducing setup times and costs. Workr is demonstrating AI-powered robotic systems at NVIDIA GTC 2026. Built on ABB technology, trained with synthetic data using NVIDIA Omniverse, deployed without operators needing programming knowledge . 🚨 I’ll be onsite in San Jose during GTC 2026, and will be showing all the cool stuff that ABB Robotics prepared this year! Can’t wait! 🫡 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:48

🚨 BREAKING: ABB Robotics + NVIDIA close the sim-to-real gap with 99% accuracy! 👾 ABB Robotics is integrating NVIDIA Omniverse libraries into RobotStudio to deliver physical AI for industry, closing the gap from virtual training to real-world deployment with up to 99% accuracy. RobotStudio HyperReality, available second half of 2026, will fundamentally change how quickly manufacturers can scale production: reducing costs by up to 40%, accelerating time-to-market by 50%, and cutting setup and commissioning times by up to 80%. For decades, the deficit between simulation accuracy and real-world lighting, materials, and environments has limited manufacturers' ability to design advanced manufacturing processes in the virtual world. The only robot manufacturer with a virtual controller running the same firmware as the hardware, ensuring near-perfect correlation between simulation and real-world performance. The system uses physically accurate simulations and foundation models endlessly optimized with real-world data feedback. These models can train any number of ABB robots anywhere in the world with industrial-grade reliability. Foxconn is using RobotStudio HyperReality for consumer electronics assembly. Assembly robots are trained virtually using synthetic data to perfect multiple production processes across various scenarios, then moved to production lines with 99% accuracy. This eliminates physical training and tests, reducing setup times and costs. Workr is demonstrating AI-powered robotic systems at NVIDIA GTC 2026. Built on ABB technology, trained with synthetic data using NVIDIA Omniverse, deployed without operators needing programming knowledge . 🚨 I’ll be onsite in San Jose during GTC 2026, and will be showing all the cool stuff that ABB Robotics prepared this year! Can’t wait! 🫡 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

22,482 Aufrufe • vor 4 Monaten

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.
1:14

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 Aufrufe • vor 4 Monaten

The next frontier of AI is physical. 🤖 From simulation to synthetic data, developers are training robots to understand the real world with NVIDIA Cosmos, Isaac simulation frameworks and open-source physics engine, Newton. Watch the video to see how partners like Hon Hai Technology Group (Foxconn), Hexagon, Humanoid, Noble Machines, and Skild AI are building the future. 🎥
0:51

The next frontier of AI is physical. 🤖 From simulation to synthetic data, developers are training robots to understand the real world with NVIDIA Cosmos, Isaac simulation frameworks and open-source physics engine, Newton. Watch the video to see how partners like Hon Hai Technology Group (Foxconn), Hexagon, Humanoid, Noble Machines, and Skild AI are building the future. 🎥

NVIDIA Robotics

21,664 Aufrufe • vor 3 Monaten

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:59

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

25,718 Aufrufe • vor 2 Monaten

Anthropic’s CEO says AI-driven robotics could become a multi-trillion-dollar industry as models learn to generalize in the real world. Robotics is where AI starts to touch the $50T+ physical economy across manufacturing, logistics, agriculture, construction and healthcare.
1:54

Anthropic’s CEO says AI-driven robotics could become a multi-trillion-dollar industry as models learn to generalize in the real world. Robotics is where AI starts to touch the $50T+ physical economy across manufacturing, logistics, agriculture, construction and healthcare.

Shay Boloor

73,843 Aufrufe • vor 4 Monaten

What’s next in robotics starts with physical AI. 🤖 By combining decades of machine data with AI, simulation, and digital twins, leaders like AGIBOT, FRANKA ROBOTICS, CaterpillarInc, and PlusAI are scaling intelligent machines across industries and redefining what robots can do in the physical world. Watch the full video 📹
0:52

What’s next in robotics starts with physical AI. 🤖 By combining decades of machine data with AI, simulation, and digital twins, leaders like AGIBOT, FRANKA ROBOTICS, CaterpillarInc, and PlusAI are scaling intelligent machines across industries and redefining what robots can do in the physical world. Watch the full video 📹

NVIDIA Robotics

39,126 Aufrufe • vor 4 Monaten

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:
0:44

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:

Max Zhaoshuo Li 李赵硕

1,077,699 Aufrufe • vor 1 Monat

This New AI Just Nuked the Competition Genesis AI is here to wipe out every outdated simulation and totally dominate AI world. - Open sourced - Create data from prompt - 430,000x faster than real-time - Create 4D physical worlds in seconds with text prompt Let's dive in:
0:31

This New AI Just Nuked the Competition Genesis AI is here to wipe out every outdated simulation and totally dominate AI world. - Open sourced - Create data from prompt - 430,000x faster than real-time - Create 4D physical worlds in seconds with text prompt Let's dive in:

el.cine

358,440 Aufrufe • vor 1 Jahr

📢 Announcing one of the most exciting works from us this year on **scalable robot policy evaluation through real-to-sim transfer**, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is **orders of magnitude slower** than the development of language agents. We propose a new framework where simulation performance **strongly correlates** with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing **state-of-the-art correlation** between simulation and reality for deformable object manipulation. It provides a **scalable and reproducible evaluation engine** for robot learning. 🌐
3:04

📢 Announcing one of the most exciting works from us this year on **scalable robot policy evaluation through real-to-sim transfer**, moving toward a scalable evaluation engine with structured world models that capture the appearance, geometry, and dynamics of environments involving deformable objects. 🤖 Evaluation remains one of the biggest bottlenecks in building general-purpose robots. Today, robots are still evaluated only in the real world, which is **orders of magnitude slower** than the development of language agents. We propose a new framework where simulation performance **strongly correlates** with the real world (r > 0.9), even for deformable objects. The key difference from existing work lies in the correlation between simulation and reality: if a robot model performs better in the digital world, does it also perform better in the real world? This question has long made people hesitant about simulation-based evaluation — especially for deformable objects. We are changing that. Our pipeline achieves effective real-to-sim transfer, establishing **state-of-the-art correlation** between simulation and reality for deformable object manipulation. It provides a **scalable and reproducible evaluation engine** for robot learning. 🌐

Yunzhu Li

39,864 Aufrufe • vor 8 Monaten

2. NVIDIA is building the GPT of humanoid robots. They just launched Isaac GR00T N1.5 - a foundation model for general purpose robotics. Here’s how it works: → A human demos the task once → Cosmos (their physics AI model) generates 1,000s of variations → Omniverse simulates the motions in high fidelity → The robot trains entirely in simulation → Then fine-tunes itself in the real world Robots can now learn general skills across tasks, tools, even body types with just one human demo. AI isn’t just thinking in text anymore. It’s perceiving. Reasoning. Moving. Physical AI is here and it’s training itself.
2:21

2. NVIDIA is building the GPT of humanoid robots. They just launched Isaac GR00T N1.5 - a foundation model for general purpose robotics. Here’s how it works: → A human demos the task once → Cosmos (their physics AI model) generates 1,000s of variations → Omniverse simulates the motions in high fidelity → The robot trains entirely in simulation → Then fine-tunes itself in the real world Robots can now learn general skills across tasks, tools, even body types with just one human demo. AI isn’t just thinking in text anymore. It’s perceiving. Reasoning. Moving. Physical AI is here and it’s training itself.

Shruti

70,359 Aufrufe • vor 1 Jahr

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,334 Aufrufe • vor 26 Tagen

The 375ai Mainnet is live! 375ai indexes the real world, transforming video, audio and environmental data into real time intelligence using edge AI. A new data layer for the physical world, now on @Solana
0:55

The 375ai Mainnet is live! 375ai indexes the real world, transforming video, audio and environmental data into real time intelligence using edge AI. A new data layer for the physical world, now on @Solana

375ai

225,976 Aufrufe • vor 8 Monaten