Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

What if you could feel a world model? Introducing HapticWorld — an interactive world simulator that predicts joint torque alongside visual observations and renders it back to the human operator in real time. HapticWorld: → Accurately predicts torque values → Improves data collection efficiency by rendering torque feedback to...

jamesjingxi's profile picture

James (Jingxi) Xu

1,516 subscribers

38,140 görüntüleme • 12 gün önce •via X (Twitter)

Bilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

4 Yorum

James (Jingxi) Xu profil fotoğrafı
James (Jingxi) Xu12 gün önce

We compare the performance of autonomous policy rollouts, with and without the torque modality, on three contact-rich tasks — microwave opening, whiteboard wiping, and box flipping — in both the real world and HapticWorld.

James (Jingxi) Xu profil fotoğrafı
James (Jingxi) Xu12 gün önce

HapticWorld handles long-horizon teleoperations very well. This is a single, uncut teleoperation episode of more than two minutes inside HapticWorld. Both the predicted video and the predicted torque remain stable over the whole interaction: the scene does not drift or collapse as the autoregressive rollout grows, and the rendered torque keeps tracking contact throughout.

James (Jingxi) Xu profil fotoğrafı
James (Jingxi) Xu12 gün önce

🌐 Website: 🎥 Video: 📄 Paper: Joint work with @StPenng @litian_liang @YXWangBot @callmektbot @MCutkosky

Haoyu Xiong profil fotoğrafı
Haoyu Xiong12 gün önce

Congrats Jingxi! I always wanted to have a world simulator that can recreate the haptic demo in the Stanford robotics center basement.

Benzer Videolar

Can we learn a 3D world model that predicts object dynamics directly from videos? Introducing Particle-Grid Neural Dynamics: a learning-based simulator for deformable objects that trains from real-world videos. Website: ArXiv: Code: Demo: To appear at #RSS2025
0:34

Can we learn a 3D world model that predicts object dynamics directly from videos? Introducing Particle-Grid Neural Dynamics: a learning-based simulator for deformable objects that trains from real-world videos. Website: ArXiv: Code: Demo: To appear at #RSS2025

Kaifeng Zhang

46,516 görüntüleme • 1 yıl önce

What if we can simulate an *interactive 3D world*, from a single image, in the wild, in real time? Introducing PointWorld-1B: a large pre-trained 3D world model that predicts env dynamics given RGB-D capture and robot actions. 🌐 from Stanford University NVIDIA
2:08

What if we can simulate an *interactive 3D world*, from a single image, in the wild, in real time? Introducing PointWorld-1B: a large pre-trained 3D world model that predicts env dynamics given RGB-D capture and robot actions. 🌐 from Stanford University NVIDIA

Wenlong Huang

279,840 görüntüleme • 9 ay önce

Train the AI behind real-world robots We’re hiring Data Labelers to annotate images and videos to train Optimus & autonomous systems. You’ll work with real production data & directly impact how these systems learn and operate in the real world Come join!
0:33

Train the AI behind real-world robots We’re hiring Data Labelers to annotate images and videos to train Optimus & autonomous systems. You’ll work with real production data & directly impact how these systems learn and operate in the real world Come join!

Tesla Recruiting

667,555 görüntüleme • 5 ay önce

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,867 görüntüleme • 4 ay önce

Policies trained on real robot data via imitation can be surprisingly capable. But for domains like dexterous manipulation, they are often not quite good enough: they move slowly, miss grasps, make unreliable contact, and fail under small perturbations. Can we improve them without any additional data collection on the real robot? In SCORE, we show that we can improve real-world diffusion/flow policies cheaply by using simulation to simply learn how to steer them on deployment. This leads to large gains in real-world success and speed across a variety of tasks, without requiring additional real-world experience: 🧵 (1/10)
0:44

Policies trained on real robot data via imitation can be surprisingly capable. But for domains like dexterous manipulation, they are often not quite good enough: they move slowly, miss grasps, make unreliable contact, and fail under small perturbations. Can we improve them without any additional data collection on the real robot? In SCORE, we show that we can improve real-world diffusion/flow policies cheaply by using simulation to simply learn how to steer them on deployment. This leads to large gains in real-world success and speed across a variety of tasks, without requiring additional real-world experience: 🧵 (1/10)

Abhishek Gupta

35,358 görüntüleme • 3 ay önce

Introducing Universal Simulator (UniSim), an interactive simulator of the real world. Interactive website: Paper:
1:57

Introducing Universal Simulator (UniSim), an interactive simulator of the real world. Interactive website: Paper:

Sherry Yang

440,007 görüntüleme • 3 yıl önce

At #GTC26 we are showing AlpaDreams — generative, interactive closed-loop simulation for autonomous driving. World models generate environments in real time, policies or drivers interact, and the simulator updates the world. If you are at GTC, come by the booth and try it out!
0:49

At #GTC26 we are showing AlpaDreams — generative, interactive closed-loop simulation for autonomous driving. World models generate environments in real time, policies or drivers interact, and the simulator updates the world. If you are at GTC, come by the booth and try it out!

Zian Wang

17,039 görüntüleme • 7 ay önce

15–30 minutes of real-world robot data. That's now enough to go from sim-to-real failure to working robot. Let’s see… You train a robot in simulation. You deploy it in the real world. It fails. The physics don't match. So you try to fine-tune it with real data, but… you never have enough real data, and the fine-tuning breaks everything the simulation taught it. SimDist fixes this with one key decision: don't transfer the policy. Transfer the world model. Keep the reward and value knowledge from simulation frozen. Only update the part that's actually wrong, how the robot predicts physics. Now the robot doesn't have to relearn the entire task in the real world. It already knows what success looks like. It just needs to correct its understanding of how the real world moves. The part that makes this work: they also trained on failures and recoveries; not just perfect demonstrations. Without that, the planner finds the gaps and exploits them. With it, the robot can tell a good future from a bad one. That's all it needs. Results on peg insertion, table leg assembly, locomotion on slippery and uneven surfaces. Tasks that require precision, force, and quick reaction. Thanks for sharing, Tyler Westenbroek ([Tyler Westenbroek]. Interactive visualization + paper: ——- Weekly robotics and AI insights. Subscribe free:
0:41

15–30 minutes of real-world robot data. That's now enough to go from sim-to-real failure to working robot. Let’s see… You train a robot in simulation. You deploy it in the real world. It fails. The physics don't match. So you try to fine-tune it with real data, but… you never have enough real data, and the fine-tuning breaks everything the simulation taught it. SimDist fixes this with one key decision: don't transfer the policy. Transfer the world model. Keep the reward and value knowledge from simulation frozen. Only update the part that's actually wrong, how the robot predicts physics. Now the robot doesn't have to relearn the entire task in the real world. It already knows what success looks like. It just needs to correct its understanding of how the real world moves. The part that makes this work: they also trained on failures and recoveries; not just perfect demonstrations. Without that, the planner finds the gaps and exploits them. With it, the robot can tell a good future from a bad one. That's all it needs. Results on peg insertion, table leg assembly, locomotion on slippery and uneven surfaces. Tasks that require precision, force, and quick reaction. Thanks for sharing, Tyler Westenbroek ([Tyler Westenbroek]. Interactive visualization + paper: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

14,222 görüntüleme • 5 ay önce

The A vector potential is a real field that applies torque to spacetime.
1:02

The A vector potential is a real field that applies torque to spacetime.

Ashton Forbes

18,294 görüntüleme • 2 ay önce

To bring generalist intelligent robots to the real world, we have to overcome the data scarcity problem. At Rhoda, we are solving it by reformulating robot policies as video generation. Today, we introduce the Direct Video-Action Model (DVA)
1:34

To bring generalist intelligent robots to the real world, we have to overcome the data scarcity problem. At Rhoda, we are solving it by reformulating robot policies as video generation. Today, we introduce the Direct Video-Action Model (DVA)

Rhoda AI

79,314 görüntüleme • 7 ay önce

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.
2:05

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 görüntüleme • 1 yıl önce

Introducing Starchild-1 from Odyssey, the first ever real-time multimodal world model. This a model that can generate interactive simulations of the world that you can—for the first time ever—hear. Starchild-1 represents a big step towards a general-purpose world simulator.
2:15

Introducing Starchild-1 from Odyssey, the first ever real-time multimodal world model. This a model that can generate interactive simulations of the world that you can—for the first time ever—hear. Starchild-1 represents a big step towards a general-purpose world simulator.

Oliver Cameron

247,268 görüntüleme • 5 ay önce

Bernt Børnich says teleoperation data and synthetic data are like a crutch to bootstrap your way to robots learning in the real world.
1:52

Bernt Børnich says teleoperation data and synthetic data are like a crutch to bootstrap your way to robots learning in the real world.

The Humanoid Hub

28,051 görüntüleme • 1 yıl önce

What if robots could improve themselves by learning from their own failures in the real-world? Introducing 𝗣𝗟𝗗 (𝗣𝗿𝗼𝗯𝗲, 𝗟𝗲𝗮𝗿𝗻, 𝗗𝗶𝘀𝘁𝗶𝗹𝗹) — a recipe that enables Vision-Language-Action (VLA) models to self-improve for high-precision manipulation tasks. PLD couples real-world residual reinforcement learning with standard supervised fine-tuning — letting robots discover, recover, and distill their own data flywheel. Quick 🧵
0:59

What if robots could improve themselves by learning from their own failures in the real-world? Introducing 𝗣𝗟𝗗 (𝗣𝗿𝗼𝗯𝗲, 𝗟𝗲𝗮𝗿𝗻, 𝗗𝗶𝘀𝘁𝗶𝗹𝗹) — a recipe that enables Vision-Language-Action (VLA) models to self-improve for high-precision manipulation tasks. PLD couples real-world residual reinforcement learning with standard supervised fine-tuning — letting robots discover, recover, and distill their own data flywheel. Quick 🧵

Wenli Xiao

186,346 görüntüleme • 1 yıl önce

The FDA is removing restrictions on using real-world data in drug and device applications. If a product works in the real world, we ought to know.
0:46

The FDA is removing restrictions on using real-world data in drug and device applications. If a product works in the real world, we ought to know.

Dr. Marty Makary

64,876 görüntüleme • 10 ay önce

Training fully autonomous robots takes more data than the real world can easily provide. Simulation lets us train, test, and deploy autonomous software without being limited by robots, space, or cost. Here’s OM1 running in simulation side-by-side to the real world.
1:26

Training fully autonomous robots takes more data than the real world can easily provide. Simulation lets us train, test, and deploy autonomous software without being limited by robots, space, or cost. Here’s OM1 running in simulation side-by-side to the real world.

OpenMind

121,631 görüntüleme • 10 ay önce

say hi to kled v3, the data collection layer for agi. any dataset a lab can describe, collected from the real world by 500,000 people in under 72 hours.
13:21

say hi to kled v3, the data collection layer for agi. any dataset a lab can describe, collected from the real world by 500,000 people in under 72 hours.

Avi Patel

1,262,397 görüntüleme • 26 gün önce

BREAKING: Elon Musk says SpaceX and Tesla data, could make Grok excellent at real-world engineering. “We've got a real opportunity with the SpaceX training data and maybe with the Tesla training data to make AI extremely capable in real-world engineering. No one has yet made AI excellent at real-world engineering. And I think that's an opportunity for Grok using the training data from SpaceX and Tesla.”
0:32

BREAKING: Elon Musk says SpaceX and Tesla data, could make Grok excellent at real-world engineering. “We've got a real opportunity with the SpaceX training data and maybe with the Tesla training data to make AI extremely capable in real-world engineering. No one has yet made AI excellent at real-world engineering. And I think that's an opportunity for Grok using the training data from SpaceX and Tesla.”

DogeDesigner

66,090 görüntüleme • 1 ay önce