Yuke Zhu
@yukez • 24,619 subscribers
Associate Professor @UTCompSci | Director @NVIDIAAI Co-Leading GEAR | CS PhD @Stanford | Building generalist robot autonomy in the wild | Opinions are my own
Videos
Today, we're introducing SimFoundry, our real2sim2real framework at NVIDIA GEAR that automatically turns real-world scenes into simulation-ready worlds from a single image or video. Website: Paper: This work marks a major step for our team toward leveraging simulations and synthetic data for foundation model training and systematic policy evaluation at scale. Code will be open-sourced soon. Stay tuned!
Yuke Zhu82,381 次观看 • 4 个月前
Sharing a copy of my ICRA'26 keynote talk from three months ago. 3 key messages: 1. Building a human-like machine, a dream that has fascinated humankind for centuries, is rapidly becoming a reality. 2. Human data is, by far, the most scalable source of training data to fuel robot foundation models. 3. World models are a "data sponge" that can absorb massive amounts of multimodal data for policy generalization. Video: Slides: The talk brought together our work over the past year: GR00T, EgoScale, SONIC, WAM, and more. Here’s the same demo I showed on stage: Our (now old!) model learning to assemble the YCB airplane with zero teleop data. We've made incredible progress since then. What's coming next? 🚀
Yuke Zhu11,409 次观看 • 1 个月前
The million-dollar question in humanoid robotics is: Can humanoids tap into Internet-scale training data such as online videos due to their human-like physique? Our #CoRL2024 oral paper showed the promise of humanoids learning new skills from single video demonstrations. (1/n)
Yuke Zhu69,582 次观看 • 2 年前
We took a short break from robotics to build a human-level agent to play Competitive Pokémon. Partially observed. Stochastic. Long-horizon. Now mastered with Offline RL + Transformers. Our agent, trained on 475k+ human battles, hits the top 10% on Pokémon Showdown leaderboards. No search or heuristics, just sequence modeling. Today, we're open-sourcing our Metamon platform with our algorithms, data, and environments: 🌐 We are excited to see how our work accelerates research on building generally capable AI agents, and more importantly, inspires the next generation of Pokémon trainers!
Yuke Zhu50,783 次观看 • 1 年前
Today, we publicly released RoboCasa365, a large-scale simulation benchmark for training and systematically evaluating generalist robot models. Built upon our original RoboCasa framework, it offers: • 2,500 realistic kitchen environments; • 365 everyday tasks (basic skills + long-horizon mobile manipulation); • Over 3,200 objects with many articulated fixtures/appliances. All are designed for fully controlled, reproducible benchmarking of robotic policies. Progress in robotic foundation models is real. But it’s still hard to answer basic questions like: How close are we to general-purpose autonomy? What factors drive generalization? What are the model/data scaling curves like? Real-world eval is slow and noisy, and existing sims (like LIBERO, which we built 3 years ago) often lack sufficient task and scene diversity. This benchmark comes with 2,200+ hours of demonstrations and 500K+ trajectories to support studies of multi-task training, pretraining, and continual learning at scale. Check it out at
Yuke Zhu24,748 次观看 • 8 个月前
Excited to share our latest progress on legged manipulation with humanoids. We created a VR interface to remote control the Draco-3 robot 🤖, which cooks ramen for hungry graduate students at night. We can't wait for the day it will help us at home in the real world! #humanoid
Yuke Zhu68,341 次观看 • 3 年前
没有更多内容可加载