ๆญฃๅœจๅŠ ่ฝฝ่ง†้ข‘...

่ง†้ข‘ๅŠ ่ฝฝๅคฑ่ดฅ

Introduce ๐Œ๐จ๐›๐ข๐ฅ๐ž ๐€๐‹๐Ž๐‡๐€๐Ÿ„ -- Learning! With 50 demos, our robot can autonomously complete complex mobile manipulation tasks: - cook and serve shrimp๐Ÿฆ - call and take elevator๐Ÿ›— - store a 3Ibs pot to a two-door cabinet Open-sourced! Co-led Tony Zhao, Chelsea Finn

1,567,687 ๆฌก่ง‚็œ‹ โ€ข 2 ๅนดๅ‰ โ€ขvia X (Twitter)

10 ๆก่ฏ„่ฎบ

Zipeng Fu ็š„ๅคดๅƒ
Zipeng Fu2 ๅนดๅ‰

Our robot can consistently handle these tasks, succeeding: - 9 times in a row for Wipe Wine - 5 times for Call Elevator - robust against distractors for Use Cabinet - extrapolate to chairs unseen during training

Zipeng Fu ็š„ๅคดๅƒ
Zipeng Fu2 ๅนดๅ‰

How do we achieve this with only 50 demos? The key is to co-train imitation learning algorithms with static ALOHA data. We found this to consistently improve performance, especially for tasks that require precise manipulation.

Zipeng Fu ็š„ๅคดๅƒ
Zipeng Fu2 ๅนดๅ‰

Co-training (1) improves the performance across all tasks, (2) is compatible with ACT, Diffusion Policy and VINN, (3) is robust to different data mixtures.

Zipeng Fu ็š„ๅคดๅƒ
Zipeng Fu2 ๅนดๅ‰

We open-source all the software and data of Mobile ALOHA! Project Website ๐Ÿ›œ: Code for Imitation Learning ๐Ÿ–ฅ๏ธ: Data ๐Ÿ“Š:

Zipeng Fu ็š„ๅคดๅƒ
Zipeng Fu2 ๅนดๅ‰

Want to dive deeper into the hardware of Mobile ALOHA? Check out ๐Œ๐จ๐›๐ข๐ฅ๐ž ๐€๐‹๐Ž๐‡๐€๐Ÿ„ -- Hardware from co-lead @tonyzzhao!

Karol Hausman ็š„ๅคดๅƒ
Karol Hausman2 ๅนดๅ‰

@tonyzzhao @chelseabfinn Awesome project, congrats!

Zipeng Fu ็š„ๅคดๅƒ
Zipeng Fu2 ๅนดๅ‰

@tonyzzhao @chelseabfinn Thanks Karol!

Nick Dobos ็š„ๅคดๅƒ
Nick Dobos2 ๅนดๅ‰

@tonyzzhao @chelseabfinn Did you eat the shrimp?! How did it taste!?

Qualy the lightbulb ็š„ๅคดๅƒ
Qualy the lightbulb2 ๅนดๅ‰

@tonyzzhao @chelseabfinn > cook and serve shrimp Are you trying to provoke me?

Luddite Design ็š„ๅคดๅƒ
Luddite Design2 ๅนดๅ‰

@tonyzzhao @chelseabfinn Can it?

็›ธๅ…ณ่ง†้ข‘

We might be solving the wrong problem in robotics. Thatโ€™s what this makes clear. UMI โ†’ Universal Manipulation Interface A simple $400 gripper that lets you teach robots by demonstration. You hold it like a tool. Show the task. The robot learns. No teleoperation. No expensive hardware. No robot-specific data. Stanford open-sourced everything โ†’ hardware, code, datasets. What stands out to me is the bottleneck. Not algorithms. Data. Teleoperation โ†’ ~35 demos/hour UMI โ†’ ~111 demos/hour And the data transfers across robots โ†’ UR5, Franka, others. The design is surprisingly practical: โ†’ GoPro fisheye lens (155ยฐ FOV) + mirrors for depth โ†’ SLAM + IMU for precise 6DoF tracking โ†’ latency matching for dynamic tasks โ†’ diffusion policies for multimodal actions Then it scales. Cheng Chi takes this further with Sunday Robotics (with Tony Zhao). A $200 glove โ†’ deployed in 500+ homes โ†’ ~10 million real-world interactions. Not lab data. Real human behavior. Their robot learns dishes, laundry, espresso โ†’ with zero robot-specific data. This is where the shift becomes obvious. From training robots in controlled environments โ†’ to learning directly from humans at scale So hereโ€™s the real question: Will robotics be unlocked by better modelsโ€ฆ or by unlocking data? #ArtificialIntelligence #Robotics #AI #Innovation #FutureOfWork

Pascal Bornet

186,223 ๆฌก่ง‚็œ‹ โ€ข 3 ไธชๆœˆๅ‰

In my past research experience, finding or developing an appropriate simulation environment, dataset, and benchmark has always been a challenge. Missing features, limited support, or unexpected bugs often occupied my days and nights. Moreover, current simulation platforms are relatively fragmentedโ€”making it challenging to replicate the success of the RT-X dataset in unifying community efforts. Introducing RoboVerse, we provide a unified platform, dataset, and benchmark for scalable and generalizable robot learning. We hope to build a shared foundation to combine the community efforts. RoboVerse includes: MetaSim: We carefully designed a configuration system and a universal interface to align current robotic simulators. With MetaSim, you can use any simulator with the same codeโ€”bringing together the communityโ€™s diverse efforts under one framework! RoboVerse Dataset and Benchmark: We unify popular simulation environments and benchmarks into a single cohesive system and introduce the RoboVerse datasetโ€”a large-scale, high-quality synthetic dataset. Additionally, we propose a standardized benchmark across both imitation learning and reinforcement learning. A cool feature enabled by our unified framework: Hybrid Simulation! You can now integrate physics engines and renderers from different simulatorsโ€”e.g., using MuJoCo precise physics with Isaac photorealistic rendering. This not only elevates simulation fidelity but also significantly enhances real-world transfer performance across complex robotic applications. Hopefully, our teamโ€™s efforts could serve the robotic community to thrive vibrantly in the years to come. RoboVerse is open-sourced๐Ÿฅณ!!! Project Page: Documentation: Github Repo: Paper:

Haoran Geng

84,249 ๆฌก่ง‚็œ‹ โ€ข 1 ๅนดๅ‰

Can GPT-4 teach a robot hand to do pen spinning tricks better than you do? I'm excited to announce Eureka, an open-ended agent that designs reward functions for robot dexterity at super-human level. Itโ€™s like Voyager in the space of a physics simulator API! Eureka bridges the gap between high-level reasoning (coding) and low-level motor control. It is a โ€œhybrid-gradient architectureโ€: a black box, inference-only LLM instructs a white box, learnable neural network. The outer loop runs GPT-4 to refine the reward function (gradient-free), while the inner loop runs reinforcement learning to train a robot controller (gradient-based). We are able to scale up Eureka thanks to IsaacGym, a GPU-accelerated physics simulator that speeds up reality by 1000x. On a benchmark suite of 29 tasks across 10 robots, Eureka rewards outperform expert human-written ones on 83% of the tasks by 52% improvement margin on average. We are surprised that Eureka is able to learn pen spinning tricks, which are very difficult even for CGI artists to animate frame by frame! Eureka also enables a new form of in-context RLHF, which is able to incorporate a human operatorโ€™s feedback in natural language to steer and align the reward functions. It can serve as a powerful co-pilot for robot engineers to design sophisticated motor behaviors. As usual, we open-source everything! Welcome you all to check out our video gallery and try the codebase today: Paper: Code: Deep dive with me: ๐Ÿงต

Jim Fan

2,674,250 ๆฌก่ง‚็œ‹ โ€ข 2 ๅนดๅ‰

Chinaโ€™s pretty humanoid robot stuns by opening a car door in a โ€˜worldโ€™s firstโ€™ | Jijo Malayil, Interesting Engineering Mornine used onboard sensors and full-body control to locate the handle, adjust posture, and open a car doorโ€”no human input needed. AiMOGA Robotics has claimed to have reached a significant milestone in embodied AI with its humanoid robot, Mornine, autonomously opening a car door inside a functioning Chery dealership in China. Relying solely on onboard sensors, full-body motion control, and end-to-end reinforcement learning, Mornine performed the task without any human input. Unlike scripted or teleoperated robots, Mornie identified the door handle, adjusted its posture, and used coordinated force across its limbs and torso to complete the actionโ€”demonstrating advanced autonomy in a real-world setting. โ€œThe deployment marks one of the first instances of a service robot executing such a high-friction, physical interaction in a live commercial setting,โ€ said the firm in a statement. In April, at the Shanghai Auto Show, automotive brands Omoda and Jaecoo, subsidiaries of Chery Automobile, introduced Mornine, designed for use in car dealerships. From sim to service Opening a car door may seem like a simple task, but AiMOGA Robotics views it as a pivotal moment in roboticsโ€”signaling a shift from simulation to real-world service, and from basic command execution to autonomous capability. Using only onboard sensors and full-body motion control, Mornine identified the door handle, adjusted her posture, and applied coordinated force across her limbs to open the doorโ€”entirely without human intervention. Mornineโ€™s advanced sensor suite includes 3D LiDAR, depth and wide-angle cameras, and a visual-language model (VLM), enabling real-time perception of door position and opening status. Uniquely, Mornine wasnโ€™t explicitly programmed to recognize door handles. Instead, she learned through reinforcement learning, undergoing millions of simulated cycles to focus on the right region and perform the task independently. โ€œWe never explicitly told the robot what a door handle is. It learned to focus on that region by itself,โ€ said the engineering team at AiMOGA Robotics in a statement. The learned model was transferred to the real world using Sim2Real methods. Mornine continuously gathers live sensor data during operation, which feeds into a cloud-based training loop, allowing her to improve through continuous learning in real-world settings, reports Robotics Tomorrow. Now active in multiple Chery 4S dealerships in China, Mornine not only opens car doors but also assists with customer greetings, vehicle introductions, and item deliveryโ€”marking a step forward in humanoid robotics for commercial retail environments. AI meets retail Originally introduced as the AiMOGA Robot, Mornine was developed to support dealership sales by performing tasks such as explaining vehicle specifications, leading showroom tours, serving refreshments, and engaging with customers in multiple languages. First conceived by Chery as a virtual character to appeal to Generation Z using metaverse and virtual human technologies, Mornine gradually evolved into a real-world interactive humanoid. After multiple iterations of character and model design, Mornine debuted as a digital persona in animations, livestreams, and promotional content, gaining brand recognition. Chery later expanded the concept beyond the virtual space, resulting in the creation of the AiMOGA humanoid robot. Leveraging Cheryโ€™s expertise in autonomous driving, environmental sensing, and control systems, AiMOGA features full-stack capabilities in perception, cognition, decision-making, and execution. It uses multimodal sensingโ€”combining speech, vision, and environmental dataโ€”to interpret user gestures, commands, and showroom dynamics. A bionic motion system and automotive-grade hardware enable dexterous movement and upright mobility, while multi-robot collaboration allows for coordinated tasks like guided tours. At the decision-making layer, Deepseekโ€™s large language models enable natural language understanding and personalized interaction. In April 2025, Mornine officially began commercial service as an โ€œIntelligent Sales Consultantโ€ at the OMODA C5 JOYSTAR 4S dealership in Kuala Lumpur, Malaysiaโ€”marking her full transition from a virtual concept to a real-world humanoid sales assistant.

Owen Gregorian

67,975 ๆฌก่ง‚็œ‹ โ€ข 1 ๅนดๅ‰

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that **force-related data wasnโ€™t captured**: humans feel haptic feedback through the mechanical springs, but the robot couldnโ€™t leverage that info, limiting the dataโ€™s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a **portable visuo-tactile gripper** by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. ๐Ÿ”— We demonstrate **cross-modal representation learning** and **downstream policy learning** on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before โ€” and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature โ€™19): - 3D-ViTac (CoRL โ€™24):

Yunzhu Li

13,188 ๆฌก่ง‚็œ‹ โ€ข 1 ๅนดๅ‰

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:

Jim Fan

332,199 ๆฌก่ง‚็œ‹ โ€ข 2 ๅนดๅ‰