
Yilun Du
@du_yilun • 14,384 subscribers
Assistant Professor at Harvard @KempnerInst + CS. PhD @MIT_CSAIL, BS MIT. Generative Models, Compositionality, Embodied Agents, Robot Learning.
Videos

Robotics has spent years scaling demonstration data and model size. What happens when we scale EXPLORATION for robot agents? We're incredibly excited to share RAX, an agentic robotics system that learns by scaling online exploration. A coding agent connects directly to the robot’s perception and control APIs, learns from feedback, and builds programs that perceive, plan, and act. The model stays frozen. As exploration scales, the agent improves its programs and builds up reusable skills. The results: • ZERO teleoperation or action demonstrations on real robots—from insertion with 0.1 mm clearance and air-fryer manipulation to deformable objects and long-horizon mobile whole-body control. • +37 percentage points on average over leading learned policies across five simulation benchmarks: RoboDojo, BEHAVIOR-1K, HumanoidArena, DexJoCo’s bimanual tasks, and LIBERO-Pro. • One framework spanning single-arm, bimanual, dexterous, humanoid, and mobile robots, with new embodiments connected through APIs. The craziest part: on each real-world embodiment, going from scratch to mastering the tasks took roughly two hours. No teleop. No model finetuning. We’re open-sourcing the full framework AND every experiment session. We can’t wait to see what people build with it! Project + sessions: Paper: GitHub: N Project fully led by Runhan Huang ✈️ NeurIPS 2026 Atlanta and Xiangcheng Zhang who are absolutely awesome. How it works, experiments, and robot videos below 🧵 (1/N)
Yilun Du76,924 просмотров • 1 месяц назад

Sharing our work at NeurIPS Conference on reasoning with EBMs! We learn an EBM over simple subproblems and combine EBMs at test-time to solve complex reasoning problems (3-SAT, graph coloring, crosswords). Generalizes well to complex 3-SAT / graph coloring/ N-queens problems.
Yilun Du48,030 просмотров • 1 год назад
Больше нет контента для загрузки