Jiafei Duan

@DJiafei • 8,037 subscribers

Assistant Professor at @NUScomputing | Robotics & AI PhD @uwcse | Host of @RoboPapers | Ex-@allen_ai robotics lead, @NVIDIA | Magician | my opinion is my alone.

Live Cam

Jiafei Duan

Shorts

Singapore’s first humanoid robot fight really cool to see a knockout!

Humanoid fighting looks easy. Humanoid catching is hard. You have ~1 second to perceive, predict, and move your whole body to act. Meet Reflex: our framework for catching human-thrown boxes on a Unitree G1 using onboard RGB-D. Thread 👇

I trained a controller for Microduck using a 192-neuron Male CNS connectome subset. Watch its modeled neural activity drive exploratory head and body movements. Fun stuff.

My prediction: we’re about to see hundreds of robotics papers built on SAM3D. Waiting for the first one to drop 👀 SAM3 &amp; SAM3D from AI at Meta are just too good to be true!

Can your VLA perform collision avoidance when explicitly instructed in language? The answer is no. However, with VLS steering it is possible now! The same policy can successfully avoid collisions when given the instruction.

Robot paper of the day: RoboBallet: Planning for multirobot reaching with graph neural networks and reinforcement learning UCL + Google DeepMind + Intrinsic built an AI planner that choreographs teams of arms to work in tight spaces without collisions—planning in seconds, not days.

Videos

Watch Anya Live

VLAs? WAMs? Are language and video models the right foundations for robotics? Introducing Grounded Action Model (GAM): a new paradigm that builds robot foundation models on top of a pretrained 3D grounding model. Ground first. Then learn to act. 🧵👇

Ran the first policy in my new lab at NUS Computing. Zero-shot, fresh out of the box.

I think GPT-6 Astra will become the default baseline for future robotics papers. But for now, it would be interesting to find the boundary of its capabilities. At least for now, it still can’t solve this.

Is Opus 5.5 better than GPT-6 Astra at Physical Intelligence "literally" ? put GPT-6 Astra and Claude Opus 5.5 in two Unitree G1s in MuJoCo Ring Fence. Every 2 sim seconds: state → LLM → joint targets → act. Goal: force the other out of the ring, and Astra won after 31 second.

Was GPT-6 Astra trained on robot data? Got me curious. Borrowed Jay Chooi's code, ran an open-loop rollout on a MolmoAct2 dataset trajectory to find out. Here's what came out 👇

GPT-6 Astra vs Opus 5.5 vs Grok 4.7 vs MolmoAct2. Real time. No sped-up demos. Zero-shot. No cherry-picking. LLM robot control is progressing, but we’re definitely not there yet. Low-level control for robotics still matters according to Moravec’s paradox.

I trained a controller for Microduck using a 192-neuron Male CNS connectome subset. Watch its modeled neural activity drive exploratory head and body movements. Fun stuff.

Randomly stumbled on this video and wow. We humans are amazing manipulators. Memorization of skillful actions plays a part, but the sense of touch is what's truly crucial.

Are you training your VLAs or WAMs correctly? Your action expert may be learning vision–action shortcuts that undermine generalization beyond the training distribution. Introducing Latent Interface Training (LIT 🔥): Learn to act first, then learn how to use vision. 🧵👇

Given the strong community adoption and real-world deployment of MolmoAct2 on YAM, we're introducing zero-shot evaluation of MolmoAct2 Bimanual YAM in simulation. Now you can test out our models without a real-world robot and build on them! Code: Simulation built on Maniskill!

Humans use pointing to communicate plans intuitively. Compared to language, pointing gives more precise guidance to robot behaviors. Can we teach a robot how to point like humans? Introducing RoboPoint 🤖👉, an open-source VLM instruction-tuned to point. Check out our new work:

Every time I watch this video, I can't help but wonder: why don't we have robot butlers in our homes yet? The hardware seemed capable 14 years ago with teleoperation. Is it just the "robot brain" we're missing, or is there more to the puzzle?

Finally hooded! Collecting Egocentric data even during graduation! Huge thanks to my advisor Dieter Fox and Ranjay Krishna !

My prediction: we’re about to see hundreds of robotics papers built on SAM3D. Waiting for the first one to drop 👀 SAM3 & SAM3D from AI at Meta are just too good to be true!