Loading video...
Video Failed to Load
Gemini + π0 = actually useful robots! (Similar to what Physical Intelligence did with "Hi Robot") I can now verbally tell the robot that I'm building a red Lego wall or wooden tower, and it will infer the next steps by itself and pass me the necessary pieces, tools,... show more
25,485 views • 1 year ago •via X (Twitter)
10 Comments

And sometimes when pi0 wakes-up it does a wild-drunken behaviours:D But super exciting, thanks @RemiCadene and @LeRobotHF community! We are at stage we can actually build useful robots for constrained environments and specific tasks that genuinely generate economic value.

Here is the video with Legos. Also one of the wooden bars is too far to reach, so the arm moves to the next target. I think with the next generation of VLA models trained on top of video generation models, we should automatically see a step-change improvement in accuracy.

@physical_int Try GROOT Mimic

@physical_int Hi @shreyasgite, great work! I will soon get my hand on the SO-ARM101, and I would like to apply pi0 for the arm as well. Any guidance how to get on with it? Thanks :D

@physical_int For training you can use lerobot as it is; for inference I changed few things in control_utils.py file under robot devices;) Will make a quick tutorial this week or next✌️

@physical_int Did you finetune or something?

@physical_int Totally sick!! What are you using the grasping points for if you’re using pi0 and not conditioning with it? Also which gemni model?

@physical_int Gemini 2.0 flash. Not using grasping points in this one, but used it for other training set as grasping the objects correctly is still a bit tricky.

@physical_int Maybe a simple Readme about how all the pieces fit together, please? Will get us noobs acclimated super-fast. Thank you for your valuable contributions.

@physical_int Here you go;)
