Today, we're releasing an open-source async inference stack for all models currently hosted on Hugging Face, powering the world's cutest robots, built with love by the team at LeRobot Details in 🧵

Results: adopting our modified inference mode results in (1) attempts at recovery, and (2) a ~2x speedup in task completion, with comparable success rates. We've been hard at work to make this smooth for the community to build upon. Now, we support all action-chunking policies!

We speed-up action execution starting to compute the next action chunk before the current one has been exhausted. The underlying intuition is: if your world model is good enough, then you should effectively be able to start planning early

We designed our async stack to also be inherently remote: spin-up a server to run your policy on any machine, and send action chunks to your robot over the network Think GPUs on robots *without* GPUs on board

We're very excited to democratize remote, fast inference for the open-source community. Check out our blogpost for more details on the async stack. Get started with your robot and favorite policy, today! - blogpost:

@huggingface @LeRobotHF was waiting for thisssss!!!

@huggingface @LeRobotHF Excited to finally release!

@huggingface @LeRobotHF nice!

@huggingface @LeRobotHF Thank you for sharing man @philfung! Lmk when you try it out ;)

@huggingface @LeRobotHF Open-sourcing AI inference could democratize robotics development in exciting and unforeseen ways.

@huggingface @LeRobotHF thank you! precisely our point---all of this is likely already in place in big labs, gated by the mordor gates of closed source. We felt it was about time we released for the public!