We’re bringing powerful AI directly onto robots with Gemini Robotics On-Device. 🤖 It’s our first vision-language-action model to help make robots faster, highly efficient, and adaptable to new tasks and environments - without needing a constant internet connection. 🧵

What makes this new model unique? 🔵 It has the generality and dexterity of Gemini Robotics - but it can run locally on the device 🔵 It can handle a wide variety of complex, two-handed tasks out of the box 🔵 It can learn new skills with as few as 50-100 demonstrations

From humanoids to industrial bi-arm robots, the model supports multiple embodiments, even though it was pre-trained on ALOHA - while following instructions from humans. 💬 These tasks may seem easy for us but require fine motor skills, precise manipulation and more. ↓

We're also launching the Gemini Robotics software development kit (SDK) to help developers fine-tune the model for their own applications, including by testing it in the MuJoCo physics simulator. 🌐

Our new on-device solution runs independent of a data network - making it optimal for applications needing speed, or situations with poor connectivity. We’re excited to continue exploring the future of bringing AI into the physical world. Find out more →

Breakdown of the paper behind it: Title: Gemini Robotics: Bringing AI into the Physical World The study addresses the challenge of bringing advanced artificial intelligence, particularly large multimodal models that excel in digital tasks, into the physical world to control robots. While these models show impressive general abilities in areas like understanding text and images, making robots truly useful requires them to understand and interact with the physical world competently and safely. This involves what the paper calls "embodied reasoning," which is the common sense humans have about 3D environments, object relationships, and basic physics. Current robots often lack this deep understanding, limiting their ability to perform complex, general tasks. ...

Will you open source it?

Impressive. These on-device models put Gemini Robotics firmly on track to become the 'Android of Robotics'. In the limit, OEMs will just need to focus on building the best robotics hardware, and simply use Gemini for the 'brain'.

This is incredible right into the future. Google is on a roll this year. Loving it. Gemini 🔥

will you offer decentralization with your robot? @ArcBlock_io is making it easy to add and you can give your users and robot buyers true control and protection of their data.

can it solve a rubik’s cube