Gemini-powered robot can now effectively debug itself! I've been... obsessed with two main questions in robotics: can robots learn from their own mistakes without humans in the loop, and how much can we leverage synthetic data? Spoiler: yes, and it's surprisingly elegant once you have the right primitives in place. The architecture is fairly simple (and optimized for GPU_Poor users): Component I: Gemini Brain ♊️ - Gemini 2.0 Flash analyzes all training episodes through both camera perspectives - Gemini 2.0 Pro creates a summary of training data, highlighting biases, limitations, etc. - Train policy p0 on this initial data, run evaluation episodes - Ask Gemini to categorize successes vs. failures (more insightful than you'd expect) - Based on both analyses, Gemini generates specific augmentation recommendations What's interesting here isn't that we're using LLMs for robotics - it's that we're closing the loop between perception, failure analysis, and targeted data generation. Component II: Data Generation with Scene Consistency The tricky part was maintaining consistency across both camera perspectives while generating new data. Three current augmentations: - Frame flipping and polarity reversals - Grounded-SAM + OpenCV for object color manipulation - Gemini to identify empty space and generate distractions in the scene …and repeat, ha! I'm using the so100 robot arm and Sarah’s Vintage from Hugging Face. And the APIs and models in Gemini family are Ace! Thank you Logan Kilpatrick Patrick Loeber and team for this. In thread The Circus of Making It Actually Work🧵:show more

Pi0 vs. ACT with BBox conditioning 🟦 Not many... know you can push ACT to *almost-Pi0* generalisation by conditioning on bounding boxes (BBoxes). How is the training data collected? • Generate BBoxes for all pick-and-place objects in the scene.(I used Gemini) • Pick-and-place targets are selected randomly. • Add the BBox coordinates to the robot’s state. • Overlay the BBoxes in the visualisation so you know what to grab and where to drop. During inference: • Generate BBoxes for every object again. • Click the object you want to pick and its target spot; those BBoxes get added to the robot state. • Let the robot do the work for you 😃 Setup: - Trained ACT for 100k steps and fine-tuned Pi0 for only 20k. - Training data is 60 episodes and had *only* LEGO bricks. - Using single front camera (Laptop in this case) Got the idea from xun in LeRobot discord. Here’s ACT vs Pi0 on a toy car that isn’t in the dataset. 1/3show more

🆕 Gemini in #GoogleSheets can now generate charts &... graphs to visualize data. You can also ask Gemini questions to glean insights & understand important patterns in your data. This is currently rolling out for Workspace Labs & Alpha users →show more

New in Chrome DevTools! Debug an app's full performance... trace with Gemini! After recording a trace, you can now chat with Gemini about the entire trace, related Performance insights, and even connected field data - all without needing to select specific context beforehand!show more

🚨BREAKING: Google just merged Gemini and NotebookLM into one... unified workspace and it changes everything about how you use AI for deep work. It's called Notebooks in Gemini and it's the personal knowledge base that power users have been begging for. You create a notebook for a project, drop in your files, PDFs, and documents, give Gemini custom instructions, and every chat you have stays organized in one place. No more hunting through old conversations. No more re-uploading the same files every session. The wildest part is the sync. Anything you add in Gemini automatically appears in NotebookLM. Anything you add in NotebookLM automatically appears in Gemini. One source of truth. Two powerful apps. Zero friction switching between them. So you can start a research notebook in Gemini, ask it questions all week, then flip to NotebookLM to generate a Cinematic Video Overview from the same material. Next morning, open Gemini and ask it to write a full report on exactly what you just watched. That workflow used to take three apps and a lot of copy-pasting. Now it's one notebook. Rolling out this week to Google AI Ultra, Pro, and Plus subscribers on web. Mobile and free users coming soon. What do you think?show more

📣 Gemini in Android Studio for businesses is here!... → Develop with confidence, powered by AI, with added privacy, security, and management for organizations. Get the same Gemini you love, enhanced for sensitive data protection.show more

✍️tldraw is using Gemini 3 Flash to power its... natural language computer, tldraw computer. Networks of components on an infinite canvas each use Gemini to process data and generate content that flows into other components. These flows are faster, smarter, and more reliable with Gemini 3 Flash.show more

👀You can now connect your Google Business Profile to... Gemini and make a Business Notebook. It has access to customer reviews, questions and also performance data. It can tell you trends, new market opportunities and you can ask Gemini to update your profile with new business hours, etc.show more

Introducing Gemini 2.5 Pro Web Crawler 🌐 Powered by... Google’s most intelligent model and Firecrawl It finds and re-rank pages, navigate using Gemini 2.5 Pro, and return data based on an objective.show more

Take the drawings on the fridge to the silver... screen. With video generation in the Google Gemini on #Pixel10 Pro you can upload an image and transform it with motion and sound based on the scene you describe:¹show more

1/ Gemini 2.5 is here, and it’s our most... intelligent AI model ever. Our first 2.5 model, Gemini 2.5 Pro Experimental is a state-of-the-art thinking model, leading in a wide range of benchmarks – with impressive improvements in enhanced reasoning and coding and now #1 on Arena by a significant margin. With a model this intelligent, we wanted to get it to people as quickly as possible. Find it on Google AI Studio and in the Google Gemini for Gemini Advanced users now – and in Vertex in the coming weeks. This is the start of a new era of thinking models – and we can’t wait to see where things go from here.show more

Smartest AI update for Chrome You can now use... Gemini from anywhere: - Type "@" in the top bar - Select "Chat with Gemini" - Write your prompt next Press Enter and Chrome will answer you directly using Gemini! No longer need to open a new tab, access Gemini and then finally type your prompt. Everything is done directly from the top bar in one go: a huge productivity boost!show more

🛠️ Getting started with the Gemini API? Check out... our collection of prompts and code samples for developers to build with the API and leverage the power of Gemini 1.5 Pro and 1.5 Flash in Google AI Studio. ✨ →show more

The Gemini 2.0 era is here. And we’re excited... for you to start building with it. A quick rewind of what we just released ⏪ Gemini 2.0 Flash ⚡ comes with low latency and better performance. 🔵 You can now access an experimental version in G3mini on the web, while Gemini Advanced users can try Deep Research, a new AI research assistant. 🔵 Developers can begin building through the Gemini API in Google AI Studio and Vertex AI 2.0 is also enabling new research prototypes of AI agents, including: 🔵 Project Astra, which explores future capabilities of a universal AI assistant 🔵 Project Mariner, which shows what’s possible for human-agent interaction, starting with your browser 🔵 Jules, an experimental AI-powered coding agent Finally, we’re exploring how 2.0 can be used in agents across domains — from navigating the virtual world of video games to applying its spatial reasoning capabilities to robotics. 🤖show more

The new era of Gmail with Gemini means you... can ask your inbox questions out loud and get instant answers from buried information. Voice search in Gmail will be coming later this year. Learn more about Gmail in the Gemini era →show more

Our latest and most capable image generation model is... available in Gemini: Meet Imagen 4. Everyone can make images for free in the Gemini App today.show more

GOOGLE 🔥: Gemini for Business will get a new... experience for collaborative Projects, where teams can work in a shared environment. Besides that, Google is rolling out Workflow Agents that can work on automation tasks across various apps. The same functionality is now available on Gemini Enterprise and will become better integrated into the core Gemini for Business experience. Is it only me, or does Gemini for Business feel much better than consumer-facing Gemini?show more

Last month we dropped the Gemini app for macOS.... In the coming weeks, we’ll be bringing Gemini Spark to the Gemini desktop app so it can help with tasks like organizing your local files, or extracting PDF data directly into Google Sheets. #GoogleIOshow more

Continuing the momentum from yesterday, we're excited to share... that you can use the Live API for Gemini models through Vertex AI in Firebase. This means you can add more conversational interactions in your apps, like allowing customers to ask audio questions and get responses. 🔊 Learn more →show more

🎨 Rapidly prototype interactive 3D concepts with Gemini. Learn... how to use Gemini in the Canvas web app to build 3D environments and interactive models using WebXR APIs. Start building → Explore more examples below 🧵show more

In the coming weeks, Gemini Live will connect with... the Google apps you love–like Calendar, Keep, Tasks, and Maps. Soon you can just point your camera and ask Gemini to add an invite to your calendar. #GoogleIOshow more