Loading video...

Video Failed to Load

Go Home

New in Chrome DevTools! Debug an app's full performance trace with Gemini! After recording a trace, you can now chat with Gemini about the entire trace, related Performance insights, and even connected field data - all without needing to select specific context beforehand!

686,880 views • 7 months ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Gemini-powered robot can now effectively debug itself! I've been obsessed with two main questions in robotics: can robots learn from their own mistakes without humans in the loop, and how much can we leverage synthetic data? Spoiler: yes, and it's surprisingly elegant once you have the right primitives in place. The architecture is fairly simple (and optimized for GPU_Poor users): Component I: Gemini Brain ♊️ - Gemini 2.0 Flash analyzes all training episodes through both camera perspectives - Gemini 2.0 Pro creates a summary of training data, highlighting biases, limitations, etc. - Train policy p0 on this initial data, run evaluation episodes - Ask Gemini to categorize successes vs. failures (more insightful than you'd expect) - Based on both analyses, Gemini generates specific augmentation recommendations What's interesting here isn't that we're using LLMs for robotics - it's that we're closing the loop between perception, failure analysis, and targeted data generation. Component II: Data Generation with Scene Consistency The tricky part was maintaining consistency across both camera perspectives while generating new data. Three current augmentations: - Frame flipping and polarity reversals - Grounded-SAM + OpenCV for object color manipulation - Gemini to identify empty space and generate distractions in the scene …and repeat, ha! I'm using the so100 robot arm and Sarah’s Vintage from Hugging Face. And the APIs and models in Gemini family are Ace! Thank you Logan Kilpatrick Patrick Loeber and team for this. In thread The Circus of Making It Actually Work🧵:

Shreyas Gite

47,245 views • 1 year ago