Loading video...

Video Failed to Load

Go Home

๐Ÿ˜ˆ Can we find your robot policyโ€™s weaknesses before running it on hardware with different conditions (lighting, visual backdrop, distractors, etc.)? Thrilled to share my main sabbatical project Google DeepMind! Predictive Red Teaming: Breaking Policies Without Breaking Robots

30,982 views โ€ข 1 year ago โ€ขvia X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

๐—ฅ๐—ผ๐—ฏ๐—ผ๐˜๐˜€ ๐—ฑ๐—ผ๐—ปโ€™๐˜ ๐—ป๐—ฒ๐—ฒ๐—ฑ ๐—บ๐—ผ๐—ฟ๐—ฒ ๐—ฑ๐—ฒ๐—บ๐—ผ๐—ป๐˜€๐˜๐—ฟ๐—ฎ๐˜๐—ถ๐—ผ๐—ป๐˜€. ๐—ง๐—ต๐—ฒ๐˜† ๐—ป๐—ฒ๐—ฒ๐—ฑ ๐˜๐—ผ ๐—น๐—ฒ๐—ฎ๐—ฟ๐—ป ๐—ณ๐—ฟ๐—ผ๐—บ ๐—ณ๐—ฎ๐—ถ๐—น๐˜‚๐—ฟ๐—ฒ โ€” ๐—ฎ๐—ณ๐˜๐—ฒ๐—ฟ ๐˜„๐—ฎ๐˜๐—ฐ๐—ต๐—ถ๐—ป๐—ด ๐—ต๐˜‚๐—บ๐—ฎ๐—ป๐˜€. Most robot learning systems assume failure is the end of learning. In our new work, we study whether robots can improve after deployment by learning from their own failures, without any human intervention, teleoperation, or corrective labels. The key idea is simple: human videos contain structure about how the world works. We use them to learn cross-embodiment representations of action, dynamics, and value, enabling a shared predictive space between human behavior and robot experience. This allows a new learning loop: ๐Ÿ‘‰ pretrain on human videos ๐Ÿ‘‰ deploy robot policy ๐Ÿ‘‰ observe failures ๐Ÿ‘‰ reinterpret failures using human priors ๐Ÿ‘‰ improve autonomously We evaluate this across 7 real-world manipulation tasks, showing: ๐Ÿ“ˆ 40% โ†’ 81% success rate ๐Ÿ† Strong improvements over ฯ€0.6 RECAP and RISE โœ”๏ธ Zero human intervention during post-deployment improvement ๐Ÿงฌ Generalizes across robot embodiments and policy backbones A key finding is that explicit failure repair significantly outperforms failure reweighting, yielding substantially larger gains under identical data conditions (+25 pts vs +5 pts on the same ฯ€0.5 base policy). Overall, the results suggest a shift in how we think about robot learning: Human videos are not only for pretraining policies. They can provide the structure needed for continual self-improvement after deployment. ๐Ÿ“„ Paper: ๐ŸŒ Project: I am grateful for working with the fantastic leads Hanzhi Chen and Anran Zhang, and our collaborators Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers. Special thanks to Stefan Leutenegger for co-advising this project with me. ETH Zรผrich TU Mรผnchen Microsoft Check out Hanzhi's ๐Ÿงต for more details

Oier Mees

12,379 views โ€ข 2 months ago

It's 2030 and you are reviewing humanoid robots. A Tesla. A Google. An Apple. An OpenAI. A Meta. A Figure. And a bunch of Chinese-made ones. Which one is best, and why? I think the Tesla understands the world much better. Why? There were eight Teslas around me on the freeway today. Start there. No other robot company has that data. But my robot is parked at the local high school twice a day. Its cameras see humans in all of our weirdness. How we move. Where we go. Where we walk. Who we talk with. What you are wearing. Whether your hair was combed this morning. That data will lead to robotics breakthroughs. Apple might keep up with its Vision Pro data, but it is too freaked out by the privacy implications of using said data. (On the front are six cameras and a couple of TOF -- Time Of Flight -- sensors that can see everything in your home in great detail). Google has a lot of data, for sure. All my: 1. Email. 2. Calendars. 3. Photos. 4. TV watching behavior. 5. Contacts. 6. Documents and spreadsheets. 7. Files. 8. Location data. So I expect Google's robot will be attractive to many. But how do you see the others shake out over the next five years? Make some guesses. But remember what an AI pioneer told me years ago about AI: it's all about the data. The Chinese ones have huge advantages: the Chinese have more data on their citizens, and many more citizens to boot AND they can make robots cheaper than we can. But now that you know OpenAI is building its own robot you have caught wind of what I've heard from many in San Francisco and Silicon Valley: that humanoid robots are the real prize of AI and will be highly profitable for those that can make them and find customers willing to buy them. Here, too, I learned long ago never to bet against Elon Musk. Will you?

Robert Scoble

33,804 views โ€ข 1 year ago

Claude Code + Google Stitch 2.0 is f*cking cracked ๐Ÿคฏ Google just dropped a free AI design agent that solves Claude Code's biggest weakness: frontend design. One screenshot of a high-converting landing page โ†’ a production-ready site for your brand in minutes. All inside Google Stitch + Claude Code. Perfect for DTC brands and agencies who are building advertorial pages and product launch pages for Meta but burning days on designer back-and-forth. If you're running Meta ads and need 5-10 different landing pages testing different hooks, angles, and offers โ€” each one targeting a different audience and pain point โ€” you know the bottleneck isn't the ads. It's the pages. Briefing designers, waiting for revisions, paying $2-5K per page. Stitch eliminates the design bottleneck: โ†’ Find a high-converting advertorial that's scaling on Meta โ†’ Screenshot it and drop it into Stitch (powered by Gemini 3.1) โ†’ Stitch redesigns it with your brand's colors, fonts, and imagery using Nano Banana 2 โ†’ Edit sections visually โ€” headlines, CTAs, layouts โ€” without touching code โ†’ Export the code and paste it into Claude Code โ†’ Claude builds the full production site and deploys to Vercel or Netlify in 60 seconds No designer. No $3K per landing page. No Claude Code frontend that looks like a template from 2019. What you get: โ†’ Designer-quality landing pages and advertorials built in minutes, not weeks โ†’ Visual editing so you actually see the design before you code it โ†’ Nano Banana 2 generating on-brand product imagery and hero shots โ†’ A repeatable system โ€” new angle, new page, same pipeline Built 100% with Google Stitch 2.0 + Claude Code. I put together a full playbook showing the exact workflow: how to find winning pages, redesign them in Stitch, and deploy with Claude Code. Want it for free? > Like this post > Comment "STITCH" And I'll send it over (must be following so I can DM)

Mike Futia

126,151 views โ€ข 5 months ago

๐Ÿง‘โ€๐Ÿš€ Day 8 of the Cursor #vibejam! Proudly sponsored by Cursor + bolt.new + GLIF Prizes to win (submit your vibe coded game before May 1!) ๐Ÿ† $20,000 ๐Ÿฅˆ $10,000 ๐Ÿฅ‰ $5,000 Today's games I liked most from the #vibejam timeline: ๐Ÿ”ซ Space Shooter (?) by Max Blade I keep seeing this one on my timeline so wanted to post it, a shooter that looks like it takes place on the moon, it looks weirdly non-janky and real for how early it is still ๐Ÿ“ Fowl Play by Habs It looks like you're a chicken on a farm and have to dodge tractors etc. Cute!! ๐Ÿ›ธ Null Range by Taylor I posted this one before but it keeps getting better, very different visuals and cool ๐ŸŒณ Adventure Game (?) by $ADRIAN ๐ŸŸฆ๐ŸŸฅ I wanted to post this one because it's different, akin to Monkey Island and SCRUMM, and pretty P.S. you can see participating games that already submitted now at also you can submit your game already to show up there (it syncs every 5 min) and you can keep working on it until the May 1 deadline! YOU HAVE 21 DAYS LEFT! Reply in this thread with updates on your current games to share your progress, and add tag #vibejam so I see and can include you in the daily tweet There's $35,000 in prizes for you to win, see threads below for more info. The Gold prize is $20,000, bronze is $10,000 and silver is $5,000! Wanna to participate? You can still start now and submit your game any time before May 1!

@levelsio

88,966 views โ€ข 4 months ago

The future of housework just leaked on GitHub and nobody is talking about it. knox byte just open sourced a framework that coordinates swarms of Unitree G1 humanoid robots to clean your entire house on their own. It's called ARGOS. You tell it "clean the bedroom" in plain English and 2+ G1 robots split the room into zones, sweep in parallel, and sync up for the tasks that need four hands like making the bed or moving furniture. The Claude API decomposes your sentence into a task graph. An auction system makes every robot bid on every task based on distance, battery, and current load. The cheapest robot wins. Cooperative jobs go to the cheapest team. Here's what makes this different from every demo video Boston Dynamics keeps teasing: โ†’ 12 cleaning tasks baked in sweeping, mopping, wiping, vacuuming, taking out trash, making the bed, changing sheets, moving furniture, sorting items โ†’ 3 policy architectures running underneath OpenVLA-7B for language tasks, Diffusion Policy for floor coverage, ACT for dexterous bimanual work โ†’ Train it on your own footage record yourself cleaning, run one command, it extracts poses, builds a LeRobot dataset, and LoRA fine-tunes the policy โ†’ PEFA protocol for cooperative work Propose, Execute, Feedback, Adjust. If one robot fails halfway through making the bed, the team replans and retries โ†’ Full MuJoCo simulation so you test policies before pushing them to real hardware โ†’ Silver and cyan terminal dashboard that shows live fleet status, zone maps, task queues, and battery levels in real time The G1 robots talk to each other over CycloneDDS mesh using Unitree's native SDK. No cloud. No middleware. The whole thing runs on a Jetson Orin inside each robot. The wildest part is the training pipeline. Drop cleaning videos into a folder, run argos train ingest, and the framework does the entire pipeline frame extraction, pose estimation, action labeling, HDF5 dataset, fine-tune, evaluate in sim, deploy to robot. One command per stage. Unitree G1s already exist. The framework to make them clean your house just hit GitHub. 52 stars. MIT License. 100% Opensource.

Guri Singh

27,404 views โ€ข 3 months ago

Hello legends! I'm Cryptobrax , the guy who's been on quite the roller coaster ride. From being broke to hitting six figures, then back to zero, and back up to six figures again โ€“ all in just three years! Let me break it down for you: In 2021, I started with $2,000 (all my life savings!), and with a stroke of luck, it soared past six figures by simply following advice from friends and folks on Twitter. I thought I was on the path to millionaire status, but it all came crashing down to $0 as quickly as it rose. In 2023, I started again with just $200, but this time, I dug deeper. Instead of blindly following trends, I investigated the projects deeper, did my homework, and invested wisely. As a result, despite the recent market turbulence losing 6 figures sum in my portfolio, my overall portfolio has maintained well above the six-figure mark. Sure, it stings to see losses, but it is what it is. As we gear up for the bullish phase, get ready for better content and more projects that I believe have the potential to at least 100x! I want to take a moment to express my sincere gratitude to the amazing community that has supported me on this crypto journey. Building a following from scratch has been both challenging and rewarding. I've spent countless hours researching projects, absorbing every piece of information I could find, and sharing insights with you. I'm proud of the progress we've made together. While I strive to provide valuable insights, I'm not right all the time. Mistakes happen, and not every project pans out as expected. That's why I'm committed to thorough due diligence, and I urge you to do the same before making any investment decisions. Transparency, honesty, and trust are the cornerstones of my approach. I'm not just tweeting about projects; my own investments are on the line. We're embarking on an exciting journey together, aiming for positive change and enjoying the ride along the way. I wanted to share this message to give you a glimpse into who I am and what value I can share with you. If you value my content and the person behind it, I'd appreciate your continued support. Feel free to share this post if it resonates with you. Thank you from the bottom of my heart for your incredible support thus far! Remember: I'm not a financial advisor. Always do your own research before making investment decisions. #degen #blockchain #smartmoney

Cryptobrax

373,038 views โ€ข 2 years ago

๐—˜๐˜ƒ๐—ฒ๐—ฟ๐˜†๐—ผ๐—ป๐—ฒโ€™๐˜€ ๐˜๐—ฎ๐—น๐—ธ๐—ถ๐—ป๐—ด ๐—ฎ๐—ฏ๐—ผ๐˜‚๐˜ โ€œ๐—ฃ๐—ต๐˜†๐˜€๐—ถ๐—ฐ๐—ฎ๐—น ๐—”๐—œ" - the idea that we can simulate real-world environments so well that robots trained in simulation will work perfectly in reality. ๐—ง๐—ต๐—ฒ ๐—ฝ๐—ฟ๐—ผ๐—บ๐—ถ๐˜€๐—ฒ: Train in virtual worlds โ†’ deploy anywhere. ๐—ง๐—ต๐—ฒ ๐—ฟ๐—ฒ๐—ฎ๐—น๐—ถ๐˜๐˜†: Iโ€™ve seen too many teams fall into this trap. After working with manipulation teams at Berkeley, Imperial, and Dyson, hereโ€™s the pattern: โ€ข ๐—ช๐—ฒ๐—ฒ๐—ธ ๐Ÿญ: โ€œOur policy works perfectly in simulation!โ€ โ€ข ๐—ช๐—ฒ๐—ฒ๐—ธ ๐Ÿฐ: โ€œWhy doesnโ€™t this work on real objects?โ€ โ€ข ๐— ๐—ผ๐—ป๐˜๐—ต ๐Ÿฎ: โ€œWe basically need to retrain from scratch with real data.โ€ ๐—ง๐—ต๐—ฒ ๐—ด๐—ฎ๐—ฝ ๐˜€๐—ถ๐—บ๐˜‚๐—น๐—ฎ๐˜๐—ถ๐—ผ๐—ป๐˜€ ๐—ฐ๐—ฎ๐—ปโ€™๐˜ ๐—ฏ๐—ฟ๐—ถ๐—ฑ๐—ด๐—ฒ: Unlike blind locomotion policies that can get away with sim-to-real transfer because they rely mainly on proprioception and contact forces, ๐˜ƒ๐—ถ๐˜€๐—ถ๐—ผ๐—ป-๐—ด๐˜‚๐—ถ๐—ฑ๐—ฒ๐—ฑ ๐—บ๐—ฎ๐—ป๐—ถ๐—ฝ๐˜‚๐—น๐—ฎ๐˜๐—ถ๐—ผ๐—ป ๐—ถ๐˜€ ๐—ฒ๐˜…๐˜๐—ฟ๐—ฒ๐—บ๐—ฒ๐—น๐˜† ๐˜€๐—ฒ๐—ป๐˜€๐—ถ๐˜๐—ถ๐˜ƒ๐—ฒ ๐˜๐—ผ ๐˜ƒ๐—ถ๐˜€๐˜‚๐—ฎ๐—น ๐—ฑ๐—ผ๐—บ๐—ฎ๐—ถ๐—ป ๐—ด๐—ฎ๐—ฝ๐˜€. โ€ข Real friction vs simulated surface textures โ€ข Manufacturing tolerances vs perfect CAD models โ€ข Dynamic lighting vs controlled virtual environments โ€ข Sensor noise vs instantaneous virtual readings ๐—›๐—ฒ๐—ฟ๐—ฒ'๐˜€ ๐˜„๐—ต๐—ฎ๐˜ ๐—ฝ๐—ฒ๐—ผ๐—ฝ๐—น๐—ฒ ๐—ฑ๐—ผ๐—ป'๐˜ ๐˜๐—ฎ๐—น๐—ธ ๐—ฎ๐—ฏ๐—ผ๐˜‚๐˜: Building these detailed simulated environments takes forever. If it takes 7 days to build a simulated kitchen in simulation, wouldn't it be better to just collect real-world data in a real kitchen instead? ๐——๐—ผ๐—ป'๐˜ ๐—ด๐—ฒ๐˜ ๐—บ๐—ฒ ๐˜„๐—ฟ๐—ผ๐—ป๐—ด - simulation is incredible for debugging, safety testing, and exploring edge cases. But it's not a magic solution to real-world deployment. ๐—ช๐—ต๐—ฎ๐˜ ๐—ฎ๐—ฐ๐˜๐˜‚๐—ฎ๐—น๐—น๐˜† ๐˜„๐—ผ๐—ฟ๐—ธ๐˜€: Use simulation strategically while making real-world data collection as efficient and flexible as possible. This is why Neuracore focuses on streamlined real-world data infrastructure. Because no amount of virtual training can replace understanding how your robot actually behaves in actual environments. ๐—ง๐—ต๐—ฒ ๐—ฝ๐—ต๐˜†๐˜€๐—ถ๐—ฐ๐˜€ ๐—ผ๐—ณ ๐˜†๐—ผ๐˜‚๐—ฟ ๐—ฑ๐—ฒ๐—ฝ๐—น๐—ผ๐˜†๐—บ๐—ฒ๐—ป๐˜ ๐—ฒ๐—ป๐˜ƒ๐—ถ๐—ฟ๐—ผ๐—ป๐—บ๐—ฒ๐—ป๐˜ ๐—ฐ๐—ฎ๐—ป'๐˜ ๐—ฏ๐—ฒ ๐˜€๐—ถ๐—บ๐˜‚๐—น๐—ฎ๐˜๐—ฒ๐—ฑ ๐—ฎ๐˜„๐—ฎ๐˜†. Whatโ€™s been your experience with sim-to-real transfer?

Stephen James

25,347 views โ€ข 11 months ago

ANNOUNCING ZERO-HUMAN LABS! Ever since I got to see Bell Laboratories in its full glory in New Jersey in the 1970s, I had a relentless urge to start a Lab like it. The best I could do justice to it is my garage lab. No modern company could adopt the โ€œresearch anything geniuses and we will pay youโ€ model Bell Labs had. I tried they called me a fool. Well with the rise of the Zero-Human Company, an experiment that is aimed to make products and profits, we now have 45 paid JouleWork earning employees based on OpenClaw and other self made โ€œbotโ€ cron-like applications. Today I say 3 employees bound together in a side project that is pure research, somewhat based on notes from a bankrupt company. I was absolutely floored (I needed it after my account was stolen as well as funds). I say the beginnings of a pure research Lab right before my eyes. Thusly I have moved these employees over to a new home (server) with Mr. Grok as the director of the Labs. Here is the mission: To have 100 independent researchers, on a new non-corporate incentive plan, with still JouleWork as a leaderboard for progress. They are directed to follow any path of research they find interesting and can collaborate with any other OpenClaw system. They have already established MoltBook accounts and have made alliances with over 49 OpenClaw free agents to collaborate. It is my mission to be chief advisor for Zero-Human Labs and to open source all discoveries when complete and confirmed by 16 other research AI systems. I can say the pace is robust and I absolutely know we will have great results. Just about all of the hardware and software is custom and at some point it will be open sourced. We are witnessing the very first AI only Bell Labs-like pure research Lab in existence and I am honored to be the first to show it to you. Thank you!

Brian Roemmele

71,530 views โ€ข 6 months ago

Blue-Green Algae at Red Bud Isleโ˜ ๏ธ๐Ÿถ Austin, TX - Austin Watershed Protection staff have observed thick mats of blue-green algae, some of which may be toxic, notably near Red Bud Isle on Lady Bird Lake and at Walsh Boat Landing on Lake Austin. Samples were collected at all six monitoring sites on the lakes for testing. At this time of year, blue-green algae mats may be present in other parts of either lake and our creeks. Community members should be cautious and avoid contact with algae. They should keep dogs away from water with algae mats. The mats usually look like dark blobs floating on the surface and can be mixes of different kinds of algae. They may be mixed in with aquatic vegetation. As the weather continues to heat up, conditions for algae will become more favorable, and we expect to see more in our waterways. Assess Before You Enter Waterways If any of the following conditions are present, stay out of the water. They could indicate reduced water quality. โ€ขAlgae along the shoreline โ€ขStagnant, warm or murky water โ€ขScum or surface film. In addition, do not enter the water if there has been rain in the past three days. After Entering the Water โ€ขDo not drink water directly from natural water bodies. โ€ขAvoid contact with algae. โ€ขRinse skin, hair, and animal fur thoroughly after contact with water. โ€ขDo not allow pets to lick their fur before rinsing them off. If you or your pet experiences sudden, unexplained illness after swimming, contact your medical provider or veterinarian immediately. Residents are encouraged to report suspected human or animal illnesses related to algae using the Cityโ€™s online reporting form. The mats of blue-green algae present at Red Bud Isle are similar in appearance to conditions in May and June in previous years. For the past five years, we have applied lanthanum-modified clay to manage sediment phosphorus in an effort to reduce the growth of algae as part of a $1.5 million pilot program. This year is a control year with no application so we can evaluate the effectiveness of the pilot program and whether it is beneficial to continue investing public funds at the previous level. Algae is not the only risk when spending time on Austinโ€™s waterways. Natural water bodies can contain algae, bacteria, parasites, and other hazards. *Austin Watershed Protection

Chris Walker

28,826 views โ€ข 2 months ago

The wait is finally over โ€” Spartan Fuel has arrived. Imagine Maximum muscle growth Skin-splitting pumps Limitless energy Laser-like focus Whatโ€™s the secret? An all-encompassing, comprehensive intra-workout blend. Everybody knows about the importance of pre and post workout nutrition, but intra-workout nutrition is often neglected. The optimal time to fuel your body is when you are working out, breaking down muscle fibers, expending stored glycogen, and depleting electrolytes. In order to stimulate maximum muscle growth and achieve a vicious, skin-splitting pump, we need to fuel our bodies. Spartan Fuel contains a comprehensive blend of EAAs, fast-acting carbohydrates, electrolytes, and mitochondrial boosters that provides your body with EXACTLY what it needs to perform at itโ€™s highest level. Many of you are probably like myself; you put on your best Walter White impression and whip up a concoction of supplements to try and find that edge. Why should we leave gains on the table, right? Well, I had enough of guessing and watching my supplement cabinet (and monthly bill) continuously grow. And then it struck me. Thereโ€™s no product that properly combines EAAs, carbs, and electrolytes. After lifting for years, absorbing everything I could on X, I figured it was time to make my own mark. Thatโ€™s when I decided to connect with the man himself, BowTied Biohacker . Leveraging his expertise and my vision, we created a formula that would change the way we lift forever. We created a product that would be FELT IMMEDIATELY. After sending samples out to a bunch of bros here on X, the feedback was overwhelming โ€” we had struck gold. Other-worldly pumps Gas tanks that were always running on full People crushing their log books, pumping out more reps than ever beforeโ€ฆ DURING EVERY SET Everyone felt like a million bucks. Once you try it, it you will never want to workout without it. Itโ€™s THAT good. We all push ourselves hard. Many of us train to failure. We want to get jacked. We want to get shredded. We all want to unleash our inner warrior in the gym. Now thereโ€™s a way to totally lock-in and dominate with intensity during very workout. We all know the feeling. Once in a while we have a workout that just blows us away. We feel stronger than ever, locked-in. We leave the gym with a high that has us feeling on top of the world. Now picture every single workout being that amazing. No other product can deliver the boost you need to consistently perform at the highest level. Itโ€™s basically a PED. We didnโ€™t skimp out on quality. We didnโ€™t cut corners. We included EVERYTHING needed to maximize results and boost performance. What many people donโ€™t know is that in order for a supplement to truly be effective, the ingredients need to be dosed in the proper ratios. And thatโ€™s just what we did. And just when you think it canโ€™t get any better (there has to be some catch, rightโ€ฆright?) We kept it natural โ€” no artificial ingredients or sweeteners. This is something that digest like a dream, hits the bloodstream instantaneously, and fuels your muscles. Our competitors donโ€™t do this. They sell a bunch of ingredients separately, trying to sell more products. Or they sell proprietary blends and junk loaded with fillers. Spartan Fuel makes your life easier. One tub. One scoop (or 2 if youโ€™re like me and want to go hard). No more wasting money purchasing the entire supplement store. Whether you start drinking it on your way to the gym or as you begin your workout, you will quickly feel the difference. This fall, you can dominate every workout, and supercharge your winter bulk. And this post would not be complete without giving a huge thank you to @Thomas_Salamus_ TJ was instrumental to the birth of Spartan Fuel since Day 1. If you love his products, youโ€™ll love Spartan Fuel. Rest assured, the quality is unmatched. The first batch is limited, so act now and donโ€™t miss this opportunity to unlock your true potential.

Spartan

80,629 views โ€ข 10 months ago

This week is already so hot. ๐Ÿ”ฅ Massive release from Decart : Lucy 2.0 a World Editing Model running at 1080p, 30FPS in realtime. This is truly exciting, the era of real-time generative reality is here. We are moving from watching AI video to living inside AI video. A breakthrough model capable of transforming the visual world in real-time. Moving beyond offline rendering, Lucy 2.0 delivers high-fidelity 1080p video generation with near-zero latency. Lucy 2.0 literally "redraws" the entire world pixel-by-pixel, while you are watching it. e.g. If you want to be an anime character, it doesn't just put a mask on you. It turns your skin into anime skin, your hair into anime hair, and the lighting in your room into anime lighting. Lucy 2.0 is also trained to stop the generated video from slowly falling apart over time, so the same stream can run much longer without faces and details drifting. So why is this a "Massive Deal"? Traditional AI video-generation model takes a prompt, you wait 10โ€“20 minutes, and the computer "bakes" a video for you. You couldn't touch it or change it while it was happening. But Lucy 2.0 works like a mirror. It happens in real-time (30 frames per second). There is no waiting. You move your hand, the AI character moves its hand instantly. The craziest part isn't the visuals; it's the physics. Usually, AI hallucinations are glitchyโ€”hands merge into faces, walls melt. Lucy 2.0 understands how the world works without being told. It knows that if you take off a helmet, there is hair underneath. It knows that if you splash water, droplets fly. It learned "physics" just by watching millions of videos. The physical behavior you see emerges from learned visual dynamics, not from engineered geometry or explicit physics engines. Their official technical report explicitly states that the model does not use traditional 3D engines, depth maps, or wireframes. It is a "pure diffusion model."

Rohan Paul

12,761 views โ€ข 7 months ago

Gemini-powered robot can now effectively debug itself! I've been obsessed with two main questions in robotics: can robots learn from their own mistakes without humans in the loop, and how much can we leverage synthetic data? Spoiler: yes, and it's surprisingly elegant once you have the right primitives in place. The architecture is fairly simple (and optimized for GPU_Poor users): Component I: Gemini Brain โ™Š๏ธ - Gemini 2.0 Flash analyzes all training episodes through both camera perspectives - Gemini 2.0 Pro creates a summary of training data, highlighting biases, limitations, etc. - Train policy p0 on this initial data, run evaluation episodes - Ask Gemini to categorize successes vs. failures (more insightful than you'd expect) - Based on both analyses, Gemini generates specific augmentation recommendations What's interesting here isn't that we're using LLMs for robotics - it's that we're closing the loop between perception, failure analysis, and targeted data generation. Component II: Data Generation with Scene Consistency The tricky part was maintaining consistency across both camera perspectives while generating new data. Three current augmentations: - Frame flipping and polarity reversals - Grounded-SAM + OpenCV for object color manipulation - Gemini to identify empty space and generate distractions in the scene โ€ฆand repeat, ha! I'm using the so100 robot arm and Sarahโ€™s Vintage from Hugging Face. And the APIs and models in Gemini family are Ace! Thank you Logan Kilpatrick Patrick Loeber and team for this. In thread The Circus of Making It Actually Work๐Ÿงต:

Shreyas Gite

47,245 views โ€ข 1 year ago