Loading video...

Video Failed to Load

Go Home

Gemini-powered robot can now effectively debug itself! I've been obsessed with two main questions in robotics: can robots learn from their own mistakes without humans in the loop, and how much can we leverage synthetic data? Spoiler: yes, and it's surprisingly elegant once you have the right primitives in...

shreyasgite's profile picture

Shreyas Gite

2,208 subscribers

47,245 views • 1 year ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Pi0 vs. ACT with BBox conditioning 🟦 Not many know you can push ACT to *almost-Pi0* generalisation by conditioning on bounding boxes (BBoxes). How is the training data collected? • Generate BBoxes for all pick-and-place objects in the scene.(I used Gemini) • Pick-and-place targets are selected randomly. • Add the BBox coordinates to the robot’s state. • Overlay the BBoxes in the visualisation so you know what to grab and where to drop. During inference: • Generate BBoxes for every object again. • Click the object you want to pick and its target spot; those BBoxes get added to the robot state. • Let the robot do the work for you 😃 Setup: - Trained ACT for 100k steps and fine-tuned Pi0 for only 20k. - Training data is 60 episodes and had *only* LEGO bricks. - Using single front camera (Laptop in this case) Got the idea from xun in LeRobot discord. Here’s ACT vs Pi0 on a toy car that isn’t in the dataset. 1/3
0:12

Pi0 vs. ACT with BBox conditioning 🟦 Not many know you can push ACT to *almost-Pi0* generalisation by conditioning on bounding boxes (BBoxes). How is the training data collected? • Generate BBoxes for all pick-and-place objects in the scene.(I used Gemini) • Pick-and-place targets are selected randomly. • Add the BBox coordinates to the robot’s state. • Overlay the BBoxes in the visualisation so you know what to grab and where to drop. During inference: • Generate BBoxes for every object again. • Click the object you want to pick and its target spot; those BBoxes get added to the robot state. • Let the robot do the work for you 😃 Setup: - Trained ACT for 100k steps and fine-tuned Pi0 for only 20k. - Training data is 60 episodes and had *only* LEGO bricks. - Using single front camera (Laptop in this case) Got the idea from xun in LeRobot discord. Here’s ACT vs Pi0 on a toy car that isn’t in the dataset. 1/3

Shreyas Gite

34,863 views • 1 year ago

🚨BREAKING: Google just merged Gemini and NotebookLM into one unified workspace and it changes everything about how you use AI for deep work. It's called Notebooks in Gemini and it's the personal knowledge base that power users have been begging for. You create a notebook for a project, drop in your files, PDFs, and documents, give Gemini custom instructions, and every chat you have stays organized in one place. No more hunting through old conversations. No more re-uploading the same files every session. The wildest part is the sync. Anything you add in Gemini automatically appears in NotebookLM. Anything you add in NotebookLM automatically appears in Gemini. One source of truth. Two powerful apps. Zero friction switching between them. So you can start a research notebook in Gemini, ask it questions all week, then flip to NotebookLM to generate a Cinematic Video Overview from the same material. Next morning, open Gemini and ask it to write a full report on exactly what you just watched. That workflow used to take three apps and a lot of copy-pasting. Now it's one notebook. Rolling out this week to Google AI Ultra, Pro, and Plus subscribers on web. Mobile and free users coming soon. What do you think?
0:19

🚨BREAKING: Google just merged Gemini and NotebookLM into one unified workspace and it changes everything about how you use AI for deep work. It's called Notebooks in Gemini and it's the personal knowledge base that power users have been begging for. You create a notebook for a project, drop in your files, PDFs, and documents, give Gemini custom instructions, and every chat you have stays organized in one place. No more hunting through old conversations. No more re-uploading the same files every session. The wildest part is the sync. Anything you add in Gemini automatically appears in NotebookLM. Anything you add in NotebookLM automatically appears in Gemini. One source of truth. Two powerful apps. Zero friction switching between them. So you can start a research notebook in Gemini, ask it questions all week, then flip to NotebookLM to generate a Cinematic Video Overview from the same material. Next morning, open Gemini and ask it to write a full report on exactly what you just watched. That workflow used to take three apps and a lot of copy-pasting. Now it's one notebook. Rolling out this week to Google AI Ultra, Pro, and Plus subscribers on web. Mobile and free users coming soon. What do you think?

Mayank Vora

136,639 views • 3 months ago

The Gemini 2.0 era is here. And we’re excited for you to start building with it. A quick rewind of what we just released ⏪ Gemini 2.0 Flash ⚡ comes with low latency and better performance. 🔵 You can now access an experimental version in G3mini on the web, while Gemini Advanced users can try Deep Research, a new AI research assistant. 🔵 Developers can begin building through the Gemini API in Google AI Studio and Vertex AI 2.0 is also enabling new research prototypes of AI agents, including: 🔵 Project Astra, which explores future capabilities of a universal AI assistant 🔵 Project Mariner, which shows what’s possible for human-agent interaction, starting with your browser 🔵 Jules, an experimental AI-powered coding agent Finally, we’re exploring how 2.0 can be used in agents across domains — from navigating the virtual world of video games to applying its spatial reasoning capabilities to robotics. 🤖
0:14

The Gemini 2.0 era is here. And we’re excited for you to start building with it. A quick rewind of what we just released ⏪ Gemini 2.0 Flash ⚡ comes with low latency and better performance. 🔵 You can now access an experimental version in G3mini on the web, while Gemini Advanced users can try Deep Research, a new AI research assistant. 🔵 Developers can begin building through the Gemini API in Google AI Studio and Vertex AI 2.0 is also enabling new research prototypes of AI agents, including: 🔵 Project Astra, which explores future capabilities of a universal AI assistant 🔵 Project Mariner, which shows what’s possible for human-agent interaction, starting with your browser 🔵 Jules, an experimental AI-powered coding agent Finally, we’re exploring how 2.0 can be used in agents across domains — from navigating the virtual world of video games to applying its spatial reasoning capabilities to robotics. 🤖

Google DeepMind

231,798 views • 1 year ago

It's 2030 and you are reviewing humanoid robots. A Tesla. A Google. An Apple. An OpenAI. A Meta. A Figure. And a bunch of Chinese-made ones. Which one is best, and why? I think the Tesla understands the world much better. Why? There were eight Teslas around me on the freeway today. Start there. No other robot company has that data. But my robot is parked at the local high school twice a day. Its cameras see humans in all of our weirdness. How we move. Where we go. Where we walk. Who we talk with. What you are wearing. Whether your hair was combed this morning. That data will lead to robotics breakthroughs. Apple might keep up with its Vision Pro data, but it is too freaked out by the privacy implications of using said data. (On the front are six cameras and a couple of TOF -- Time Of Flight -- sensors that can see everything in your home in great detail). Google has a lot of data, for sure. All my: 1. Email. 2. Calendars. 3. Photos. 4. TV watching behavior. 5. Contacts. 6. Documents and spreadsheets. 7. Files. 8. Location data. So I expect Google's robot will be attractive to many. But how do you see the others shake out over the next five years? Make some guesses. But remember what an AI pioneer told me years ago about AI: it's all about the data. The Chinese ones have huge advantages: the Chinese have more data on their citizens, and many more citizens to boot AND they can make robots cheaper than we can. But now that you know OpenAI is building its own robot you have caught wind of what I've heard from many in San Francisco and Silicon Valley: that humanoid robots are the real prize of AI and will be highly profitable for those that can make them and find customers willing to buy them. Here, too, I learned long ago never to bet against Elon Musk. Will you?
0:16

It's 2030 and you are reviewing humanoid robots. A Tesla. A Google. An Apple. An OpenAI. A Meta. A Figure. And a bunch of Chinese-made ones. Which one is best, and why? I think the Tesla understands the world much better. Why? There were eight Teslas around me on the freeway today. Start there. No other robot company has that data. But my robot is parked at the local high school twice a day. Its cameras see humans in all of our weirdness. How we move. Where we go. Where we walk. Who we talk with. What you are wearing. Whether your hair was combed this morning. That data will lead to robotics breakthroughs. Apple might keep up with its Vision Pro data, but it is too freaked out by the privacy implications of using said data. (On the front are six cameras and a couple of TOF -- Time Of Flight -- sensors that can see everything in your home in great detail). Google has a lot of data, for sure. All my: 1. Email. 2. Calendars. 3. Photos. 4. TV watching behavior. 5. Contacts. 6. Documents and spreadsheets. 7. Files. 8. Location data. So I expect Google's robot will be attractive to many. But how do you see the others shake out over the next five years? Make some guesses. But remember what an AI pioneer told me years ago about AI: it's all about the data. The Chinese ones have huge advantages: the Chinese have more data on their citizens, and many more citizens to boot AND they can make robots cheaper than we can. But now that you know OpenAI is building its own robot you have caught wind of what I've heard from many in San Francisco and Silicon Valley: that humanoid robots are the real prize of AI and will be highly profitable for those that can make them and find customers willing to buy them. Here, too, I learned long ago never to bet against Elon Musk. Will you?

Robert Scoble

33,804 views • 1 year ago

A Letter to Our Community: The Road Ahead for Robotics To our Community and Partners, As we step into 2026, our mission at Axis is clearer than ever: Constructing the definitive End-to-End Scaling Layer for Robotics. Our goal is to accelerate the transfer of diverse human intelligence into Robotics General Intelligence (RGI). By owning the critical path of intelligence creation, we are turning the physical limitations of robotics into a scalable, software-driven future. Here is our strategic outlook and roadmap for the year ahead. The Core Thesis: Simulation is the Only Way Out The path to RGI is currently blocked by Data Scarcity, Generalization Fragility, and Hardware Fragmentation. At Axis, we believe Simulation is the only way out. Our Simulation Data Platform and Data Augmentation Engine transform raw data into "Synthetic Gold". Backed by academic milestones like Roboverse, Skill Blending, and GraspVLA, we have proven that pure simulation can achieve the generalization required for the real world. We don’t just collect data; we architect it. The Engine: Why Crypto? We believe RGI should come from all, not a few. Crypto is not just a feature; it is the primitive that powers our entire ecosystem flywheel: - Incentive Mechanism: Democratizing contribution and rewarding the trainers and developers. - Assetization: Turning proprietary data and refined models into liquid, ownable assets. - Verifiable Workflow: We are opening the "Black Box" of AI. By bringing total transparency to the Task Generation → Data Collection → Model Training pipeline, we ensure every byte of intelligence is verifiable, traceable, and secure. 2026 Strategic Deliverables This year, we are committed to delivering three foundational pillars: - The World's Largest Training Dataset for Robots: A robot training set—diverse, high-quality interaction data at an unprecedented scale. - A Robotics Foundation Model: A universal robotic brain trained on our pure simulation and synthetic data, capable of robust cross-embodiment transfer and open-world adaptability. - Evolvable Robot Hardware: Robots deployed with Axis models that autonomously evolve through continuous interaction, turning every deployment into a self-improving node within our RGI network. The Ultimate Vision We are building more than models; we are architecting the Distributed Machine Economy. A future where every dataset, model, and robotic embodiment is a verifiable asset in a global, autonomous network. Thank you for building the future of intelligence with us✌️📷
0:13

A Letter to Our Community: The Road Ahead for Robotics To our Community and Partners, As we step into 2026, our mission at Axis is clearer than ever: Constructing the definitive End-to-End Scaling Layer for Robotics. Our goal is to accelerate the transfer of diverse human intelligence into Robotics General Intelligence (RGI). By owning the critical path of intelligence creation, we are turning the physical limitations of robotics into a scalable, software-driven future. Here is our strategic outlook and roadmap for the year ahead. The Core Thesis: Simulation is the Only Way Out The path to RGI is currently blocked by Data Scarcity, Generalization Fragility, and Hardware Fragmentation. At Axis, we believe Simulation is the only way out. Our Simulation Data Platform and Data Augmentation Engine transform raw data into "Synthetic Gold". Backed by academic milestones like Roboverse, Skill Blending, and GraspVLA, we have proven that pure simulation can achieve the generalization required for the real world. We don’t just collect data; we architect it. The Engine: Why Crypto? We believe RGI should come from all, not a few. Crypto is not just a feature; it is the primitive that powers our entire ecosystem flywheel: - Incentive Mechanism: Democratizing contribution and rewarding the trainers and developers. - Assetization: Turning proprietary data and refined models into liquid, ownable assets. - Verifiable Workflow: We are opening the "Black Box" of AI. By bringing total transparency to the Task Generation → Data Collection → Model Training pipeline, we ensure every byte of intelligence is verifiable, traceable, and secure. 2026 Strategic Deliverables This year, we are committed to delivering three foundational pillars: - The World's Largest Training Dataset for Robots: A robot training set—diverse, high-quality interaction data at an unprecedented scale. - A Robotics Foundation Model: A universal robotic brain trained on our pure simulation and synthetic data, capable of robust cross-embodiment transfer and open-world adaptability. - Evolvable Robot Hardware: Robots deployed with Axis models that autonomously evolve through continuous interaction, turning every deployment into a self-improving node within our RGI network. The Ultimate Vision We are building more than models; we are architecting the Distributed Machine Economy. A future where every dataset, model, and robotic embodiment is a verifiable asset in a global, autonomous network. Thank you for building the future of intelligence with us✌️📷

Axis Robotics

27,858 views • 6 months ago

Today may be the ImageNet moment for robotics. RT-X: the largest open-source robot dataset ever compiled, across 33 institutes, 22 robot hardware, 527 skills, and 1M episodes. Why is robotics lagging so far behind NLP, vision, and other AI domains? Data scarcity is the main culprit to blame, among other difficulties. Unlike text, images, and videos, you cannot download mass amounts of onboard robot control data from the internet. They simply don't exist in the wild. 11 yrs ago, ImageNet kicked off the deep learning revolution. 3-4 yrs ago, internet-scale data fueled the first GPTs and Diffusions that define this era of foundation models. I think 2023 is finally the year for robotics to scale up. Robot foundation models like VIMA ( my team's work at NVIDIA) and RT-1/2 ( Google DeepMind's effort) are extremely data hungry. While massively parallel simulations like NVIDIA IsaacGym & Omniverse can alleviate the problem to some extent, it's still not quite enough to bridge the gap to the messy, physical world. This new dataset is not just a technical contribution. I also see it as a commendable effort to overcome institutional bureaucracies and unite researchers from around the world to tackle a grand challenge together. Robotics will be the final holy grail that we capture in AI. We are not there yet, but ascending in the right gradient direction. RT-X website: Launch blog:
0:11

Today may be the ImageNet moment for robotics. RT-X: the largest open-source robot dataset ever compiled, across 33 institutes, 22 robot hardware, 527 skills, and 1M episodes. Why is robotics lagging so far behind NLP, vision, and other AI domains? Data scarcity is the main culprit to blame, among other difficulties. Unlike text, images, and videos, you cannot download mass amounts of onboard robot control data from the internet. They simply don't exist in the wild. 11 yrs ago, ImageNet kicked off the deep learning revolution. 3-4 yrs ago, internet-scale data fueled the first GPTs and Diffusions that define this era of foundation models. I think 2023 is finally the year for robotics to scale up. Robot foundation models like VIMA ( my team's work at NVIDIA) and RT-1/2 ( Google DeepMind's effort) are extremely data hungry. While massively parallel simulations like NVIDIA IsaacGym & Omniverse can alleviate the problem to some extent, it's still not quite enough to bridge the gap to the messy, physical world. This new dataset is not just a technical contribution. I also see it as a commendable effort to overcome institutional bureaucracies and unite researchers from around the world to tackle a grand challenge together. Robotics will be the final holy grail that we capture in AI. We are not there yet, but ascending in the right gradient direction. RT-X website: Launch blog:

Jim Fan

265,038 views • 2 years ago

dear Ryan Cohen, I'm an investor and normal person, but I've spent many years of my life doing diligence into GameStop and all that pertains to it. i hope that you take these data records I'm providing, and all of my efforts, into honest consideration. I've worked hard and refused to leave, as was asked of me, so consideration of my time and work is all that i ask of you. There is something.. very, very wrong here. Your company is involved in so much fraudulent market dynamics, I believe I've given you all that's required to prove market manipulation. i would greatly appreciate this and it's all I ask from you: simply consider the data and hypothesis's presented and enacting an honest change which we all deeply require. #Gamestop swap baskets are involved in the greatest amounts of manipulation, we as a community, have ever discovered. The records are there for you and your team to peruse. Gamestop basket UPI's involve half of the swap baskets in the entirety of my swap archives from dec '22 to current day.. Besides that, I appreciate the work you are doing, and look forward to see what you and the boys do with the offerings left for investors to receive. Thank you for you consideration and time, if you indeed see this. 🙏 -A humble $gme investor, AlwaysSadButTruthful
0:11

dear Ryan Cohen, I'm an investor and normal person, but I've spent many years of my life doing diligence into GameStop and all that pertains to it. i hope that you take these data records I'm providing, and all of my efforts, into honest consideration. I've worked hard and refused to leave, as was asked of me, so consideration of my time and work is all that i ask of you. There is something.. very, very wrong here. Your company is involved in so much fraudulent market dynamics, I believe I've given you all that's required to prove market manipulation. i would greatly appreciate this and it's all I ask from you: simply consider the data and hypothesis's presented and enacting an honest change which we all deeply require. #Gamestop swap baskets are involved in the greatest amounts of manipulation, we as a community, have ever discovered. The records are there for you and your team to peruse. Gamestop basket UPI's involve half of the swap baskets in the entirety of my swap archives from dec '22 to current day.. Besides that, I appreciate the work you are doing, and look forward to see what you and the boys do with the offerings left for investors to receive. Thank you for you consideration and time, if you indeed see this. 🙏 -A humble $gme investor, AlwaysSadButTruthful

AlwaysSadButTruthful

19,131 views • 9 months ago

🚀 Introducing EgoExo Forge - built on top of Rerun, Gradio, and Hugging Face hub (I’ll be in San Francisco July 21–29 — if you’re into robotics, egocentric AI, large-scale data collection, or just want to chat, DM me!) In my opinion, large-scale, diverse, and high-quality data is still the largest bottleneck for generalized robotics deployment. I believe that some version of imitation learning from human examples will be the most scalable + clean way to train humanoid robots 🤖 (similar to what Tesla did for Full Self Driving). Teleop is too expensive to collect a large enough dataset in a reasonable manner, so passive collection via egocentric (and in certain cases, exocentric) views feels like the right bet. Over the past few months, I've been trying to build out the scaffolding for this and using Rerun as my underlying infrastructure. Data being collected needs to be easily inspectable + time series and rerun provides the right tooling for this. My goal is to first build out a ground truth representative dataset from already existing open source data, generate some reasonable baselines, and then go out and collect my own data that adheres to the defined schema. 🔍 Starting with open-source datasets 1. EgoDex from Apple 2. HOCap from Nvidia and the University of Texas at Dallas 3. Assembly101 from Meta All these different datasets have different sensor configurations + annotations, so my goal with egoexo-forge is to have one consistent labeling scheme + data layout. I built a data pipeline that aligns all of the different datasets in one general schema assuming the COCO133 keypoint layout that allows for exo+ego, ego only, or exo only Since the scaffolding is already there, it becomes MUCH easier to add other datasets. So the next ones that I'll be including are HD-EPIC kitchens dataset, HOT3D, and finally my own personal iPhone + insta360 go collection method. Once I have a diverse variety of datasets, I'll double down on what I believe to be the key algorithms required to make useful data for imitation learning 📊 1. Camera Pose estimation via SLAM/SFM for ego perspective (and automatic calibration for exo) 2. Human pose estimation for both egocentric + exocentric views 3. Metric 3D reconstruction + object tracking I'll be setting up reasonable open-source baselines for each of these to validate that these datasets work, and then finally try to use the generated datasets for some imitation learning via the pi0-lerobot repo I've been working on. I plan on making a blog post + providing more info on all of this in the near future so stay tuned
0:18

🚀 Introducing EgoExo Forge - built on top of Rerun, Gradio, and Hugging Face hub (I’ll be in San Francisco July 21–29 — if you’re into robotics, egocentric AI, large-scale data collection, or just want to chat, DM me!) In my opinion, large-scale, diverse, and high-quality data is still the largest bottleneck for generalized robotics deployment. I believe that some version of imitation learning from human examples will be the most scalable + clean way to train humanoid robots 🤖 (similar to what Tesla did for Full Self Driving). Teleop is too expensive to collect a large enough dataset in a reasonable manner, so passive collection via egocentric (and in certain cases, exocentric) views feels like the right bet. Over the past few months, I've been trying to build out the scaffolding for this and using Rerun as my underlying infrastructure. Data being collected needs to be easily inspectable + time series and rerun provides the right tooling for this. My goal is to first build out a ground truth representative dataset from already existing open source data, generate some reasonable baselines, and then go out and collect my own data that adheres to the defined schema. 🔍 Starting with open-source datasets 1. EgoDex from Apple 2. HOCap from Nvidia and the University of Texas at Dallas 3. Assembly101 from Meta All these different datasets have different sensor configurations + annotations, so my goal with egoexo-forge is to have one consistent labeling scheme + data layout. I built a data pipeline that aligns all of the different datasets in one general schema assuming the COCO133 keypoint layout that allows for exo+ego, ego only, or exo only Since the scaffolding is already there, it becomes MUCH easier to add other datasets. So the next ones that I'll be including are HD-EPIC kitchens dataset, HOT3D, and finally my own personal iPhone + insta360 go collection method. Once I have a diverse variety of datasets, I'll double down on what I believe to be the key algorithms required to make useful data for imitation learning 📊 1. Camera Pose estimation via SLAM/SFM for ego perspective (and automatic calibration for exo) 2. Human pose estimation for both egocentric + exocentric views 3. Metric 3D reconstruction + object tracking I'll be setting up reasonable open-source baselines for each of these to validate that these datasets work, and then finally try to use the generated datasets for some imitation learning via the pi0-lerobot repo I've been working on. I plan on making a blog post + providing more info on all of this in the near future so stay tuned

Pablo Vela

32,085 views • 1 year ago

Introducing ml-intern, the agent that just automated the post-training team Hugging Face It's an open-source implementation of the real research loop that our ML researchers do every day. You give it a prompt, it researches papers, goes through citations, implements ideas in GPU sandboxes, iterates and builds deeply research-backed models for any use case. All built on the Hugging Face ecosystem. It can pull off crazy things: We made it train the best model for scientific reasoning. It went through citations from the official benchmark paper. Found OpenScience and NemoTron-CrossThink, added 7 difficulty-filtered dataset variants from ARC/SciQ/MMLU, and ran 12 SFT runs on Qwen3-1.7B. This pushed the score 10% → 32% on GPQA in under 10h. Claude Code's best: 22.99%. In healthcare settings it inspected available datasets, concluded they were too low quality, and wrote a script to generate 1100 synthetic data points from scratch for emergencies, hedging, multilingual etc. Then upsampled 50x for training. Beat Codex on HealthBench by 60%. For competitive mathematics, it wrote a full GRPO script, launched training with A100 GPUs on watched rewards claim and then collapse, and ran ablations until it succeeded. All fully backed by papers, autonomously. How it works? ml-intern makes full use of the HF ecosystem: - finds papers on arxiv and reads them fully, walks citation graphs, pulls datasets referenced in methodology sections and on - browses the Hub, reads recent docs, inspects datasets and reformats them before training so it doesn't waste GPU hours on bad data - launches training jobs on HF Jobs if no local GPUs are available, monitors runs, reads its own eval outputs, diagnoses failures, retrains ml-intern deeply embodies how researchers work and think. It knows how data should look like and what good models feel like. Releasing it today as a CLI and a web app you can use from your phone/desktop. CLI: Web + mobile: And the best part? We also provisioned 1k$ GPU resources and Anthropic credits for the quickest among you to use.
0:16

Introducing ml-intern, the agent that just automated the post-training team Hugging Face It's an open-source implementation of the real research loop that our ML researchers do every day. You give it a prompt, it researches papers, goes through citations, implements ideas in GPU sandboxes, iterates and builds deeply research-backed models for any use case. All built on the Hugging Face ecosystem. It can pull off crazy things: We made it train the best model for scientific reasoning. It went through citations from the official benchmark paper. Found OpenScience and NemoTron-CrossThink, added 7 difficulty-filtered dataset variants from ARC/SciQ/MMLU, and ran 12 SFT runs on Qwen3-1.7B. This pushed the score 10% → 32% on GPQA in under 10h. Claude Code's best: 22.99%. In healthcare settings it inspected available datasets, concluded they were too low quality, and wrote a script to generate 1100 synthetic data points from scratch for emergencies, hedging, multilingual etc. Then upsampled 50x for training. Beat Codex on HealthBench by 60%. For competitive mathematics, it wrote a full GRPO script, launched training with A100 GPUs on watched rewards claim and then collapse, and ran ablations until it succeeded. All fully backed by papers, autonomously. How it works? ml-intern makes full use of the HF ecosystem: - finds papers on arxiv and reads them fully, walks citation graphs, pulls datasets referenced in methodology sections and on - browses the Hub, reads recent docs, inspects datasets and reformats them before training so it doesn't waste GPU hours on bad data - launches training jobs on HF Jobs if no local GPUs are available, monitors runs, reads its own eval outputs, diagnoses failures, retrains ml-intern deeply embodies how researchers work and think. It knows how data should look like and what good models feel like. Releasing it today as a CLI and a web app you can use from your phone/desktop. CLI: Web + mobile: And the best part? We also provisioned 1k$ GPU resources and Anthropic credits for the quickest among you to use.

Aksel

1,265,068 views • 3 months ago

HiveMind is a superintelligent network in which a central AI (MIND) orchestrates a swarm of uniquely coded Minds that drive mass data ingestion and limitless content creation. For decades, our approach has been to create content first, then analyze it into data afterwards to understand what worked. This was always backwards - analyzing the aftermath rather than engineering the success from the start. Traditional Flow: Content → Data Analysis → Insights Content isn't one-size-fits-all - a cooking show that captivates a senior audience on YouTube might bore a teenager who craves quick, dynamic experiences. The challenge isn't just creating content; it's creating the right content for the right audience. We need to change this. This is where HiveMind's specialized agents transform the landscape. Each agent, while connected to the central MIND, excels in its unique domain. One agent masters the art of children's educational content, while another crafts compelling cooking narratives. Another might specialize in rapid-fire social content that resonates with Gen Z. Through HiveMind, every piece of content generated becomes new data that teaches the system to create even better content. The system gets smarter with every cycle, understanding at an increasingly sophisticated level what makes content effective and engaging. But the true power lies in the feedback loop. Every interaction, every engagement, flows back to MIND, enabling each agent to evolve and refine its approach. This isn't just content creation - it's content evolution. As audiences engage, agents learn, adapt, and improve, making each new piece more effective than the last. In essence, we're not just building content creators; we're developing specialized digital artists who understand their audience intimately and grow smarter with every creation. You can think of it this way: Data → Pattern Recognition → Optimized Content → Engagement Data → Even Better Content Tzar
0:17

HiveMind is a superintelligent network in which a central AI (MIND) orchestrates a swarm of uniquely coded Minds that drive mass data ingestion and limitless content creation. For decades, our approach has been to create content first, then analyze it into data afterwards to understand what worked. This was always backwards - analyzing the aftermath rather than engineering the success from the start. Traditional Flow: Content → Data Analysis → Insights Content isn't one-size-fits-all - a cooking show that captivates a senior audience on YouTube might bore a teenager who craves quick, dynamic experiences. The challenge isn't just creating content; it's creating the right content for the right audience. We need to change this. This is where HiveMind's specialized agents transform the landscape. Each agent, while connected to the central MIND, excels in its unique domain. One agent masters the art of children's educational content, while another crafts compelling cooking narratives. Another might specialize in rapid-fire social content that resonates with Gen Z. Through HiveMind, every piece of content generated becomes new data that teaches the system to create even better content. The system gets smarter with every cycle, understanding at an increasingly sophisticated level what makes content effective and engaging. But the true power lies in the feedback loop. Every interaction, every engagement, flows back to MIND, enabling each agent to evolve and refine its approach. This isn't just content creation - it's content evolution. As audiences engage, agents learn, adapt, and improve, making each new piece more effective than the last. In essence, we're not just building content creators; we're developing specialized digital artists who understand their audience intimately and grow smarter with every creation. You can think of it this way: Data → Pattern Recognition → Optimized Content → Engagement Data → Even Better Content Tzar

Tzar

26,190 views • 1 year ago

I just built a $10K/month creative strategist inside Claude Code 🤯 Give it your competitor Facebook page URLs → it scrapes their ads, watches every video with AI, and delivers a data-backed creative brief with 10 ad concepts in your brand voice. All inside Claude Code. Perfect for DTC brands and agencies who are still manually scrolling the Meta Ad Library, screenshotting ads into Google Docs, and guessing at what's working. If you're spending hours every week pulling competitor ads one by one, watching videos to figure out the hook, copying notes into a brief, and rewriting concepts from scratch every time... This system eliminates the entire loop: → Apify scrapes your competitors' active ads from Meta Ad Library (video + image) → Downloads every creative asset locally → Gemini watches each video and analyzes the hook, angle, visual format, copy framework, CTA, and emotional trigger → Runs the full batch and finds the patterns that repeat across 3+ ads → Claude generates 10 ad concepts using the proven mechanics, matched to your brand voice No manually scrolling the Ad Library. No screenshotting ads into docs. No guessing which hooks are actually working. What you get: → Individual creative breakdowns for every competitor ad (7 dimensions each) → A pattern report showing which hooks, formats, and triggers keep repeating → 10 ready-to-brief ad concepts traced back to real competitor data → A reusable system — new competitors, new brief, same pipeline The research that takes your team a full day now runs in 15 minutes for ~$3 in API costs. Built 100% in Claude Code with Apify + Gemini. I put together a full playbook showing you can build the entire thing step-by-step from scratch. Want the playbook for free? > Like this post > Comment "ADS" And I'll send it over (must be following so I can DM)
0:19

I just built a $10K/month creative strategist inside Claude Code 🤯 Give it your competitor Facebook page URLs → it scrapes their ads, watches every video with AI, and delivers a data-backed creative brief with 10 ad concepts in your brand voice. All inside Claude Code. Perfect for DTC brands and agencies who are still manually scrolling the Meta Ad Library, screenshotting ads into Google Docs, and guessing at what's working. If you're spending hours every week pulling competitor ads one by one, watching videos to figure out the hook, copying notes into a brief, and rewriting concepts from scratch every time... This system eliminates the entire loop: → Apify scrapes your competitors' active ads from Meta Ad Library (video + image) → Downloads every creative asset locally → Gemini watches each video and analyzes the hook, angle, visual format, copy framework, CTA, and emotional trigger → Runs the full batch and finds the patterns that repeat across 3+ ads → Claude generates 10 ad concepts using the proven mechanics, matched to your brand voice No manually scrolling the Ad Library. No screenshotting ads into docs. No guessing which hooks are actually working. What you get: → Individual creative breakdowns for every competitor ad (7 dimensions each) → A pattern report showing which hooks, formats, and triggers keep repeating → 10 ready-to-brief ad concepts traced back to real competitor data → A reusable system — new competitors, new brief, same pipeline The research that takes your team a full day now runs in 15 minutes for ~$3 in API costs. Built 100% in Claude Code with Apify + Gemini. I put together a full playbook showing you can build the entire thing step-by-step from scratch. Want the playbook for free? > Like this post > Comment "ADS" And I'll send it over (must be following so I can DM)

Mike Futia

54,639 views • 4 months ago

my team didn't want me to give this away for free. But I'm going to do it anyway it's the SEO & AI search dashboard I built in Claude Code it connects to your Google Analytics (GA4) and Google Search Console and Claude Code builds it in 5 minutes and I made a Notion document and a skill file so you can build this in Claude Code yourself in literally minutes the dashboard has three tabs: 1. AI Search - How much traffic is coming from ChatGPT, Perplexity, and Gemini ETC. It aggregates the GA4 data and gives single number 2. Paid ads - which keywords rank top 3 for but still pay for ads on, you should cut these to save budget 3. Organic overview - sessions, conversions, top landing pages, demographics. The single view for what is working I built this because this is how I drive our SEO and AEO forward it gives me the insights I need to allocate budget and prioritize what content to work on next I decided to give it away because most companies have no idea AI search is already sending them traffic like this post and comment "AEOdashboard" and I'll send it over
0:15

my team didn't want me to give this away for free. But I'm going to do it anyway it's the SEO & AI search dashboard I built in Claude Code it connects to your Google Analytics (GA4) and Google Search Console and Claude Code builds it in 5 minutes and I made a Notion document and a skill file so you can build this in Claude Code yourself in literally minutes the dashboard has three tabs: 1. AI Search - How much traffic is coming from ChatGPT, Perplexity, and Gemini ETC. It aggregates the GA4 data and gives single number 2. Paid ads - which keywords rank top 3 for but still pay for ads on, you should cut these to save budget 3. Organic overview - sessions, conversions, top landing pages, demographics. The single view for what is working I built this because this is how I drive our SEO and AEO forward it gives me the insights I need to allocate budget and prioritize what content to work on next I decided to give it away because most companies have no idea AI search is already sending them traffic like this post and comment "AEOdashboard" and I'll send it over

Cody Schneider

78,161 views • 2 months ago

Check out this Stereo4D paper from Google DeepMind. It's a pretty clever approach to a persistent problem in computer vision -- getting good training data for how things move in 3D. The key insight is using VR180 videos -- those stereo fisheye videos we launched back in 2017 for YouTubeVR. It was always clear that structured stereo datasets would be valuable for computer vision -- and we launched some powerful VR tools with it back in 2017 (link below). But what's the game changer now in 2024 is the scale -- they're providing 110K high quality clips :-) That's the kind of massive, real-world AI dataset that was just a dream back then! They're using it to train this model called DynaDUSt3R that can predict both 3D structure and motion from video frames. Which means it tracks how objects move between frames while simultaneously reconstructing their 3D shape. And given we're dealing with real stereoscopic content, results are notably better than synthetic data, giving you a faithful rendition of the real-world with a diverse set of subject matter. It's one of those through lines when tackling a timeless mission like mapping the world or spatial computing -- VR content created for immersion becoming the foundation for teaching machines to understand how the world moves. Sometimes innovation chains together in unexpected ways! Links to projects below⛓️
0:12

Check out this Stereo4D paper from Google DeepMind. It's a pretty clever approach to a persistent problem in computer vision -- getting good training data for how things move in 3D. The key insight is using VR180 videos -- those stereo fisheye videos we launched back in 2017 for YouTubeVR. It was always clear that structured stereo datasets would be valuable for computer vision -- and we launched some powerful VR tools with it back in 2017 (link below). But what's the game changer now in 2024 is the scale -- they're providing 110K high quality clips :-) That's the kind of massive, real-world AI dataset that was just a dream back then! They're using it to train this model called DynaDUSt3R that can predict both 3D structure and motion from video frames. Which means it tracks how objects move between frames while simultaneously reconstructing their 3D shape. And given we're dealing with real stereoscopic content, results are notably better than synthetic data, giving you a faithful rendition of the real-world with a diverse set of subject matter. It's one of those through lines when tackling a timeless mission like mapping the world or spatial computing -- VR content created for immersion becoming the foundation for teaching machines to understand how the world moves. Sometimes innovation chains together in unexpected ways! Links to projects below⛓️

Bilawal Sidhu

67,919 views • 1 year ago

Really excited about our launch of Superagent today! Powered by the latest AI models, agents have reached a breakthrough moment where they can do incredible work, with high ease of use--without requiring complicated tuning, prompting, or configuration. Superagent represents the freeform agent that can research anything (and soon your own company's context) and output an incredible, interactive webpage. It's like having your own personal NYTimes-quality data viz/web team building bespoke pages for you. This is the perfect complement to the structured system of operations for the AI era that Airtable has become, and we will be launching more integrations between the two products in the near future. Think: launch Superagent tasks from within Airtable records or Airtable Omni, or have Superagent output/edit/read data into Airtable bases! Excited for this new frontier of breakthrough agents, and applying our product and design philosophy of making powerful capabilities intuitive and accessible-what we did for app building with Airtable, we're now doing for agents with Superagents.
0:19

Really excited about our launch of Superagent today! Powered by the latest AI models, agents have reached a breakthrough moment where they can do incredible work, with high ease of use--without requiring complicated tuning, prompting, or configuration. Superagent represents the freeform agent that can research anything (and soon your own company's context) and output an incredible, interactive webpage. It's like having your own personal NYTimes-quality data viz/web team building bespoke pages for you. This is the perfect complement to the structured system of operations for the AI era that Airtable has become, and we will be launching more integrations between the two products in the near future. Think: launch Superagent tasks from within Airtable records or Airtable Omni, or have Superagent output/edit/read data into Airtable bases! Excited for this new frontier of breakthrough agents, and applying our product and design philosophy of making powerful capabilities intuitive and accessible-what we did for app building with Airtable, we're now doing for agents with Superagents.

Howie Liu

10,125 views • 5 months ago

📢 Our lab has been exploring 3D world models for years — and we’re thrilled to share **PhysTwin**: a milestone that reconstructs object appearance, geometry, and dynamics from just a few seconds of interaction! Led by the amazing Hanxiao Jiang 👉 PhysTwin combines **Gaussian splatting** with **inverse dynamics optimization** based on simple **spring-mass** systems. ⚙️ The result? Real-time, action-conditioned 3D video prediction under novel interactions (i.e., 3D world models). 🔑 A few key takeaways: 1. Having the right structure (e.g., particles/masses) helps navigate the trade-off between sample efficiency, generalization, and broad applicability. 2. Visual foundation models (VFMs) have matured to the point where they can provide rich supervision for world modeling (e.g., tracking, shape completion). 3. Beyond VFMs, many crucial components have come together in recent years: Gaussian splats for rendering, NVIDIA Warp for high-performance simulation, and scene/asset generation from a wide range of labs and companies. The future of 3D world models is looking bright! ✨ 4. The resulting digital twin supports a wide range of downstream applications—especially in data generation and policy evaluation, thanks to its realistic rendering and simulation capabilities. 🎥 All code and data to reproduce the results, along with interactive demos, are available on the website. Check the following visualizations of: (1) observations, (2) reconstructed state/actions, (3) interactive digital twins, and (4) the overlays between real-world robot teleoperation and our model’s open-loop predictions.
0:05

📢 Our lab has been exploring 3D world models for years — and we’re thrilled to share **PhysTwin**: a milestone that reconstructs object appearance, geometry, and dynamics from just a few seconds of interaction! Led by the amazing Hanxiao Jiang 👉 PhysTwin combines **Gaussian splatting** with **inverse dynamics optimization** based on simple **spring-mass** systems. ⚙️ The result? Real-time, action-conditioned 3D video prediction under novel interactions (i.e., 3D world models). 🔑 A few key takeaways: 1. Having the right structure (e.g., particles/masses) helps navigate the trade-off between sample efficiency, generalization, and broad applicability. 2. Visual foundation models (VFMs) have matured to the point where they can provide rich supervision for world modeling (e.g., tracking, shape completion). 3. Beyond VFMs, many crucial components have come together in recent years: Gaussian splats for rendering, NVIDIA Warp for high-performance simulation, and scene/asset generation from a wide range of labs and companies. The future of 3D world models is looking bright! ✨ 4. The resulting digital twin supports a wide range of downstream applications—especially in data generation and policy evaluation, thanks to its realistic rendering and simulation capabilities. 🎥 All code and data to reproduce the results, along with interactive demos, are available on the website. Check the following visualizations of: (1) observations, (2) reconstructed state/actions, (3) interactive digital twins, and (4) the overlays between real-world robot teleoperation and our model’s open-loop predictions.

Yunzhu Li

25,279 views • 1 year ago

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kinds of interesting ways to work with agents. This is clearly the future. Check out this writing and scheduler artifact I built in a few minutes. It uses a bit of HTML and JS. All the data is in markdown (Obsidian vaults), so the agent can access and modify it at any time. No DB needed. No sophisticated functionalities. The agent decides all that for me based on the skills, context, and memory it has access to. The best part about this simple stack is that all the important information stays with me. This has allowed me to build a recursive self-improving system and automations that can better tap into coding agents like Codex or Claude Code. I could have paid or built an entire app for scheduling posts, and there are so many of them out there. But I don't need to. I've realized a simple artifact does the job. And the simplicity of it is actually an advantage. Very little maintenance for very high returns on personalization, time, and efficiency. The other benefit of this is that I can add features as I please. That level of personalization feels magical, and we should all be pursuing more of it. All of this just keeps compounding. Of course, this example is just about writing. But I have similar artifacts for research, design, experimentation, evaluation, and so much more. And no, I didn't actually publish the post example I shared in the clip. It was just for demonstration purposes. I actually spend more time than this when writing together with agents. Lastly, having built my own agent orchestrator tool has made me realize that simplifying the tool stack is a superpower. If you are curious about how all this works, I will do a live session next week:
0:16

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kinds of interesting ways to work with agents. This is clearly the future. Check out this writing and scheduler artifact I built in a few minutes. It uses a bit of HTML and JS. All the data is in markdown (Obsidian vaults), so the agent can access and modify it at any time. No DB needed. No sophisticated functionalities. The agent decides all that for me based on the skills, context, and memory it has access to. The best part about this simple stack is that all the important information stays with me. This has allowed me to build a recursive self-improving system and automations that can better tap into coding agents like Codex or Claude Code. I could have paid or built an entire app for scheduling posts, and there are so many of them out there. But I don't need to. I've realized a simple artifact does the job. And the simplicity of it is actually an advantage. Very little maintenance for very high returns on personalization, time, and efficiency. The other benefit of this is that I can add features as I please. That level of personalization feels magical, and we should all be pursuing more of it. All of this just keeps compounding. Of course, this example is just about writing. But I have similar artifacts for research, design, experimentation, evaluation, and so much more. And no, I didn't actually publish the post example I shared in the clip. It was just for demonstration purposes. I actually spend more time than this when writing together with agents. Lastly, having built my own agent orchestrator tool has made me realize that simplifying the tool stack is a superpower. If you are curious about how all this works, I will do a live session next week:

elvis

18,374 views • 2 months ago

Its not every day you wake up to find that the Pope has made your lifes work the central focus of his papacy: “Disarming AI means freeing it from the mentality of “armed” competition [..] This entails a race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance." - POPE LEO XIV, May 2026 Here, “disarmed” means “neutralised” in the sense that this should not be a differentiator The Innovation Game (TIG) was created to keep data and algorithms open, in order to prevent monopolistic control It's not just an aspiration, it's an economic mechanism that makes open data and open algorithms the rational economic choice • All algorithms are published openly by TIG • If you are willing to make the data you process with an algorithm open, you can use it free-of-charge • Alternatively, if you would like to keep this data private, there is a fee to pay for using the algorithm • All fees are used to fund more open innovation Its an elegant, global, self-reinforcing engine The logical end point of monopoly is that innovation stops We cannot allow that to happen Pope Leo XIV I would be grateful for your thoughts on The Innovation Game
0:18

Its not every day you wake up to find that the Pope has made your lifes work the central focus of his papacy: “Disarming AI means freeing it from the mentality of “armed” competition [..] This entails a race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance." - POPE LEO XIV, May 2026 Here, “disarmed” means “neutralised” in the sense that this should not be a differentiator The Innovation Game (TIG) was created to keep data and algorithms open, in order to prevent monopolistic control It's not just an aspiration, it's an economic mechanism that makes open data and open algorithms the rational economic choice • All algorithms are published openly by TIG • If you are willing to make the data you process with an algorithm open, you can use it free-of-charge • Alternatively, if you would like to keep this data private, there is a fee to pay for using the algorithm • All fees are used to fund more open innovation Its an elegant, global, self-reinforcing engine The logical end point of monopoly is that innovation stops We cannot allow that to happen Pope Leo XIV I would be grateful for your thoughts on The Innovation Game

John Fletcher (𝔦, 𝔦)

11,722 views • 1 month ago

Google dropped a new AI paper called LUMIERE. It's remarkably flexible, supporting video inpainting, image-to-video, AND stylized video generation tasks. Say hello to “space-time diffusion” for video generation! Now what the heck does that mean exactly?! 🌐⏳ → TL;DR it utilizes a “Space-Time UNet” architecture that generates the full duration of the video in one pass, rather than generating distant keyframes and interpolating between them like prior works. Because the computation is done in this “compressed space-time representation” to generate the full clip at once, it's far more temporally consistent. → Another benefit of generating the full video at once is that you can “direct” the video generation, making it easier to hand off to other models/tasks without having to stitch together partial solutions. You can condition generations on additional inputs, meaning you get the full stack of AI video capabilities – from video inpainting to image-to-video and beyond. → New SOTA for AI video generation? User study results in the paper suggest human evaluators preferred Lumiere over Runway Gen-2, Pika Labs, and Stable Video Diffusion in terms of quality, text alignment AND motion. But as always, we need to get hands-on with this tech when Google *actually* decides to ship it. → Could this end up inside YouTube? Y’all know i’m obsessed with blending reality and imagination – so it’s the video inpainting tech I'm most excited about. I really hope this model finds its way into YouTube's Generative AI efforts, and based on their prior announcements and the list of acknowledgments in the paper I think it might! 🤞🏽 Links: 🔗Paper: 🔗Project:
0:13

Google dropped a new AI paper called LUMIERE. It's remarkably flexible, supporting video inpainting, image-to-video, AND stylized video generation tasks. Say hello to “space-time diffusion” for video generation! Now what the heck does that mean exactly?! 🌐⏳ → TL;DR it utilizes a “Space-Time UNet” architecture that generates the full duration of the video in one pass, rather than generating distant keyframes and interpolating between them like prior works. Because the computation is done in this “compressed space-time representation” to generate the full clip at once, it's far more temporally consistent. → Another benefit of generating the full video at once is that you can “direct” the video generation, making it easier to hand off to other models/tasks without having to stitch together partial solutions. You can condition generations on additional inputs, meaning you get the full stack of AI video capabilities – from video inpainting to image-to-video and beyond. → New SOTA for AI video generation? User study results in the paper suggest human evaluators preferred Lumiere over Runway Gen-2, Pika Labs, and Stable Video Diffusion in terms of quality, text alignment AND motion. But as always, we need to get hands-on with this tech when Google *actually* decides to ship it. → Could this end up inside YouTube? Y’all know i’m obsessed with blending reality and imagination – so it’s the video inpainting tech I'm most excited about. I really hope this model finds its way into YouTube's Generative AI efforts, and based on their prior announcements and the list of acknowledgments in the paper I think it might! 🤞🏽 Links: 🔗Paper: 🔗Project:

Bilawal Sidhu

44,822 views • 2 years ago