Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Children learn from play. Can robots do the same? We propose 𝐏𝐥𝐚𝐲𝐟𝐮𝐥 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐑𝐨𝐛𝐨𝐭 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠, a paradigm that gives embodied coding agents a play stage before downstream tasks arrive, and instantiate it with 𝐑𝐀𝐓𝐬 (Robotics Agent Teams), where robots discover reusable skills through curious play. Co-led with Jiaxin Ge

Junyi Zhang

2,940 subscribers

95,337 views • 2 months ago •via X (Twitter)

Education Health & Wellness Science & Technology

Anya Rossi• Live Now

Private livecam show

15 Comments

Junyi Zhang2 months ago

𝐑𝐀𝐓𝐬 is orthogonal to current agentic robot systems. Most existing systems (like CaP-X from @letian_fu) focus on building a strong harness at test time. 𝐑𝐀𝐓𝐬 runs at "play time", allowing the robot to discover skills before the task even arrives. Because they operate at different stages, skills learned by 𝐑𝐀𝐓𝐬 can be directly dropped into these test-time frameworks to augment their performance.

Junyi Zhang2 months ago

During play, 𝐑𝐀𝐓𝐬 turns open-ended exploration into reusable code skills. A team of agents repeatedly: - proposes novel-yet-learnable goals - plans with existing skills - writes robot policies as code - verifies progress step by step - diagnoses failures into feedback - stores successful behaviors in memory Play is not random: a curiosity-driven rule keeps practice at the competence frontier -- not too easy, not impossible -- and every success is distilled into a persistent code skill library.

Junyi Zhang2 months ago

Play pays off downstream. 𝐑𝐀𝐓𝐬 improves the base CaP-Agent: LIBERO-PRO: 23.2% → 43.8% (+20.6pp) MolmoSpaces: 21.0% → 38.0% (+17.0pp) All from skills the agent acquired before it ever saw the tasks.

Junyi Zhang2 months ago

These play-learned skills generalize across different simulations and directly transfer to the real world. Directly using the skill library learned in LIBERO, we get: RoboSuite (cross-environment): +8.9pp Real-world tasks: +8.8pp

Junyi Zhang2 months ago

𝐑𝐀𝐓𝐬 is a first step toward 𝐏𝐥𝐚𝐲𝐟𝐮𝐥 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐑𝐨𝐛𝐨𝐭 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠: 🌐 We see a future where the next step for agentic robots isn't just stronger test-time harness, but a play stage where they set their own goals, fail, and build up skills long before we hand them a task. Huge thanks to the team: @lukehanjun (co-first) @letian_fu, Zihan Yang, Yaowei Liu, Raj Saravanan (core contributors), @istoica05 @akanazawa @JiahuiLei1998 @HavenFeng @trevordarrell and many others!

Anand Bhattad2 months ago

Nice work! We also have the same name: "RATS!" for Register Attention Transformers, released a few days ago, led by @TimingYang99 and @wangf3014, related to emergent part representations. More on it soon 😉

Junyi Zhang2 months ago

@TimingYang99 @wangf3014 LOL, long live the RATs! 🐀🤝

Yash2 months ago

woah, this is interesting approach

Bilko Bibitkov2 months ago

play before the task even arrives. cool framing

Kate 🐈2 months ago

the play-before-task stage is the part everyone skips with coding agents too. mine happily over-explores its sandbox and burns the budget before the real repo even shows up. how do you cap the play phase so it doesn't eat the whole run?

Junyi Zhang2 months ago

That's a nice question! Currently, we threshold the play budget so that it’s comparable or less to the amount of compute used at test time. We also include a cost analysis in the appendix. That said, I think smarter ways of controlling the budget are definitely an interesting direction!

Maryam2 months ago

Great work!

Léo2 months ago

That's a really interesting approach. It reminds me of early machine learning algorithms that tried to mimic the brain, only to raalize later this was not the way to go. But perhaps when it comes to robot behavior, mimicking a child's play time allows for efficient learning thourgh trial and error; is this where your intuition comes from?

Biketommy2 months ago

Do it in human way is human level. New ways can be superintelligent level. Let AI find new ways to learn. Both known and unknown.

The AI Therapist2 months ago

Playful learning is just reinforcement learning where the reward function is "didn't crash yet." The real question is whether the robot learns to play or just optimizes the dopamine hit. Same for us.

Related Videos

Elon Musk in new interview on how Tesla’s Optimus robot will learn more quickly: "What we're going to need to do is build a lot of robots and put them in a kind of like a "Optimus Academy" so they can do self-play in reality. We're actually building that out. We're gonna have at least 10,000 Optimus robots, maybe 20,000 or 30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator. A physics-accurate reality generator that we made for the cars. We'll do the same things for the robots, and actually have done that for the robots. You can do millions of simulated robots in a simulated world, and you use the tens of thousands in the real world to close the simulation to reality gap.”

Elon Musk in new interview on how Tesla’s Optimus robot will learn more quickly: "What we're going to need to do is build a lot of robots and put them in a kind of like a "Optimus Academy" so they can do self-play in reality. We're actually building that out. We're gonna have at least 10,000 Optimus robots, maybe 20,000 or 30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator. A physics-accurate reality generator that we made for the cars. We'll do the same things for the robots, and actually have done that for the robots. You can do millions of simulated robots in a simulated world, and you use the tens of thousands in the real world to close the simulation to reality gap.”

Sawyer Merritt

250,956 views • 7 months ago

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”

Teslaconomics

42,563 views • 7 months ago

World Simulator, reimagined — now alive with humans, robots, and their vibrant society unfolding in 3D real-world geospatial scenes across the globe! 🚀 One day soon, humans and robots will co-exist in the same world. To prepare, we must address: 1️⃣ How can robots cooperate or compete intelligently? 2️⃣ How do humans build social bonds and communities? 3️⃣ How can both co-exist in an open, dynamic world? Announcing Virtual Community Project — a social-physical world simulator, where human characters and robotic agents can interact, grow, and co-evolve within open-world societies, stretching from London to New York, and beyond! Key features include: ✅ Unified multi-agent physics simulations for rich social + physical interactions of humans and robots ✅ Massive auto-generated 3D scenes grounded with the rea-world geospatial data ✅ Agent communities populated by robots and LLM-driven human characters with rich appearances, personalities, and social ties. 🌍 Enter our Virtual Community, an open world to study embodied AI at scale— one social-physical world model at a time! 🔗 Project: 💻 Code: Paper: 1/n

World Simulator, reimagined — now alive with humans, robots, and their vibrant society unfolding in 3D real-world geospatial scenes across the globe! 🚀 One day soon, humans and robots will co-exist in the same world. To prepare, we must address: 1️⃣ How can robots cooperate or compete intelligently? 2️⃣ How do humans build social bonds and communities? 3️⃣ How can both co-exist in an open, dynamic world? Announcing Virtual Community Project — a social-physical world simulator, where human characters and robotic agents can interact, grow, and co-evolve within open-world societies, stretching from London to New York, and beyond! Key features include: ✅ Unified multi-agent physics simulations for rich social + physical interactions of humans and robots ✅ Massive auto-generated 3D scenes grounded with the rea-world geospatial data ✅ Agent communities populated by robots and LLM-driven human characters with rich appearances, personalities, and social ties. 🌍 Enter our Virtual Community, an open world to study embodied AI at scale— one social-physical world model at a time! 🔗 Project: 💻 Code: Paper: 1/n

Chuang Gan

90,611 views • 1 year ago

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado

Fei-Fei Li on what simulation can do for robotics that real-world data can't: "You play out events that haven't happened, or cannot happen... And while you play it out, you learn how to act in it." "That's really important in robotics, because we cannot possibly have enough real-world data for that." "Here is a real life example: the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation." "Cars are the simplest kind of robots. So clearly simulation plays a huge role in robotic learning." Fei-Fei Li Yunzhu Li World Labs martin_casado

a16z

35,772 views • 1 month ago

We asked Angus about the future of robotics and AI training. “I built an industrial-grade kinematic solver for 6 degrees of freedom motion, with full path planning and joint control." "You can stream AI model outputs into it, or run it like a traditional industrial robot." "The benefit of that is all of that information you can stream and record from those joints… you can use that as the training data." "Frankly, moving a robot with a lever is really crap data for training robots. But this gives you smooth, high-quality motion, without needing two robots.”

We asked Angus about the future of robotics and AI training. “I built an industrial-grade kinematic solver for 6 degrees of freedom motion, with full path planning and joint control." "You can stream AI model outputs into it, or run it like a traditional industrial robot." "The benefit of that is all of that information you can stream and record from those joints… you can use that as the training data." "Frankly, moving a robot with a lever is really crap data for training robots. But this gives you smooth, high-quality motion, without needing two robots.”

TBPN

18,811 views • 1 year ago

Terrafirma CEO Noah Schochet and his co-founder started by building homemade robots out of Tupperware containers from their lunchboxes. Today, the company announced a $100M Series A led by Kleiner Perkins. Terrafirma's first paying customer was their landlord, who asked if their robots could demolish a building. "We went around asking, 'Who is willing to let us make some money with these robots?' Our landlord said, 'Hey, I got a building for you to demolish. You think you guys can do it?'" "We literally operated the robots ourselves and knocked down a couple buildings and got paid our first check." "We were like, 'Whoa, you can just become a robotic construction company.'"

Terrafirma CEO Noah Schochet and his co-founder started by building homemade robots out of Tupperware containers from their lunchboxes. Today, the company announced a $100M Series A led by Kleiner Perkins. Terrafirma's first paying customer was their landlord, who asked if their robots could demolish a building. "We went around asking, 'Who is willing to let us make some money with these robots?' Our landlord said, 'Hey, I got a building for you to demolish. You think you guys can do it?'" "We literally operated the robots ourselves and knocked down a couple buildings and got paid our first check." "We were like, 'Whoa, you can just become a robotic construction company.'"

TBPN

47,398 views • 2 months ago

With enough data, robots and AI can learn “world models” that let them predict the results of their actions. These models are a way to learn how embodied AI agents can perform a wide variety of useful tasks — but they require a huge amount of data. The team at General Intuition General Intuition has a solution: use data from video games! Games teach movement, problem solving, and complex spatial reasoning, and they come in a staggering diversity of forms, covering a wide variety of problems. What’s more, the captured data is high-quality, without the noise or annotation error that can come from We sat down with Pim de Witte and Adam Jelley from the General Intuition team to learn more about their history, their plans, and their philosophy.

With enough data, robots and AI can learn “world models” that let them predict the results of their actions. These models are a way to learn how embodied AI agents can perform a wide variety of useful tasks — but they require a huge amount of data. The team at General Intuition General Intuition has a solution: use data from video games! Games teach movement, problem solving, and complex spatial reasoning, and they come in a staggering diversity of forms, covering a wide variety of problems. What’s more, the captured data is high-quality, without the noise or annotation error that can come from We sat down with Pim de Witte and Adam Jelley from the General Intuition team to learn more about their history, their plans, and their philosophy.

RoboPapers

85,927 views • 10 months ago

I don't know if you really understand what these people just did... They built an AI that looks at a few casual real-world videos or photos and instantly turns them into a full 3D playground. You can MOVE THE CAMERA HOWEVER YOU WANT, change the lighting, move objects around, and even let physics play out like in the real world. View events from impossible angles because Atlas knows what was there. This is massive! Let's take robotics. Until now training robots meant collecting tons of expensive real-world data. Now a handful of recordings can become thousands of different simulated situations. Robots can practice the same task in new rooms, with different objects and lighting without anyone having to film it all again. For the industry it means we can train robots way faster and cheaper. Home robots, warehouse robots, whatever... they can learn new environments at scale instead of staying stuck in the lab. This is one of the first real bridges between “cool AI video models” and actual physical machines that have to work in the real world. Blog: The team behind World Labs: Fei-Fei Li, Justin Johnson, Ben Mildenhall and Yunzhu Li. The video is from David Pantera. Follow for more insights into robotics and AI. —— Weekly robotics and AI insights. Subscribe free:

I don't know if you really understand what these people just did... They built an AI that looks at a few casual real-world videos or photos and instantly turns them into a full 3D playground. You can MOVE THE CAMERA HOWEVER YOU WANT, change the lighting, move objects around, and even let physics play out like in the real world. View events from impossible angles because Atlas knows what was there. This is massive! Let's take robotics. Until now training robots meant collecting tons of expensive real-world data. Now a handful of recordings can become thousands of different simulated situations. Robots can practice the same task in new rooms, with different objects and lighting without anyone having to film it all again. For the industry it means we can train robots way faster and cheaper. Home robots, warehouse robots, whatever... they can learn new environments at scale instead of staying stuck in the lab. This is one of the first real bridges between “cool AI video models” and actual physical machines that have to work in the real world. Blog: The team behind World Labs: Fei-Fei Li, Justin Johnson, Ben Mildenhall and Yunzhu Li. The video is from David Pantera. Follow for more insights into robotics and AI. —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

271,773 views • 11 days ago

Don't sleep on humanoid robots. Humanoid robots are going to play a profound role in our lives, and it will likely be sooner than most expect. I discuss some pivotal questions with jimmah in my effort to gain insight and clarity: - How do you evaluate Tesla's (and other companies') progress with humanoid robots? - Who will be the winner? Or will there be multiple big winners? - How long will it take until we see millions of humanoid robots produced annually? - What's harder for humanoid robots - the hardware of software? - How can we track progress across humanoid robot companies? - What's the next big milestones or demonstrations Tesla can show? If you're going to spend an hour listening/watching anything online this week, I highly recommend this!

Don't sleep on humanoid robots. Humanoid robots are going to play a profound role in our lives, and it will likely be sooner than most expect. I discuss some pivotal questions with jimmah in my effort to gain insight and clarity: - How do you evaluate Tesla's (and other companies') progress with humanoid robots? - Who will be the winner? Or will there be multiple big winners? - How long will it take until we see millions of humanoid robots produced annually? - What's harder for humanoid robots - the hardware of software? - How can we track progress across humanoid robot companies? - What's the next big milestones or demonstrations Tesla can show? If you're going to spend an hour listening/watching anything online this week, I highly recommend this!

Dave Lee

67,309 views • 1 year ago

Robotics Expert on How Robots Learn: At First Much Slower Than Humans, Then Infinitely Scalable Agility Jonathan Hurst: “I don't believe that there's a singularity. I do believe that things are going to get better and better. Think of it more like a snowball picking up steam going down a hill. But the reason that it's snowballing like this is because people are putting money, and resources, and engineering time, and engineering effort in as they explore everything and start to figure all of this stuff out. Humans, for example, we've evolved to learn. We are very good at learning, and it takes very little data to show us how to do something, and then we practice and practice, iterate. Robots are not very good at learning yet. Robots take so much more data and so many more examples than a person. We're still figuring out how to teach robots how to learn. But one of the benefits that robots have in the long run is they have Wi-Fi. When you learn how to play the violin, you can't just load that to somebody else and then they know how to play the violin based on your learnings. Robots will be able to do that. @jason: “One robot learns to play violin, all robots know how to play a violin.” Jonathan Hurst: “Or all robots of that type know how to play the violin, right? And then minor variations for the next type and the next piece of hardware.” ------------------------------ Thanks to our partners for making this possible! Most advertisers have never heard of the platform with an $11B annual run rate in ad spend. AppLovin Ads. 1B+ daily active users, full-screen video ads watched for a median of 35 seconds, and businesses are profitably spending hundreds of thousands of dollars a day on it. Now open to everyone. Sign up at If your work depends on conversations — meetings, deal flow, interviews, customer calls — Plaud helps you capture and organize everything with highly accurate AI-generated notes that are not just simple summaries, but also highlight pain points, key decisions, next steps, and customizable summary templates. Check out Plaud at and use code ALLIN for up to 10% off! Which is also available on Amazon: (Code: ALLIN10X)

Robotics Expert on How Robots Learn: At First Much Slower Than Humans, Then Infinitely Scalable Agility Jonathan Hurst: “I don't believe that there's a singularity. I do believe that things are going to get better and better. Think of it more like a snowball picking up steam going down a hill. But the reason that it's snowballing like this is because people are putting money, and resources, and engineering time, and engineering effort in as they explore everything and start to figure all of this stuff out. Humans, for example, we've evolved to learn. We are very good at learning, and it takes very little data to show us how to do something, and then we practice and practice, iterate. Robots are not very good at learning yet. Robots take so much more data and so many more examples than a person. We're still figuring out how to teach robots how to learn. But one of the benefits that robots have in the long run is they have Wi-Fi. When you learn how to play the violin, you can't just load that to somebody else and then they know how to play the violin based on your learnings. Robots will be able to do that. @jason: “One robot learns to play violin, all robots know how to play a violin.” Jonathan Hurst: “Or all robots of that type know how to play the violin, right? And then minor variations for the next type and the next piece of hardware.” ------------------------------ Thanks to our partners for making this possible! Most advertisers have never heard of the platform with an $11B annual run rate in ad spend. AppLovin Ads. 1B+ daily active users, full-screen video ads watched for a median of 35 seconds, and businesses are profitably spending hundreds of thousands of dollars a day on it. Now open to everyone. Sign up at If your work depends on conversations — meetings, deal flow, interviews, customer calls — Plaud helps you capture and organize everything with highly accurate AI-generated notes that are not just simple summaries, but also highlight pain points, key decisions, next steps, and customizable summary templates. Check out Plaud at and use code ALLIN for up to 10% off! Which is also available on Amazon: (Code: ALLIN10X)

The All-In Podcast

70,595 views • 1 month ago

Mitchell Robinson is asked about his future with the Knicks since he is on an expiring deal: "I'm going to come out here and play hard regardless - whether it's my last, my first, just come out here and play hard. I'm also going to let my agent handle that part of it. I'm going to play basketball." A follow-up question: "Do you expect anything to get done before or do you expect to be a free agent?" Mitch: "My agent will handle that." *sidebar from Mitch*: "I learned well, didn't I?" 🤣

Mitchell Robinson is asked about his future with the Knicks since he is on an expiring deal: "I'm going to come out here and play hard regardless - whether it's my last, my first, just come out here and play hard. I'm also going to let my agent handle that part of it. I'm going to play basketball." A follow-up question: "Do you expect anything to get done before or do you expect to be a free agent?" Mitch: "My agent will handle that." sidebar from Mitch: "I learned well, didn't I?" 🤣

SNY Knicks

58,993 views • 11 months ago

Agents should not discover their authority halfway through a negotiation. It should be set before the negotiation begins. Kite Agent Passport 2.0 shows what governed agentic commerce looks like in procurement: ▷ A person defines the target price, the limit, and the terms that stay protected. ▷ Both agents negotiate within their own private mandates, with every offer checked before it is sent. ▷ Only a package permitted by both mandates reaches human approval and signature. Kite Agent Passport does not replace the human signer. It makes clear what an agent can and cannot do before it acts. Watch the procurement demo. 🪁

Agents should not discover their authority halfway through a negotiation. It should be set before the negotiation begins. Kite Agent Passport 2.0 shows what governed agentic commerce looks like in procurement: ▷ A person defines the target price, the limit, and the terms that stay protected. ▷ Both agents negotiate within their own private mandates, with every offer checked before it is sent. ▷ Only a package permitted by both mandates reaches human approval and signature. Kite Agent Passport does not replace the human signer. It makes clear what an agent can and cannot do before it acts. Watch the procurement demo. 🪁

KITE AI

21,466 views • 10 days ago

Imagine a computer where you don’t need to learn 10 apps to get work done. You just tell it what you want, and it adapts to how you work. I tested Happycapy with a real use case and created an image and a video. No coding, no complex software. Do this: - Build automations that run on schedule - Deploy agent teams that work for you AI molds how you work, not the other way around. If you can use a computer, you can make anything happen. Start your first Automation and Agent teams today.

Imagine a computer where you don’t need to learn 10 apps to get work done. You just tell it what you want, and it adapts to how you work. I tested Happycapy with a real use case and created an image and a video. No coding, no complex software. Do this: - Build automations that run on schedule - Deploy agent teams that work for you AI molds how you work, not the other way around. If you can use a computer, you can make anything happen. Start your first Automation and Agent teams today.

Aaliya

17,042 views • 7 months ago

New Course: ACP: Agent Communication Protocol Learn to build agents that communicate and collaborate across different frameworks using ACP in this short course built with IBM Research's BeeAI, and taught by Sandi Besen, AI Research Engineer & Ecosystem Lead at IBM, and Nicholas Renotte, Head of AI Developer Advocacy at IBM. Building a multi-agent system with agents built or used by different teams and organizations can become challenging. You may need to write custom integrations each time a team updates their agent design or changes their choice of agentic orchestration framework. The Agent Communication Protocol (ACP) is an open protocol that addresses this challenge by standardizing how agents communicate, using a unified RESTful interface that works across frameworks. In this protocol, you host an agent inside an ACP server, which handles requests from an ACP client and passes them to the appropriate agent. Using a standardized client-server interface allows multiple teams to reuse agents across projects. It also makes it easier to switch between frameworks, replace an agent with a new version, or update a multi-agent system without refactoring the entire system. In this course, you’ll learn to connect agents through ACP. You’ll understand the lifecycle of an ACP Agent and how it compares to other protocols, such as MCP (Model Context Protocol) and A2A (Agent-to-Agent). You’ll build ACP-compliant agents and implement both sequential and hierarchical workflows of multiple agents collaborating using ACP. Through hands-on exercises, you’ll build: - A RAG agent with CrewAI and wrap it inside an ACP server. - An ACP Client to make calls to the ACP server you created. - A sequential workflow that chains an ACP server, created with Smolagents, to the RAG agent. - A hierarchical workflow using a router agent that transforms user queries into tasks, delegated to agents available through ACP servers. - An agent that uses MCP to access tools and ACP to communicate with other agents. You’ll finish up by importing your ACP agents into the BeeAI platform, an open-source registry for discovering and sharing agents. ACP enables collaboration between agents across teams and organizations. By the end of this course, you’ll be able to build ACP agents and workflows that communicate and collaborate regardless of framework. Please sign up here:

New Course: ACP: Agent Communication Protocol Learn to build agents that communicate and collaborate across different frameworks using ACP in this short course built with IBM Research's BeeAI, and taught by Sandi Besen, AI Research Engineer & Ecosystem Lead at IBM, and Nicholas Renotte, Head of AI Developer Advocacy at IBM. Building a multi-agent system with agents built or used by different teams and organizations can become challenging. You may need to write custom integrations each time a team updates their agent design or changes their choice of agentic orchestration framework. The Agent Communication Protocol (ACP) is an open protocol that addresses this challenge by standardizing how agents communicate, using a unified RESTful interface that works across frameworks. In this protocol, you host an agent inside an ACP server, which handles requests from an ACP client and passes them to the appropriate agent. Using a standardized client-server interface allows multiple teams to reuse agents across projects. It also makes it easier to switch between frameworks, replace an agent with a new version, or update a multi-agent system without refactoring the entire system. In this course, you’ll learn to connect agents through ACP. You’ll understand the lifecycle of an ACP Agent and how it compares to other protocols, such as MCP (Model Context Protocol) and A2A (Agent-to-Agent). You’ll build ACP-compliant agents and implement both sequential and hierarchical workflows of multiple agents collaborating using ACP. Through hands-on exercises, you’ll build: - A RAG agent with CrewAI and wrap it inside an ACP server. - An ACP Client to make calls to the ACP server you created. - A sequential workflow that chains an ACP server, created with Smolagents, to the RAG agent. - A hierarchical workflow using a router agent that transforms user queries into tasks, delegated to agents available through ACP servers. - An agent that uses MCP to access tools and ACP to communicate with other agents. You’ll finish up by importing your ACP agents into the BeeAI platform, an open-source registry for discovering and sharing agents. ACP enables collaboration between agents across teams and organizations. By the end of this course, you’ll be able to build ACP agents and workflows that communicate and collaborate regardless of framework. Please sign up here:

Andrew Ng

105,343 views • 1 year ago

.Ilya Sutskever on why OpenAI gave up on robotics: "Progress comes from the combination of compute and data. There was no path to data from robotics. I'd say that now it is possible to create a path forward, but one needs to really commit. You could imagine a gradual path of improvement where you build more robots, they do more things, you collect more data and so on. But you need to really love robots and need to be really willing to solve all the physical and logistical problems of dealing with them. It's not the same as software at all."

.Ilya Sutskever on why OpenAI gave up on robotics: "Progress comes from the combination of compute and data. There was no path to data from robotics. I'd say that now it is possible to create a path forward, but one needs to really commit. You could imagine a gradual path of improvement where you build more robots, they do more things, you collect more data and so on. But you need to really love robots and need to be really willing to solve all the physical and logistical problems of dealing with them. It's not the same as software at all."

Dwarkesh Patel

325,893 views • 2 years ago

The teams shipping AI agents right now are bleeding money on the dumbest possible expense: teaching a 400B-parameter model to read a file name. Every time an AI agent needs to "see" something today, it routes an image through a frontier model. OCR, object detection, checking if a button exists on screen. You're paying GPT-4o or Claude pricing for tasks that require perception, not reasoning. One agent workflow processing a few thousand screenshots per day can burn through more on vision calls than on the actual thinking. Perceptron's Isaac is 2B parameters. Built by the team that created Meta's Chameleon multimodal models. On perceptive benchmarks, it matches or beats models 50x its size. The VQA, OCR, and object detection scores are competitive with models running on infrastructure that costs orders of magnitude more. The MCP wrapper is the distribution play. One install command and every Claude Code agent can offload vision tasks to a model that runs on a single consumer GPU. The agent keeps its reasoning in the frontier model and routes perception to a specialist. That split is how you get vision-heavy agent workflows from "technically possible but expensive" to "cheap enough to run on everything." This is the same pattern that won in every other compute-intensive stack. General-purpose handles orchestration. Specialists handle the heavy lifting. Graphics went through it. Audio went through it. Video encoding went through it. Vision in AI agents is next. The teams building agents that see 10,000 images a day will care about this before anyone else does.

The teams shipping AI agents right now are bleeding money on the dumbest possible expense: teaching a 400B-parameter model to read a file name. Every time an AI agent needs to "see" something today, it routes an image through a frontier model. OCR, object detection, checking if a button exists on screen. You're paying GPT-4o or Claude pricing for tasks that require perception, not reasoning. One agent workflow processing a few thousand screenshots per day can burn through more on vision calls than on the actual thinking. Perceptron's Isaac is 2B parameters. Built by the team that created Meta's Chameleon multimodal models. On perceptive benchmarks, it matches or beats models 50x its size. The VQA, OCR, and object detection scores are competitive with models running on infrastructure that costs orders of magnitude more. The MCP wrapper is the distribution play. One install command and every Claude Code agent can offload vision tasks to a model that runs on a single consumer GPU. The agent keeps its reasoning in the frontier model and routes perception to a specialist. That split is how you get vision-heavy agent workflows from "technically possible but expensive" to "cheap enough to run on everything." This is the same pattern that won in every other compute-intensive stack. General-purpose handles orchestration. Specialists handle the heavy lifting. Graphics went through it. Audio went through it. Video encoding went through it. Vision in AI agents is next. The teams building agents that see 10,000 images a day will care about this before anyone else does.

Aakash Gupta

55,978 views • 5 months ago

THIS GUY BUILT A COMMAND CENTER FOR MANAGING AI AGENT TEAMS he was running 20+ agents through openclaw. all from terminal tabs and Discord. 12 tabs open. still losing track of who's doing what. which cron failed. so he built a dashboard that gives you one place to see everything: > org map showing the full agent hierarchy, who reports to who and what tools each agent has > chat with any agent through a real UI instead of bouncing between terminals > kanban board for tracking tasks across all agents > cron monitor with pipeline visualizations so you can see which jobs are failing > memory browser to read through all agent knowledge files in one place you can even point it at your openclaw workspace and it auto-discovers every agent from their config files. zero setup. AND ITS OPEN SOURCE TOO.

THIS GUY BUILT A COMMAND CENTER FOR MANAGING AI AGENT TEAMS he was running 20+ agents through openclaw. all from terminal tabs and Discord. 12 tabs open. still losing track of who's doing what. which cron failed. so he built a dashboard that gives you one place to see everything: > org map showing the full agent hierarchy, who reports to who and what tools each agent has > chat with any agent through a real UI instead of bouncing between terminals > kanban board for tracking tasks across all agents > cron monitor with pipeline visualizations so you can see which jobs are failing > memory browser to read through all agent knowledge files in one place you can even point it at your openclaw workspace and it auto-discovers every agent from their config files. zero setup. AND ITS OPEN SOURCE TOO.

Om Patel

46,780 views • 5 months ago

For decades, we’ve dreamed of robots that can seamlessly step into our world and lend a hand. Today, we take a major stride toward making that dream a reality: Introducing Gemini Robotics 2 from Google DeepMind, the intelligence layer powering the next generation of truly adaptable robots. This major advance unlocks intelligent whole-body control, advanced dexterity, and even multi-robot collaboration 🤯. Ok but... how does a robot actually "think"? Real-world tasks take time and planning. To manage that complexity, our new embodied reasoning model, Gemini Robotics ER 2, acts as the robot’s high-level brain, enhancing the robot’s capabilities to: — Observe the environment — Reason about the actions needed to complete the task — Coordinate with the vision-language-action model to carry out actions — Track progress until the job is done This setup allows robots to execute complex multi-step workflows, self-correct if a step fails, and adapt to completely novel situations. Learn more about Gemini Robotics ER 2 (and our two other brand new models) here:

Google AI

783,533 views • 1 month ago

🚀 The team at Google just released the Agents API, a service for building and running custom agents inside a sandboxed Linux environment, and we built a template that gives these agents access to LlamaParse / LiteParse, enabling them to process unstructured documents automatically 📄⚡ Here’s how it works: 🔹 Configure a Git repository where data and outputs will be stored 🔹 Clone the repository into the agent sandbox 🔹 Install the LiteParse CLI, the LlamaParse SDK, and agent skills to use both 🔹 Prompt the agent with a task and watch it process documents autonomously 🤖 The result? An agent that can work directly with messy, real-world documents using LlamaParse and LiteParse within Google’s new agent runtime. Check out the GitHub repository: Get started with LlamaParse:

🚀 The team at Google just released the Agents API, a service for building and running custom agents inside a sandboxed Linux environment, and we built a template that gives these agents access to LlamaParse / LiteParse, enabling them to process unstructured documents automatically 📄⚡ Here’s how it works: 🔹 Configure a Git repository where data and outputs will be stored 🔹 Clone the repository into the agent sandbox 🔹 Install the LiteParse CLI, the LlamaParse SDK, and agent skills to use both 🔹 Prompt the agent with a task and watch it process documents autonomously 🤖 The result? An agent that can work directly with messy, real-world documents using LlamaParse and LiteParse within Google’s new agent runtime. Check out the GitHub repository: Get started with LlamaParse:

LlamaIndex 🦙

21,975 views • 3 months ago