Loading video...

Video Failed to Load

Go Home

Combinatorial complexity is often the bane of imitation learning - including VLA models! Jesse Zhang and Marius Memmel proposed a way around this, using VLMs to perform problem reduction for imitation. The insight is simple - 1) High-level VLM takes a complex scene/task and reducing it a minimal representation...

abhishekunique7's profile picture

Abhishek Gupta

10,867 subscribers

22,354 views • 10 months ago •via X (Twitter)

EducationHealth & WellnessScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!
1:11:51

In order for robots to be deployed in the real world, performing tasks of real value, they must be reliable. Unfortunately, even more, most robotic demos work maybe 70-80% of the time at best. The way to get better reliability is to do real-world reinforcement learning: having the robot teach itself how to perform the task up to a high level of success. The key to doing this is to start with a core of expert human data, use that to train a policy then iteratively improve it, until finally finishing with on-policy reinforcement learning. Kun Lei talks through a unified framework for imitation and reinforcement learning based on PPO, which enables this improvement process. In this episode, Kun Lei explains the theory behind his reinforcement learning method and how it allowed his robot to run in a shopping mall juicing oranges for seven hours at a time, among experiments on a wide variety of tasks and embodiments. Watch episode 58 of RoboPapers now, hosted by Michael Cho - Rbt/Acc and Chris Paxton!

RoboPapers

18,813 views • 7 months ago

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,334 views • 2 months ago

The next frontier after language models is world models, and learning to predict the next state of a world. Hear from our researchers how these models really work at a low-level and the mountain we're climbing. A general-purpose world simulator (AKA sci-fi) is now in sight.
17:45

The next frontier after language models is world models, and learning to predict the next state of a world. Hear from our researchers how these models really work at a low-level and the mountain we're climbing. A general-purpose world simulator (AKA sci-fi) is now in sight.

Odyssey

25,890 views • 8 months ago

The mere ~8% of the world population that is high-IQ, high-trust, culturally first-world is facing imminent, existential, genocidal erasure at the fists and feet of the ~92% of the world population that is low-IQ, low-trust, and culturally third-world. And it's all driven by a single factor--relative IQ--that is almost entirely genetically coded and not amenable to meaningful improvement by any means known to man. And the third-world HATES the first-world for this self-perceived inferiority--thus their mindless determination to destroy it.
2:42

The mere ~8% of the world population that is high-IQ, high-trust, culturally first-world is facing imminent, existential, genocidal erasure at the fists and feet of the ~92% of the world population that is low-IQ, low-trust, and culturally third-world. And it's all driven by a single factor--relative IQ--that is almost entirely genetically coded and not amenable to meaningful improvement by any means known to man. And the third-world HATES the first-world for this self-perceived inferiority--thus their mindless determination to destroy it.

Andrew Branca Show

218,710 views • 8 months ago

Why does the 92% of the human population that is largely the low-IQ, low-trust, high-fraud, non-white third-world hate and wish to genocide the mere 8% of the population that is high-IQ, high-trust, low-fraud, white first-world? BECAUSE we ARE the FIRST-WORLD. The retard-adjacent third-worlders with their unchangeable IQs in the 60s (Africa) and 70s (India) and 80s (Central America) KNOW that they can NEVER BE US, that they can NEVER DO WHAT WE ROUTINELY DO--establish first-world cultures and live at a first-world level. And in this modern age their inferiority is PATENTLY VISIBLE TO THEM IN EVERYTHING. Our VERY FIRST-WORLD EXISTENCE is a NEVER-ENDING HUMILIATION for them. So, OF COURSE, they HATE and wish to GENOCIDE us. FULL VIDEO HERE:
1:28

Why does the 92% of the human population that is largely the low-IQ, low-trust, high-fraud, non-white third-world hate and wish to genocide the mere 8% of the population that is high-IQ, high-trust, low-fraud, white first-world? BECAUSE we ARE the FIRST-WORLD. The retard-adjacent third-worlders with their unchangeable IQs in the 60s (Africa) and 70s (India) and 80s (Central America) KNOW that they can NEVER BE US, that they can NEVER DO WHAT WE ROUTINELY DO--establish first-world cultures and live at a first-world level. And in this modern age their inferiority is PATENTLY VISIBLE TO THEM IN EVERYTHING. Our VERY FIRST-WORLD EXISTENCE is a NEVER-ENDING HUMILIATION for them. So, OF COURSE, they HATE and wish to GENOCIDE us. FULL VIDEO HERE:

Andrew Branca Show

325,704 views • 7 months ago

For anyone to think they can deceive the international community in this digital age, you must either be out of touch or you believe everyone else is. The truth is simple: the world is watching, the world is informed, and you cannot hide reality anymore. The way out is to admit the truth, recognise the problem, and seek a real solution.
3:38

For anyone to think they can deceive the international community in this digital age, you must either be out of touch or you believe everyone else is. The truth is simple: the world is watching, the world is informed, and you cannot hide reality anymore. The way out is to admit the truth, recognise the problem, and seek a real solution.

DonAza

68,515 views • 8 months ago

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.
1:31

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.

MTS

11,641 views • 2 months ago

Yann LeCun (Yann LeCun) just revealed why he left Meta, Why are LLMs an extremely narrow field, and how "World model" is the way to build really meaningful agentic future. 🎯 Beautiful and simple in 1.5 mints. "I can’t imagine we can build agentic systems without those systems having the ability to predict, in advance, what the consequences of their actions are going to be. The way we act in the world is that we can predict the consequences of our actions, and that’s what allows us to plan. So what is a world model? Given the state of an environment, a system you want to control at time t, and given an action or intervention you imagine taking, can you predict the state of the world (or the system) at time t + 1? If you can, that’s a world model. You don’t do this at a pixel level, if it’s video. You do this in an abstract representation space, and that’s a crucial key insight." --- From 'AI House Davos" YT channel (full link in comment)
1:40

Yann LeCun (Yann LeCun) just revealed why he left Meta, Why are LLMs an extremely narrow field, and how "World model" is the way to build really meaningful agentic future. 🎯 Beautiful and simple in 1.5 mints. "I can’t imagine we can build agentic systems without those systems having the ability to predict, in advance, what the consequences of their actions are going to be. The way we act in the world is that we can predict the consequences of our actions, and that’s what allows us to plan. So what is a world model? Given the state of an environment, a system you want to control at time t, and given an action or intervention you imagine taking, can you predict the state of the world (or the system) at time t + 1? If you can, that’s a world model. You don’t do this at a pixel level, if it’s video. You do this in an abstract representation space, and that’s a crucial key insight." --- From 'AI House Davos" YT channel (full link in comment)

Rohan Paul

383,490 views • 6 months ago

President Zelenskyy: The task of Ukrainian units is to inflict a level of losses on the occupier at which Russian casualties exceed the amount of reinforcements they are able to send to their forces each month. This is a realistic task. When we speak of 50,000 Russian losses per month, that is the optimal level. It is a difficult task, without question, but it is precisely the level needed for Russia to begin weighing what it is doing and what it is fighting for. The task of the Ministry of Defense of Ukraine, our army, and all the Defense and Security Forces of Ukraine is to ensure exactly this level of Russian losses.
0:50

President Zelenskyy: The task of Ukrainian units is to inflict a level of losses on the occupier at which Russian casualties exceed the amount of reinforcements they are able to send to their forces each month. This is a realistic task. When we speak of 50,000 Russian losses per month, that is the optimal level. It is a difficult task, without question, but it is precisely the level needed for Russia to begin weighing what it is doing and what it is fighting for. The task of the Ministry of Defense of Ukraine, our army, and all the Defense and Security Forces of Ukraine is to ensure exactly this level of Russian losses.

Anton Gerashchenko

77,058 views • 6 months ago

PUTIN (2022?): "To a great extent, the level of prosperity that has been achieved in former colonial powers is grounded in the robbery of Africa, built on the grief and suffering of African people... Robbing and slavery, all of this has to be shown and taught as this is a crucial part of world history; to simply close the chapter on it is impossible and shouldn't be done. In the same way how the world should never forget what the Nazis did."
0:38

PUTIN (2022?): "To a great extent, the level of prosperity that has been achieved in former colonial powers is grounded in the robbery of Africa, built on the grief and suffering of African people... Robbing and slavery, all of this has to be shown and taught as this is a crucial part of world history; to simply close the chapter on it is impossible and shouldn't be done. In the same way how the world should never forget what the Nazis did."

COMBATE |🇵🇷

368,059 views • 3 years ago

Chinese humanoid robotics company LimX Dynamics has unveiled COSA (Cognitive Operating System of Agents). COSA is described as a unified "brain-body" architecture that allows the robot to think and act simultaneously in the real world. It integrates: - high-level cognition (reasoning, planning, adaptation) - and whole-body motion control (low-latency dynamic locomotion, manipulation) It enables the Oli robot to act on natural language instructions and perform tasks that involve both walking and manipulation while being adaptive to interruptions.
1:18

Chinese humanoid robotics company LimX Dynamics has unveiled COSA (Cognitive Operating System of Agents). COSA is described as a unified "brain-body" architecture that allows the robot to think and act simultaneously in the real world. It integrates: - high-level cognition (reasoning, planning, adaptation) - and whole-body motion control (low-latency dynamic locomotion, manipulation) It enables the Oli robot to act on natural language instructions and perform tasks that involve both walking and manipulation while being adaptive to interruptions.

The Humanoid Hub

71,168 views • 7 months ago

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.
1:29

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

The Humanoid Hub

17,065 views • 6 months ago

🚨🇺🇸🇨🇳 TRUMP: WE’RE LEADING THE WORLD, INCLUDING CHINA, BY A LOT "The press has been great in terms of this industry and they understand the importance of it. This is taking our country to a new level. We're leading the world, we're leading China, but we're leading the world by a lot, and we're leading it because of the people around the table. And all I can do is implement and make it a lot easier for you. And I'll be doing that a hundred percent. So I just want to thank you all." Source: Fox
0:48

🚨🇺🇸🇨🇳 TRUMP: WE’RE LEADING THE WORLD, INCLUDING CHINA, BY A LOT "The press has been great in terms of this industry and they understand the importance of it. This is taking our country to a new level. We're leading the world, we're leading China, but we're leading the world by a lot, and we're leading it because of the people around the table. And all I can do is implement and make it a lot easier for you. And I'll be doing that a hundred percent. So I just want to thank you all." Source: Fox

Mario Nawfal

144,931 views • 11 months ago

XPeng's all-new generation humanoid robot, IRON, not only achieves an extreme level of anthropomorphism but also possesses a "brain" that can think like a human. This will be a significant competitor to $TSLA's Optimus v3 when both robots enter mass production in 2026. ✅ IRON is the humanoid robot with the highest computing power in the world, equipped with the second-generation VLA and 3 Turing AI chips, it builds a high-level combination of capabilities—VLT + VLA + VLM—for its "cerebrum" and "cerebellum." This allows it to achieve advanced intelligence like dialogue, walking, and interaction. ✅ It is the industry's first application of all-solid-state batteries, achieving extreme lightweighting, ultra-high energy density, and ultimate safety. ✅ XPeng also introduced robot active safety protection to maximize the safety of participants in the physical AI world, and expanded the "Fourth Law," ensuring privacy data never leaves the robot.
1:02

XPeng's all-new generation humanoid robot, IRON, not only achieves an extreme level of anthropomorphism but also possesses a "brain" that can think like a human. This will be a significant competitor to $TSLA's Optimus v3 when both robots enter mass production in 2026. ✅ IRON is the humanoid robot with the highest computing power in the world, equipped with the second-generation VLA and 3 Turing AI chips, it builds a high-level combination of capabilities—VLT + VLA + VLM—for its "cerebrum" and "cerebellum." This allows it to achieve advanced intelligence like dialogue, walking, and interaction. ✅ It is the industry's first application of all-solid-state batteries, achieving extreme lightweighting, ultra-high energy density, and ultimate safety. ✅ XPeng also introduced robot active safety protection to maximize the safety of participants in the physical AI world, and expanded the "Fourth Law," ensuring privacy data never leaves the robot.

Ming

63,141 views • 9 months ago

Americans have been taught, wrongly, that low IQ people can simply be trained or civilized up to first-world cultural standards and capabilities. Give 'em a little help, they'll do the job fine. That is simply not true. IQ is innate, genetically coded, and unchangeable on the individual level. A person of low IQ is, and ALWAYS WILL BE, simple INCAPABLE of operating at a high-IQ level. This doesn't make them a BAD person. It just makes them a STUPID person--and a person who will ALWAYS BE stupid. Looking at you, low-IQ, low-trust, high-fraud, largely non-white third-world. FULL VIDEO HERE:
2:03

Americans have been taught, wrongly, that low IQ people can simply be trained or civilized up to first-world cultural standards and capabilities. Give 'em a little help, they'll do the job fine. That is simply not true. IQ is innate, genetically coded, and unchangeable on the individual level. A person of low IQ is, and ALWAYS WILL BE, simple INCAPABLE of operating at a high-IQ level. This doesn't make them a BAD person. It just makes them a STUPID person--and a person who will ALWAYS BE stupid. Looking at you, low-IQ, low-trust, high-fraud, largely non-white third-world. FULL VIDEO HERE:

Andrew Branca Show

27,030 views • 7 months ago

I ran a quick test of Gemini Robotics ER 1.5 controlling my LeKiwi using the MCP I developed before. No extra code required - just specified a new model. Unfortunately, it doesn't work well, even though I had expected it to be the best, as it is the first major VLM model trained specifically for robotics applications. The video below is a relatively good attempt. It does understand what is going on and generates a reasonable action plan, but it is not doing well with the low-level control. I guess it was not trained for this and was used together with the VLA, so it is trained to generate more high-level commands like "grab this object and put it there" rather than doing low-level control specifying every robot movement step by step. Did anybody try to use it in a similar setup? What is your experience?
1:11

I ran a quick test of Gemini Robotics ER 1.5 controlling my LeKiwi using the MCP I developed before. No extra code required - just specified a new model. Unfortunately, it doesn't work well, even though I had expected it to be the best, as it is the first major VLM model trained specifically for robotics applications. The video below is a relatively good attempt. It does understand what is going on and generates a reasonable action plan, but it is not doing well with the low-level control. I guess it was not trained for this and was used together with the VLA, so it is trained to generate more high-level commands like "grab this object and put it there" rather than doing low-level control specifying every robot movement step by step. Did anybody try to use it in a similar setup? What is your experience?

Ilia

15,601 views • 10 months ago

🚨 JUST IN: Fraud Czar JD Vance CONFIRMS his national fraud prosecution strategy is pure genius He's starting with the low level fraud... The low level exposes the whole criminal enterprise. Then, you can prosecute the TOP FRAUDSTERS 🔥 You can't ignore ANY level of fraud. $1M in fraud can lead you to tens or HUNDREDS of millions! "It's impossible to get to the high level fraudsters unless you're sometimes willing to look at the low level fraudsters too." "I mean, how many of us have watched a good mob movie and how do those prosecutions always start?" "You never start with a boss, you start with the low level guys who then give you some sense of the criminal enterprise and then you can go after the high level fraud and the high level crime." "The fact that we return to blind eye to fraud under a million dollars meant that we weren't even looking at the people who were sitting on top of some of these fraud rings!"
0:56

🚨 JUST IN: Fraud Czar JD Vance CONFIRMS his national fraud prosecution strategy is pure genius He's starting with the low level fraud... The low level exposes the whole criminal enterprise. Then, you can prosecute the TOP FRAUDSTERS 🔥 You can't ignore ANY level of fraud. $1M in fraud can lead you to tens or HUNDREDS of millions! "It's impossible to get to the high level fraudsters unless you're sometimes willing to look at the low level fraudsters too." "I mean, how many of us have watched a good mob movie and how do those prosecutions always start?" "You never start with a boss, you start with the low level guys who then give you some sense of the criminal enterprise and then you can go after the high level fraud and the high level crime." "The fact that we return to blind eye to fraud under a million dollars meant that we weren't even looking at the people who were sitting on top of some of these fraud rings!"

Eric Daugherty

16,852 views • 3 months ago

Whatever this is, it’s unbelievable. And it looks like the rot goes all the way to the top, including world leaders. It’s time to demand the TRUTH and let the chips fall where they may. A system built around this level of evil does not deserve to stand.
2:27

Whatever this is, it’s unbelievable. And it looks like the rot goes all the way to the top, including world leaders. It’s time to demand the TRUTH and let the chips fall where they may. A system built around this level of evil does not deserve to stand.

Brad Lea

24,883 views • 6 months ago

To play infield at a high level every rep must be with intent and at game speed!! That’s what the best in the world do! Yes very talented, the practice intent is at a high level as well. That is why they have the ability to make it look easy!! Practice intent translates!!
0:48

To play infield at a high level every rep must be with intent and at game speed!! That’s what the best in the world do! Yes very talented, the practice intent is at a high level as well. That is why they have the ability to make it look easy!! Practice intent translates!!

Brendan Dougherty

47,752 views • 6 months ago

Is Per Seat Pricing Dead in a World of AI: "There are two logical replacements: 1. Value-based pricing. 2. Consumption-based pricing. Talk to customers about consumption based pricing. They do not love it. The problem with value based pricing is customer and vendor have to align on the attribution of value." How do you think about this and the future of pricing in SaaS Marc Benioff Des Traynor Bret Taylor Jesse Zhang
1:50

Is Per Seat Pricing Dead in a World of AI: "There are two logical replacements: 1. Value-based pricing. 2. Consumption-based pricing. Talk to customers about consumption based pricing. They do not love it. The problem with value based pricing is customer and vendor have to align on the attribution of value." How do you think about this and the future of pricing in SaaS Marc Benioff Des Traynor Bret Taylor Jesse Zhang

Harry Stebbings

23,533 views • 10 months ago