Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL), which introduces inherent challenges such as distribution shift and causal confusion. Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning. However, applying online reinforcement learning to VLA models in autonomous driving...

ai_bites's profile picture

AI Bites | YouTube Channel

2,491 subscribers

43,556 Aufrufe • vor 8 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Fin-R1 is out on Hugging Face A Large Language Model for Financial Reasoning through Reinforcement Learning
0:12

Fin-R1 is out on Hugging Face A Large Language Model for Financial Reasoning through Reinforcement Learning

AK

89,153 Aufrufe • vor 1 Jahr

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fang
0:11

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fang

LeRobot

25,272 Aufrufe • vor 5 Monaten

See how Gemini 3 Flash and Vision-Language-Action (VLA) models work together to teach a robot how to play the board game First Orchard. This project shows how to use Gemini as the “brain” for tracking game rules and state, while the VLA handles complex physical manipulations like picking and placing fruit 🍎
0:17

See how Gemini 3 Flash and Vision-Language-Action (VLA) models work together to teach a robot how to play the board game First Orchard. This project shows how to use Gemini as the “brain” for tracking game rules and state, while the VLA handles complex physical manipulations like picking and placing fruit 🍎

Google AI Developers

26,640 Aufrufe • vor 8 Monaten

Introduce CoT-VLA – Visual Chain-of-Thought reasoning for Robot Foundation Models! 🤖 By leveraging next-frame prediction as visual chain-of-thought reasoning, CoT-VLA uses future prediction to guide action generation and unlock large-scale video data for training. #CVPR2025
0:11

Introduce CoT-VLA – Visual Chain-of-Thought reasoning for Robot Foundation Models! 🤖 By leveraging next-frame prediction as visual chain-of-thought reasoning, CoT-VLA uses future prediction to guide action generation and unlock large-scale video data for training. #CVPR2025

Qingqing Zhao

49,621 Aufrufe • vor 1 Jahr

Methane is a potent greenhouse gas driving approximately 25% of human-induced warming. MAPL-EMIT is a deep-learning model that automates global methane tracking from space to enable scalable climate action. Read the full post and check out the dataset:
0:10

Methane is a potent greenhouse gas driving approximately 25% of human-induced warming. MAPL-EMIT is a deep-learning model that automates global methane tracking from space to enable scalable climate action. Read the full post and check out the dataset:

Google Research

13,897 Aufrufe • vor 2 Monaten

Ever had a perfect image in mind that a text-to-image model just couldn't capture? Our new reinforcement learning agent, PASTA, turns image generation into a collaborative conversation, learning your style to bring your vision to life. Learn how it works:
0:19

Ever had a perfect image in mind that a text-to-image model just couldn't capture? Our new reinforcement learning agent, PASTA, turns image generation into a collaborative conversation, learning your style to bring your vision to life. Learn how it works:

Google Research

25,860 Aufrufe • vor 1 Jahr

A viral paper "Language Model Represents Space and Time" recently claims that LLMs learn "world models". As much as I like Max Tegmark's works, I disagree with their definition of world model. World model is a core concept in AI agent and decision making. It is our mental simulation of how the world works given interventions (or lack thereof). A world model captures causality and intuitive physics, telling the agent what is likely and what is impossible. It can and should be used for counterfactual reasoning, i.e. "what ifs": what would happen if I knock over a cup of water? Where would I have been if I had not taken that bus? Yann LeCun Yann LeCun says it well in his position paper ( I quote: "Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation." The first use of the term World Model in deep policy learning is attributed to hardmaru & Jürgen Schmidhuber: In their seminal paper, an agent masters shooting skills in the popular game Doom (demo below) by learning in imagination, using an internal world model as a "physics simulator". To put in a simple Python math formula, world model learns a function F(s[0:t-1], a) -> s[t:], which takes as input the observed past and current action, and outputs plausible future states. Now the definition of World Model in Tegmark's paper seems to be about predicting GPS coordinates and time eras. I see this as just a classification task with no causal learning and simulation going on. You cannot make meaningful interventions against that model, nor can you optimize any decision making in a closed feedback loop. As for the "space & time neurons", I think they are most similar to the "sentiment neuron" that OpenAI published in 2017: Predicting GPS is conceptually no different from predicting sentiment in my opinion. I don't think their experimental results are wrong - just that their conclusion is on shaky grounds. I welcome any debate! Paper link:
0:15

A viral paper "Language Model Represents Space and Time" recently claims that LLMs learn "world models". As much as I like Max Tegmark's works, I disagree with their definition of world model. World model is a core concept in AI agent and decision making. It is our mental simulation of how the world works given interventions (or lack thereof). A world model captures causality and intuitive physics, telling the agent what is likely and what is impossible. It can and should be used for counterfactual reasoning, i.e. "what ifs": what would happen if I knock over a cup of water? Where would I have been if I had not taken that bus? Yann LeCun Yann LeCun says it well in his position paper ( I quote: "Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation." The first use of the term World Model in deep policy learning is attributed to hardmaru & Jürgen Schmidhuber: In their seminal paper, an agent masters shooting skills in the popular game Doom (demo below) by learning in imagination, using an internal world model as a "physics simulator". To put in a simple Python math formula, world model learns a function F(s[0:t-1], a) -> s[t:], which takes as input the observed past and current action, and outputs plausible future states. Now the definition of World Model in Tegmark's paper seems to be about predicting GPS coordinates and time eras. I see this as just a classification task with no causal learning and simulation going on. You cannot make meaningful interventions against that model, nor can you optimize any decision making in a closed feedback loop. As for the "space & time neurons", I think they are most similar to the "sentiment neuron" that OpenAI published in 2017: Predicting GPS is conceptually no different from predicting sentiment in my opinion. I don't think their experimental results are wrong - just that their conclusion is on shaky grounds. I welcome any debate! Paper link:

Jim Fan

594,014 Aufrufe • vor 3 Jahren

Unveiling Babbel's new look! With a focus on tailoring learning experiences to meet your needs, our new brand experience shows that learning a new language is a journey. One that leads to transformation in yourself and the world around you as you learn to speak with others.
0:16

Unveiling Babbel's new look! With a focus on tailoring learning experiences to meet your needs, our new brand experience shows that learning a new language is a journey. One that leads to transformation in yourself and the world around you as you learn to speak with others.

Babbel

41,570 Aufrufe • vor 2 Jahren

Working on a new topic and hardware for a while: Feet Language Action models. VLAs teach robots to act through vision and language FLAs teach legged robots to act through contact Given a task, terrain, body state, and instruction - decide where, when, and how to make contact
0:18

Working on a new topic and hardware for a while: Feet Language Action models. VLAs teach robots to act through vision and language FLAs teach legged robots to act through contact Given a task, terrain, body state, and instruction - decide where, when, and how to make contact

Irvin (in Japan 🇯🇵)

14,026 Aufrufe • vor 4 Monaten

A new Science Robotics study highlights a reinforcement learning–based framework that integrates perceptual uncertainty into policy learning to equip humanoid robots with reactive soccer skills and dynamic behaviors.
0:19

A new Science Robotics study highlights a reinforcement learning–based framework that integrates perceptual uncertainty into policy learning to equip humanoid robots with reactive soccer skills and dynamic behaviors.

Science Magazine

56,716 Aufrufe • vor 2 Monaten

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!
0:16

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!

Tongzhou Mu 🤖🦾🦿

17,077 Aufrufe • vor 1 Jahr

This is Rivian's next-generation Autonomy hardware and software platform, which the company says is a "self-improving, end-to-end AI system for our autonomy platform that scales." • 11 cameras (65 total megapixels) • 5 radars • New front-facing LiDAR • Powered by a new in-house designed chip called RAP1 (5nm TSMC) Rivian also detailed its software-first approach to autonomy, powered by the new Rivian Autonomy Platform and an end-to-end data loop used for training. The company introduced its Large Driving Model, a foundational autonomous model trained like a Large Language Model: "Utilizing Group-Relative Policy Optimization, the LDM will distill superior driving strategies from massive datasets into the vehicle."
0:14

This is Rivian's next-generation Autonomy hardware and software platform, which the company says is a "self-improving, end-to-end AI system for our autonomy platform that scales." • 11 cameras (65 total megapixels) • 5 radars • New front-facing LiDAR • Powered by a new in-house designed chip called RAP1 (5nm TSMC) Rivian also detailed its software-first approach to autonomy, powered by the new Rivian Autonomy Platform and an end-to-end data loop used for training. The company introduced its Large Driving Model, a foundational autonomous model trained like a Large Language Model: "Utilizing Group-Relative Policy Optimization, the LDM will distill superior driving strategies from massive datasets into the vehicle."

Sawyer Merritt

283,555 Aufrufe • vor 10 Monaten

UNUSUAL Dogs Playfully Outsmart a Self-Driving Vehicle😂 In a humorous scene, a group of dogs appeared to toy with a self-driving vehicle. Each time they approached, the vehicle stopped and carefully avoided any collision. The dogs, however, seemed to interpret its stopping and flashing lights as an invitation to play, circling around it and teasing it as if it were a new toy. It seems autonomous driving systems have learned to be extremely cautious around living creatures, but the dogs have not quite gotten the message yet. Instead, they see that caution as an opportunity for fun. 😄
0:19

UNUSUAL Dogs Playfully Outsmart a Self-Driving Vehicle😂 In a humorous scene, a group of dogs appeared to toy with a self-driving vehicle. Each time they approached, the vehicle stopped and carefully avoided any collision. The dogs, however, seemed to interpret its stopping and flashing lights as an invitation to play, circling around it and teasing it as if it were a new toy. It seems autonomous driving systems have learned to be extremely cautious around living creatures, but the dogs have not quite gotten the message yet. Instead, they see that caution as an opportunity for fun. 😄

China Now

11,821 Aufrufe • vor 15 Tagen

Forces are the language of the physical world. At Eka Robotics, Inc., we're building a Vision-Force-Action Model (VFA).
0:17

Forces are the language of the physical world. At Eka Robotics, Inc., we're building a Vision-Force-Action Model (VFA).

Pulkit Agrawal

43,649 Aufrufe • vor 6 Monaten

What comes after vision-language-action models in robotics? 🤖 Join NVIDIA researchers live to explore World Action Models (WAMs), Vision-Language-Action models (VLAs), and hybrid robot foundation models, and see how NVIDIA Cosmos 3 natively represents action alongside video. 🕑 Thursday, Aug. 13 • 9:00–10:00 a.m. PT 📅Add calendar:
0:15

What comes after vision-language-action models in robotics? 🤖 Join NVIDIA researchers live to explore World Action Models (WAMs), Vision-Language-Action models (VLAs), and hybrid robot foundation models, and see how NVIDIA Cosmos 3 natively represents action alongside video. 🕑 Thursday, Aug. 13 • 9:00–10:00 a.m. PT 📅Add calendar:

NVIDIA Robotics

12,688 Aufrufe • vor 2 Monaten

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? New paper questions the common assumption that RLVR helps LLMs acquire novel reasoning abilities.
0:14

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? New paper questions the common assumption that RLVR helps LLMs acquire novel reasoning abilities.

DailyPapers

52,100 Aufrufe • vor 1 Jahr

What's the right architecture for a VLA? VLM + custom action heads (π₀)? VLM with special discrete action tokens (OpenVLA)? Custom design on top of the VLM (OpenVLA-OFT)? Or... VLM with ZERO modifications? Just predict action as text. The results will surprise you. VLA-0: Outperforms π₀, GR00T-N1, MolmoAct, SmolVLA. With ZERO changes to the VLM. 🧵⬇️
0:17

What's the right architecture for a VLA? VLM + custom action heads (π₀)? VLM with special discrete action tokens (OpenVLA)? Custom design on top of the VLM (OpenVLA-OFT)? Or... VLM with ZERO modifications? Just predict action as text. The results will surprise you. VLA-0: Outperforms π₀, GR00T-N1, MolmoAct, SmolVLA. With ZERO changes to the VLM. 🧵⬇️

Ankit Goyal

107,426 Aufrufe • vor 1 Jahr

𝗗𝗼𝗻'𝘁 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗲 𝗿𝗼𝗯𝗼𝘁 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹𝘀. 𝗦𝘁𝗲𝗲𝗿 𝘁𝗵𝗲𝗺 𝘄𝗶𝘁𝗵 𝗵𝘂𝗺𝗮𝗻 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀 𝗶𝗻𝘀𝘁𝗲𝗮𝗱, 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗲 𝗯𝗮𝘀𝗲 𝗽𝗼𝗹𝗶𝗰𝘆 Modern VLAs and world-action models can perform impressive manipulation skills, but adapting them reliably to new robots and tasks remains challenging. A natural solution is DAgger-style online imitation learning: deploy the robot, collect human corrections, and update the policy. Yet foundation models are fragile in the low-data regime, fine-tuning on a handful of interventions can improve one behavior while degrading others. Online post-training or reinforcement learning can require costly data collection and exploration, making real-world learning expensive and potentially unsafe. In our new paper, 𝗙𝗹𝗼𝘄𝗗𝗔𝗴𝗴𝗲𝗿, we take a different approach: 𝗜𝗻𝘀𝘁𝗲𝗮𝗱 𝗼𝗳 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗲 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹, 𝘄𝗲 𝗹𝗲𝗮𝗿𝗻 𝗵𝗼𝘄 𝘁𝗼 𝘀𝘁𝗲𝗲𝗿 𝗶𝘁 𝗳𝗿𝗼𝗺 𝗵𝘂𝗺𝗮𝗻 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀. The key idea is 𝗮𝗰𝘁𝗶𝗼𝗻 𝗶𝗻𝘃𝗲𝗿𝘀𝗶𝗼𝗻: we map human corrective actions back into the latent noise space of the frozen generative policy. These latent targets train a lightweight controller that adapts the robot while preserving the original model's capabilities. Across simulation and real robots, FlowDAgger: 📈 Learns from only 5–20 human intervention episodes 🏆 Outperforms supervised fine-tuning and latent-space reinforcement learning 🤖 Works across VLAs, diffusion policies, and world-action models ✔️ Provides reliable improvements without modifying the pretrained policy We believe this offers a practical path toward making robot foundation models improve during deployment, learning from the way humans naturally teach: through corrections. 📄 Paper: 🌐 Project: 💻 Code: This project was led by my amazing colleague Michael Murray with help from Daphne Chen, Simran Bagaria, Dean Fortier, Tess Hellebrekers, Harshavardhan Reddy Gajarla, Galen Mullins and Andrey Kolobov at Microsoft Research and Maya Cakmak at University of Washington
0:11

𝗗𝗼𝗻'𝘁 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗲 𝗿𝗼𝗯𝗼𝘁 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹𝘀. 𝗦𝘁𝗲𝗲𝗿 𝘁𝗵𝗲𝗺 𝘄𝗶𝘁𝗵 𝗵𝘂𝗺𝗮𝗻 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀 𝗶𝗻𝘀𝘁𝗲𝗮𝗱, 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗲 𝗯𝗮𝘀𝗲 𝗽𝗼𝗹𝗶𝗰𝘆 Modern VLAs and world-action models can perform impressive manipulation skills, but adapting them reliably to new robots and tasks remains challenging. A natural solution is DAgger-style online imitation learning: deploy the robot, collect human corrections, and update the policy. Yet foundation models are fragile in the low-data regime, fine-tuning on a handful of interventions can improve one behavior while degrading others. Online post-training or reinforcement learning can require costly data collection and exploration, making real-world learning expensive and potentially unsafe. In our new paper, 𝗙𝗹𝗼𝘄𝗗𝗔𝗴𝗴𝗲𝗿, we take a different approach: 𝗜𝗻𝘀𝘁𝗲𝗮𝗱 𝗼𝗳 𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘁𝗵𝗲 𝗳𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹, 𝘄𝗲 𝗹𝗲𝗮𝗿𝗻 𝗵𝗼𝘄 𝘁𝗼 𝘀𝘁𝗲𝗲𝗿 𝗶𝘁 𝗳𝗿𝗼𝗺 𝗵𝘂𝗺𝗮𝗻 𝗰𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝗼𝗻𝘀. The key idea is 𝗮𝗰𝘁𝗶𝗼𝗻 𝗶𝗻𝘃𝗲𝗿𝘀𝗶𝗼𝗻: we map human corrective actions back into the latent noise space of the frozen generative policy. These latent targets train a lightweight controller that adapts the robot while preserving the original model's capabilities. Across simulation and real robots, FlowDAgger: 📈 Learns from only 5–20 human intervention episodes 🏆 Outperforms supervised fine-tuning and latent-space reinforcement learning 🤖 Works across VLAs, diffusion policies, and world-action models ✔️ Provides reliable improvements without modifying the pretrained policy We believe this offers a practical path toward making robot foundation models improve during deployment, learning from the way humans naturally teach: through corrections. 📄 Paper: 🌐 Project: 💻 Code: This project was led by my amazing colleague Michael Murray with help from Daphne Chen, Simran Bagaria, Dean Fortier, Tess Hellebrekers, Harshavardhan Reddy Gajarla, Galen Mullins and Andrey Kolobov at Microsoft Research and Maya Cakmak at University of Washington

Oier Mees

13,529 Aufrufe • vor 3 Monaten

After years using autopilot, I was able to see Tesla Full Self-Driving (Supervised) in action on city streets in Rome. As a passenger I was astonished by the driving smoothness, and how it handled traffic, roundabouts, pedestrians and works on the street, better than most humans.
0:19

After years using autopilot, I was able to see Tesla Full Self-Driving (Supervised) in action on city streets in Rome. As a passenger I was astonished by the driving smoothness, and how it handled traffic, roundabouts, pedestrians and works on the street, better than most humans.

Franco Mossotto

57,435 Aufrufe • vor 8 Monaten

Dubai basically just built a driving school for autonomous cars. The new Mobility Labs lets vehicles go from closed testing zones to city-style roads and then onto real public roads, all in one place. Tesla has already tested there. Dubai is aiming for 25% of all journeys to be autonomous by 2030. It’s where autonomous driving gets ready for the real world. Tesla / Writer: Annette, Designer: Janné
0:10

Dubai basically just built a driving school for autonomous cars. The new Mobility Labs lets vehicles go from closed testing zones to city-style roads and then onto real public roads, all in one place. Tesla has already tested there. Dubai is aiming for 25% of all journeys to be autonomous by 2030. It’s where autonomous driving gets ready for the real world. Tesla / Writer: Annette, Designer: Janné

Mario Nawfal

60,086 Aufrufe • vor 1 Monat