Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL), which introduces inherent challenges such as distribution shift and causal confusion. Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning. However, applying online reinforcement learning to VLA models in autonomous driving...

ai_bites's profile picture

AI Bites | YouTube Channel

2,270 subscribers

43,496 Aufrufe • vor 4 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Fin-R1 is out on Hugging Face A Large Language Model for Financial Reasoning through Reinforcement Learning
0:12

Fin-R1 is out on Hugging Face A Large Language Model for Financial Reasoning through Reinforcement Learning

AK

89,100 Aufrufe • vor 1 Jahr

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fang
0:11

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fang

LeRobot

24,727 Aufrufe • vor 1 Monat

See how Gemini 3 Flash and Vision-Language-Action (VLA) models work together to teach a robot how to play the board game First Orchard. This project shows how to use Gemini as the “brain” for tracking game rules and state, while the VLA handles complex physical manipulations like picking and placing fruit 🍎
0:17

See how Gemini 3 Flash and Vision-Language-Action (VLA) models work together to teach a robot how to play the board game First Orchard. This project shows how to use Gemini as the “brain” for tracking game rules and state, while the VLA handles complex physical manipulations like picking and placing fruit 🍎

Google AI Developers

26,640 Aufrufe • vor 4 Monaten

Introduce CoT-VLA – Visual Chain-of-Thought reasoning for Robot Foundation Models! 🤖 By leveraging next-frame prediction as visual chain-of-thought reasoning, CoT-VLA uses future prediction to guide action generation and unlock large-scale video data for training. #CVPR2025
0:11

Introduce CoT-VLA – Visual Chain-of-Thought reasoning for Robot Foundation Models! 🤖 By leveraging next-frame prediction as visual chain-of-thought reasoning, CoT-VLA uses future prediction to guide action generation and unlock large-scale video data for training. #CVPR2025

Qingqing Zhao

48,207 Aufrufe • vor 1 Jahr

Ever had a perfect image in mind that a text-to-image model just couldn't capture? Our new reinforcement learning agent, PASTA, turns image generation into a collaborative conversation, learning your style to bring your vision to life. Learn how it works:
0:19

Ever had a perfect image in mind that a text-to-image model just couldn't capture? Our new reinforcement learning agent, PASTA, turns image generation into a collaborative conversation, learning your style to bring your vision to life. Learn how it works:

Google Research

25,808 Aufrufe • vor 9 Monaten

A viral paper "Language Model Represents Space and Time" recently claims that LLMs learn "world models". As much as I like Max Tegmark's works, I disagree with their definition of world model. World model is a core concept in AI agent and decision making. It is our mental simulation of how the world works given interventions (or lack thereof). A world model captures causality and intuitive physics, telling the agent what is likely and what is impossible. It can and should be used for counterfactual reasoning, i.e. "what ifs": what would happen if I knock over a cup of water? Where would I have been if I had not taken that bus? Yann LeCun Yann LeCun says it well in his position paper ( I quote: "Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation." The first use of the term World Model in deep policy learning is attributed to hardmaru & Jürgen Schmidhuber: In their seminal paper, an agent masters shooting skills in the popular game Doom (demo below) by learning in imagination, using an internal world model as a "physics simulator". To put in a simple Python math formula, world model learns a function F(s[0:t-1], a) -> s[t:], which takes as input the observed past and current action, and outputs plausible future states. Now the definition of World Model in Tegmark's paper seems to be about predicting GPS coordinates and time eras. I see this as just a classification task with no causal learning and simulation going on. You cannot make meaningful interventions against that model, nor can you optimize any decision making in a closed feedback loop. As for the "space & time neurons", I think they are most similar to the "sentiment neuron" that OpenAI published in 2017: Predicting GPS is conceptually no different from predicting sentiment in my opinion. I don't think their experimental results are wrong - just that their conclusion is on shaky grounds. I welcome any debate! Paper link:
0:15

A viral paper "Language Model Represents Space and Time" recently claims that LLMs learn "world models". As much as I like Max Tegmark's works, I disagree with their definition of world model. World model is a core concept in AI agent and decision making. It is our mental simulation of how the world works given interventions (or lack thereof). A world model captures causality and intuitive physics, telling the agent what is likely and what is impossible. It can and should be used for counterfactual reasoning, i.e. "what ifs": what would happen if I knock over a cup of water? Where would I have been if I had not taken that bus? Yann LeCun Yann LeCun says it well in his position paper ( I quote: "Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation." The first use of the term World Model in deep policy learning is attributed to hardmaru & Jürgen Schmidhuber: In their seminal paper, an agent masters shooting skills in the popular game Doom (demo below) by learning in imagination, using an internal world model as a "physics simulator". To put in a simple Python math formula, world model learns a function F(s[0:t-1], a) -> s[t:], which takes as input the observed past and current action, and outputs plausible future states. Now the definition of World Model in Tegmark's paper seems to be about predicting GPS coordinates and time eras. I see this as just a classification task with no causal learning and simulation going on. You cannot make meaningful interventions against that model, nor can you optimize any decision making in a closed feedback loop. As for the "space & time neurons", I think they are most similar to the "sentiment neuron" that OpenAI published in 2017: Predicting GPS is conceptually no different from predicting sentiment in my opinion. I don't think their experimental results are wrong - just that their conclusion is on shaky grounds. I welcome any debate! Paper link:

Jim Fan

593,943 Aufrufe • vor 2 Jahren

Unveiling Babbel's new look! With a focus on tailoring learning experiences to meet your needs, our new brand experience shows that learning a new language is a journey. One that leads to transformation in yourself and the world around you as you learn to speak with others.
0:16

Unveiling Babbel's new look! With a focus on tailoring learning experiences to meet your needs, our new brand experience shows that learning a new language is a journey. One that leads to transformation in yourself and the world around you as you learn to speak with others.

Babbel

40,887 Aufrufe • vor 2 Jahren

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!
0:16

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!

Tongzhou Mu 🤖🦾🦿

16,959 Aufrufe • vor 1 Jahr

This is Rivian's next-generation Autonomy hardware and software platform, which the company says is a "self-improving, end-to-end AI system for our autonomy platform that scales." • 11 cameras (65 total megapixels) • 5 radars • New front-facing LiDAR • Powered by a new in-house designed chip called RAP1 (5nm TSMC) Rivian also detailed its software-first approach to autonomy, powered by the new Rivian Autonomy Platform and an end-to-end data loop used for training. The company introduced its Large Driving Model, a foundational autonomous model trained like a Large Language Model: "Utilizing Group-Relative Policy Optimization, the LDM will distill superior driving strategies from massive datasets into the vehicle."
0:14

This is Rivian's next-generation Autonomy hardware and software platform, which the company says is a "self-improving, end-to-end AI system for our autonomy platform that scales." • 11 cameras (65 total megapixels) • 5 radars • New front-facing LiDAR • Powered by a new in-house designed chip called RAP1 (5nm TSMC) Rivian also detailed its software-first approach to autonomy, powered by the new Rivian Autonomy Platform and an end-to-end data loop used for training. The company introduced its Large Driving Model, a foundational autonomous model trained like a Large Language Model: "Utilizing Group-Relative Policy Optimization, the LDM will distill superior driving strategies from massive datasets into the vehicle."

Sawyer Merritt

282,679 Aufrufe • vor 7 Monaten

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? New paper questions the common assumption that RLVR helps LLMs acquire novel reasoning abilities.
0:14

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? New paper questions the common assumption that RLVR helps LLMs acquire novel reasoning abilities.

DailyPapers

52,092 Aufrufe • vor 1 Jahr

Forces are the language of the physical world. At Eka Robotics, Inc., we're building a Vision-Force-Action Model (VFA).
0:17

Forces are the language of the physical world. At Eka Robotics, Inc., we're building a Vision-Force-Action Model (VFA).

Pulkit Agrawal

42,874 Aufrufe • vor 2 Monaten

What's the right architecture for a VLA? VLM + custom action heads (π₀)? VLM with special discrete action tokens (OpenVLA)? Custom design on top of the VLM (OpenVLA-OFT)? Or... VLM with ZERO modifications? Just predict action as text. The results will surprise you. VLA-0: Outperforms π₀, GR00T-N1, MolmoAct, SmolVLA. With ZERO changes to the VLM. 🧵⬇️
0:17

What's the right architecture for a VLA? VLM + custom action heads (π₀)? VLM with special discrete action tokens (OpenVLA)? Custom design on top of the VLM (OpenVLA-OFT)? Or... VLM with ZERO modifications? Just predict action as text. The results will surprise you. VLA-0: Outperforms π₀, GR00T-N1, MolmoAct, SmolVLA. With ZERO changes to the VLM. 🧵⬇️

Ankit Goyal

106,689 Aufrufe • vor 8 Monaten

FleetLab 1.0 is LIVE. Join us for the first-ever Saronic hackathon — where your code will run real autonomous boats. We are building autonomous surface vessels from the ground up — designing hardware, software, and autonomy systems in-house and deploying them at scale. FleetLab 1.0 puts you in the middle of our work, challenging you to solve the same autonomy problems our team faces in the real world. Winning teams will watch their code deploy live on Saronic’s autonomous test vessels. Participants will choose one of three tracks: 📍 Track 1: UI: Visualizing Autonomous Decisions How should operators understand what an autonomous system is thinking? Design interfaces and apps that make autonomy transparent, intuitive, and mission-ready. 📍 Track 2: Autonomy + Navigation Develop logic, modeling, or machine learning approaches that improve decision-making in dynamic maritime environments. 📍 Track 3: Machine Learning Track Apply machine learning techniques to real autonomy-focused datasets and deployment scenarios. Are you a Texas-based student and ready to take on the challenge? Application link in the first comment!
0:15

FleetLab 1.0 is LIVE. Join us for the first-ever Saronic hackathon — where your code will run real autonomous boats. We are building autonomous surface vessels from the ground up — designing hardware, software, and autonomy systems in-house and deploying them at scale. FleetLab 1.0 puts you in the middle of our work, challenging you to solve the same autonomy problems our team faces in the real world. Winning teams will watch their code deploy live on Saronic’s autonomous test vessels. Participants will choose one of three tracks: 📍 Track 1: UI: Visualizing Autonomous Decisions How should operators understand what an autonomous system is thinking? Design interfaces and apps that make autonomy transparent, intuitive, and mission-ready. 📍 Track 2: Autonomy + Navigation Develop logic, modeling, or machine learning approaches that improve decision-making in dynamic maritime environments. 📍 Track 3: Machine Learning Track Apply machine learning techniques to real autonomy-focused datasets and deployment scenarios. Are you a Texas-based student and ready to take on the challenge? Application link in the first comment!

Saronic

262,713 Aufrufe • vor 4 Monaten

After years using autopilot, I was able to see Tesla Full Self-Driving (Supervised) in action on city streets in Rome. As a passenger I was astonished by the driving smoothness, and how it handled traffic, roundabouts, pedestrians and works on the street, better than most humans.
0:19

After years using autopilot, I was able to see Tesla Full Self-Driving (Supervised) in action on city streets in Rome. As a passenger I was astonished by the driving smoothness, and how it handled traffic, roundabouts, pedestrians and works on the street, better than most humans.

Franco Mossotto

57,435 Aufrufe • vor 4 Monaten

learning how to sing, play guitar & harmonica, as well as learning the body language and stage presence of a huge rock icon all in the span of only 6 months is incredibly impressive and i will get my client this nom even if i have to do it myself
0:18

learning how to sing, play guitar & harmonica, as well as learning the body language and stage presence of a huge rock icon all in the span of only 6 months is incredibly impressive and i will get my client this nom even if i have to do it myself

lee ⭐️

215,325 Aufrufe • vor 5 Monaten

the doe is learning… a new language?
0:17

the doe is learning… a new language?

_EDEN

11,085 Aufrufe • vor 1 Jahr

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)
0:10

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Animesh Garg

52,279 Aufrufe • vor 2 Jahren

Two people have been arrested after a vehicle crashed into a property in south Belfast this morning. The man (21) and the woman (33) were arrested on suspicion of dangerous driving, driving while unfit and a number of other driving-related offences.
0:17

Two people have been arrested after a vehicle crashed into a property in south Belfast this morning. The man (21) and the woman (33) were arrested on suspicion of dangerous driving, driving while unfit and a number of other driving-related offences.

Belfast Telegraph

10,797 Aufrufe • vor 1 Monat

The Hidden Language of Diffusion Models paper page: tackle the challenge of understanding concept representations in text-to-image models by decomposing an input text prompt into a small set of interpretable elements. This is achieved by learning a pseudo-token that is a sparse weighted combination of tokens from the model's vocabulary, with the objective of reconstructing the images generated for the given concept. Applied over the state-of-the-art Stable Diffusion model, this decomposition reveals non-trivial and surprising structures in the representations of concepts. For example, we find that some concepts such as "a president" or "a composer" are dominated by specific instances (e.g., "Obama", "Biden") and their interpolations. Other concepts, such as "happiness" combine associated terms that can be concrete ("family", "laughter") or abstract ("friendship", "emotion"). In addition to peering into the inner workings of Stable Diffusion, our method also enables applications such as single-image decomposition to tokens, bias detection and mitigation, and semantic image manipulation
0:19

The Hidden Language of Diffusion Models paper page: tackle the challenge of understanding concept representations in text-to-image models by decomposing an input text prompt into a small set of interpretable elements. This is achieved by learning a pseudo-token that is a sparse weighted combination of tokens from the model's vocabulary, with the objective of reconstructing the images generated for the given concept. Applied over the state-of-the-art Stable Diffusion model, this decomposition reveals non-trivial and surprising structures in the representations of concepts. For example, we find that some concepts such as "a president" or "a composer" are dominated by specific instances (e.g., "Obama", "Biden") and their interpolations. Other concepts, such as "happiness" combine associated terms that can be concrete ("family", "laughter") or abstract ("friendship", "emotion"). In addition to peering into the inner workings of Stable Diffusion, our method also enables applications such as single-image decomposition to tokens, bias detection and mitigation, and semantic image manipulation

AK

41,746 Aufrufe • vor 3 Jahren

Did you know that babies cry with an accent? This week I’ll be focusing on language acquisition during infancy. Yesterday we talked about how newborns enter the world already recognizing the distinct tones of their mothers’ voices, as well as the rhythms and patterns of what will become their native language. And that recognition is not only receptive (the language we hear and understand), but expressive (the language we speak). Which leads us to crying with accent. A study that compared the cries of French and German infants identified distinct patterns their cries - measured as early as the day after their births - that mirror the unique melodies (or prosody) of their native languages. The French infants ended their cries with a lilt that mirrored French, while German infants began their cries with intensity then dropped off at the end, mirroring the vocalizations of their German parents. It’s still more evidence that some basic forms of language learning are influenced by children’s experience (learning) in utero. When it comes to our earliest vocalizations, newborns rely heavily on cries - which often tend to be a bit screechier than they will become in time - alongside gurgles, grunts and squeaks (like those shown in the video here.) But in just a matter of weeks these basic sounds make way for a much broader array of vocalizations. We’ll look at a few of those tomorrow. This new arrival was shared to TT by
0:14

Did you know that babies cry with an accent? This week I’ll be focusing on language acquisition during infancy. Yesterday we talked about how newborns enter the world already recognizing the distinct tones of their mothers’ voices, as well as the rhythms and patterns of what will become their native language. And that recognition is not only receptive (the language we hear and understand), but expressive (the language we speak). Which leads us to crying with accent. A study that compared the cries of French and German infants identified distinct patterns their cries - measured as early as the day after their births - that mirror the unique melodies (or prosody) of their native languages. The French infants ended their cries with a lilt that mirrored French, while German infants began their cries with intensity then dropped off at the end, mirroring the vocalizations of their German parents. It’s still more evidence that some basic forms of language learning are influenced by children’s experience (learning) in utero. When it comes to our earliest vocalizations, newborns rely heavily on cries - which often tend to be a bit screechier than they will become in time - alongside gurgles, grunts and squeaks (like those shown in the video here.) But in just a matter of weeks these basic sounds make way for a much broader array of vocalizations. We’ll look at a few of those tomorrow. This new arrival was shared to TT by

Dan Wuori

115,574 Aufrufe • vor 1 Jahr