Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely... on Imitation Learning (IL), which introduces inherent challenges such as distribution shift and causal confusion. Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning. However, applying online reinforcement learning to VLA models in autonomous driving is hindered by inefficient exploration in continuous action spaces. MindDrive, a VLA framework comprising a large language model (LLM) with two distinct sets of LoRA parameters. The one LLM serves as a Decision Expert for scenario reasoning and driving decision-making, while the other acts as an Action Expert that dynamically maps linguistic decisions into feasible trajectories. Paper Title: MindDrive: A Vision-Language-Action Model for Autonomous Driving via Project: Link:show more

Fin-R1 is out on Hugging Face A Large Language... Model for Financial Reasoning through Reinforcement Learningshow more

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning... Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fangshow more

See how Gemini 3 Flash and Vision-Language-Action (VLA) models... work together to teach a robot how to play the board game First Orchard. This project shows how to use Gemini as the “brain” for tracking game rules and state, while the VLA handles complex physical manipulations like picking and placing fruit 🍎show more

Introduce CoT-VLA – Visual Chain-of-Thought reasoning for Robot Foundation... Models! 🤖 By leveraging next-frame prediction as visual chain-of-thought reasoning, CoT-VLA uses future prediction to guide action generation and unlock large-scale video data for training. #CVPR2025show more

Ever had a perfect image in mind that a... text-to-image model just couldn't capture? Our new reinforcement learning agent, PASTA, turns image generation into a collaborative conversation, learning your style to bring your vision to life. Learn how it works:show more

A viral paper "Language Model Represents Space and Time"... recently claims that LLMs learn "world models". As much as I like Max Tegmark's works, I disagree with their definition of world model. World model is a core concept in AI agent and decision making. It is our mental simulation of how the world works given interventions (or lack thereof). A world model captures causality and intuitive physics, telling the agent what is likely and what is impossible. It can and should be used for counterfactual reasoning, i.e. "what ifs": what would happen if I knock over a cup of water? Where would I have been if I had not taken that bus? Yann LeCun Yann LeCun says it well in his position paper ( I quote: "Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation." The first use of the term World Model in deep policy learning is attributed to hardmaru & Jürgen Schmidhuber: In their seminal paper, an agent masters shooting skills in the popular game Doom (demo below) by learning in imagination, using an internal world model as a "physics simulator". To put in a simple Python math formula, world model learns a function F(s[0:t-1], a) -> s[t:], which takes as input the observed past and current action, and outputs plausible future states. Now the definition of World Model in Tegmark's paper seems to be about predicting GPS coordinates and time eras. I see this as just a classification task with no causal learning and simulation going on. You cannot make meaningful interventions against that model, nor can you optimize any decision making in a closed feedback loop. As for the "space & time neurons", I think they are most similar to the "sentiment neuron" that OpenAI published in 2017: Predicting GPS is conceptually no different from predicting sentiment in my opinion. I don't think their experimental results are wrong - just that their conclusion is on shaky grounds. I welcome any debate! Paper link:show more

Unveiling Babbel's new look! With a focus on tailoring... learning experiences to meet your needs, our new brand experience shows that learning a new language is a journey. One that leads to transformation in yourself and the world around you as you learn to speak with others.show more

🤔 How to fine-tune an Imitation Learning policy (e.g.,... Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!show more

This is Rivian's next-generation Autonomy hardware and software platform,... which the company says is a "self-improving, end-to-end AI system for our autonomy platform that scales." • 11 cameras (65 total megapixels) • 5 radars • New front-facing LiDAR • Powered by a new in-house designed chip called RAP1 (5nm TSMC) Rivian also detailed its software-first approach to autonomy, powered by the new Rivian Autonomy Platform and an end-to-end data loop used for training. The company introduced its Large Driving Model, a foundational autonomous model trained like a Large Language Model: "Utilizing Group-Relative Policy Optimization, the LDM will distill superior driving strategies from massive datasets into the vehicle."show more

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs... Beyond the Base Model? New paper questions the common assumption that RLVR helps LLMs acquire novel reasoning abilities.show more

Forces are the language of the physical world. At... Eka Robotics, Inc., we're building a Vision-Force-Action Model (VFA).show more

What's the right architecture for a VLA? VLM +... custom action heads (π₀)? VLM with special discrete action tokens (OpenVLA)? Custom design on top of the VLM (OpenVLA-OFT)? Or... VLM with ZERO modifications? Just predict action as text. The results will surprise you. VLA-0: Outperforms π₀, GR00T-N1, MolmoAct, SmolVLA. With ZERO changes to the VLM. 🧵⬇️show more

FleetLab 1.0 is LIVE. Join us for the first-ever... Saronic hackathon — where your code will run real autonomous boats. We are building autonomous surface vessels from the ground up — designing hardware, software, and autonomy systems in-house and deploying them at scale. FleetLab 1.0 puts you in the middle of our work, challenging you to solve the same autonomy problems our team faces in the real world. Winning teams will watch their code deploy live on Saronic’s autonomous test vessels. Participants will choose one of three tracks: 📍 Track 1: UI: Visualizing Autonomous Decisions How should operators understand what an autonomous system is thinking? Design interfaces and apps that make autonomy transparent, intuitive, and mission-ready. 📍 Track 2: Autonomy + Navigation Develop logic, modeling, or machine learning approaches that improve decision-making in dynamic maritime environments. 📍 Track 3: Machine Learning Track Apply machine learning techniques to real autonomy-focused datasets and deployment scenarios. Are you a Texas-based student and ready to take on the challenge? Application link in the first comment!show more

After years using autopilot, I was able to see... Tesla Full Self-Driving (Supervised) in action on city streets in Rome. As a passenger I was astonished by the driving smoothness, and how it handled traffic, roundabouts, pedestrians and works on the street, better than most humans.show more

learning how to sing, play guitar & harmonica, as... well as learning the body language and stage presence of a huge rock icon all in the span of only 6 months is incredibly impressive and i will get my client this nom even if i have to do it myselfshow more

the doe is learning… a new language?

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with... supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)show more

The Hidden Language of Diffusion Models paper page: tackle... the challenge of understanding concept representations in text-to-image models by decomposing an input text prompt into a small set of interpretable elements. This is achieved by learning a pseudo-token that is a sparse weighted combination of tokens from the model's vocabulary, with the objective of reconstructing the images generated for the given concept. Applied over the state-of-the-art Stable Diffusion model, this decomposition reveals non-trivial and surprising structures in the representations of concepts. For example, we find that some concepts such as "a president" or "a composer" are dominated by specific instances (e.g., "Obama", "Biden") and their interpolations. Other concepts, such as "happiness" combine associated terms that can be concrete ("family", "laughter") or abstract ("friendship", "emotion"). In addition to peering into the inner workings of Stable Diffusion, our method also enables applications such as single-image decomposition to tokens, bias detection and mitigation, and semantic image manipulationshow more

Two people have been arrested after a vehicle crashed... into a property in south Belfast this morning. The man (21) and the woman (33) were arrested on suspicion of dangerous driving, driving while unfit and a number of other driving-related offences.show more

Introducing Alpamayo 1.5. Based on community feedback, we’ve updated... our 10B-parameter chain-of-thought reasoning VLA model to be a more interactive and steerable engine for autonomous vehicle development. Built on the Cosmos-Reason2 VLM backbone, this release adds support for navigation guidance and flexible camera configurations while providing new post-training scripts for model adaptation. 🤗 Learn more:show more