ๆญฃๅœจๅŠ ่ฝฝ่ง†้ข‘...

่ง†้ข‘ๅŠ ่ฝฝๅคฑ่ดฅ

่ฟ”ๅ›ž้ฆ–้กต

๐Ÿค” How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, Iโ€™ve been struggling with this problem for a while. Hereโ€™s why itโ€™s tough: 1๏ธโƒฃ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by...

tongzhou_mu's profile picture

Tongzhou Mu ๐Ÿค–๐Ÿฆพ๐Ÿฆฟ

3,335 subscribers

17,018 ๆฌก่ง‚็œ‹ โ€ข 1 ๅนดๅ‰ โ€ขvia X (Twitter)

็ง‘ๅญฆๆŠ€ๆœฏๆ•™่‚ฒ
LIVE

Anya Rossiโ€ข Live Now

Private livecam show

0 ๆก่ฏ„่ฎบ

ๆš‚ๆ— ่ฏ„่ฎบ

ๅŽŸๅง‹ๅธ–ๅญ็š„่ฏ„่ฎบๅฐ†ๆ˜พ็คบๅœจ่ฟ™้‡Œ

็›ธๅ…ณ่ง†้ข‘

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -
0:15

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -

Abhishek Gupta

11,430 ๆฌก่ง‚็œ‹ โ€ข 1 ๅนดๅ‰

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper๏ผ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! ๐Ÿงต๐Ÿ‘‡ ๐Ÿ”— Check out our paper:
0:18

How can we more effectively leverage robot data from different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper๏ผ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! ๐Ÿงต๐Ÿ‘‡ ๐Ÿ”— Check out our paper:

Chenfeng_X

28,321 ๆฌก่ง‚็œ‹ โ€ข 2 ๅนดๅ‰

Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL), which introduces inherent challenges such as distribution shift and causal confusion. Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning. However, applying online reinforcement learning to VLA models in autonomous driving is hindered by inefficient exploration in continuous action spaces. MindDrive, a VLA framework comprising a large language model (LLM) with two distinct sets of LoRA parameters. The one LLM serves as a Decision Expert for scenario reasoning and driving decision-making, while the other acts as an Action Expert that dynamically maps linguistic decisions into feasible trajectories. Paper Title: MindDrive: A Vision-Language-Action Model for Autonomous Driving via Project: Link:
0:12

Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL), which introduces inherent challenges such as distribution shift and causal confusion. Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning. However, applying online reinforcement learning to VLA models in autonomous driving is hindered by inefficient exploration in continuous action spaces. MindDrive, a VLA framework comprising a large language model (LLM) with two distinct sets of LoRA parameters. The one LLM serves as a Decision Expert for scenario reasoning and driving decision-making, while the other acts as an Action Expert that dynamically maps linguistic decisions into feasible trajectories. Paper Title: MindDrive: A Vision-Language-Action Model for Autonomous Driving via Project: Link:

AI Bites | YouTube Channel

43,496 ๆฌก่ง‚็œ‹ โ€ข 7 ไธชๆœˆๅ‰

๐——๐—ผ๐—ป'๐˜ ๐—ณ๐—ถ๐—ป๐—ฒ-๐˜๐˜‚๐—ป๐—ฒ ๐—ฟ๐—ผ๐—ฏ๐—ผ๐˜ ๐—ณ๐—ผ๐˜‚๐—ป๐—ฑ๐—ฎ๐˜๐—ถ๐—ผ๐—ป ๐—บ๐—ผ๐—ฑ๐—ฒ๐—น๐˜€. ๐—ฆ๐˜๐—ฒ๐—ฒ๐—ฟ ๐˜๐—ต๐—ฒ๐—บ ๐˜„๐—ถ๐˜๐—ต ๐—ต๐˜‚๐—บ๐—ฎ๐—ป ๐—ฐ๐—ผ๐—ฟ๐—ฟ๐—ฒ๐—ฐ๐˜๐—ถ๐—ผ๐—ป๐˜€ ๐—ถ๐—ป๐˜€๐˜๐—ฒ๐—ฎ๐—ฑ, ๐˜„๐—ถ๐˜๐—ต๐—ผ๐˜‚๐˜ ๐—ฐ๐—ต๐—ฎ๐—ป๐—ด๐—ถ๐—ป๐—ด ๐˜๐—ต๐—ฒ ๐—ฏ๐—ฎ๐˜€๐—ฒ ๐—ฝ๐—ผ๐—น๐—ถ๐—ฐ๐˜† Modern VLAs and world-action models can perform impressive manipulation skills, but adapting them reliably to new robots and tasks remains challenging. A natural solution is DAgger-style online imitation learning: deploy the robot, collect human corrections, and update the policy. Yet foundation models are fragile in the low-data regime, fine-tuning on a handful of interventions can improve one behavior while degrading others. Online post-training or reinforcement learning can require costly data collection and exploration, making real-world learning expensive and potentially unsafe. In our new paper, ๐—™๐—น๐—ผ๐˜„๐——๐—”๐—ด๐—ด๐—ฒ๐—ฟ, we take a different approach: ๐—œ๐—ป๐˜€๐˜๐—ฒ๐—ฎ๐—ฑ ๐—ผ๐—ณ ๐—ฐ๐—ต๐—ฎ๐—ป๐—ด๐—ถ๐—ป๐—ด ๐˜๐—ต๐—ฒ ๐—ณ๐—ผ๐˜‚๐—ป๐—ฑ๐—ฎ๐˜๐—ถ๐—ผ๐—ป ๐—บ๐—ผ๐—ฑ๐—ฒ๐—น, ๐˜„๐—ฒ ๐—น๐—ฒ๐—ฎ๐—ฟ๐—ป ๐—ต๐—ผ๐˜„ ๐˜๐—ผ ๐˜€๐˜๐—ฒ๐—ฒ๐—ฟ ๐—ถ๐˜ ๐—ณ๐—ฟ๐—ผ๐—บ ๐—ต๐˜‚๐—บ๐—ฎ๐—ป ๐—ฐ๐—ผ๐—ฟ๐—ฟ๐—ฒ๐—ฐ๐˜๐—ถ๐—ผ๐—ป๐˜€. The key idea is ๐—ฎ๐—ฐ๐˜๐—ถ๐—ผ๐—ป ๐—ถ๐—ป๐˜ƒ๐—ฒ๐—ฟ๐˜€๐—ถ๐—ผ๐—ป: we map human corrective actions back into the latent noise space of the frozen generative policy. These latent targets train a lightweight controller that adapts the robot while preserving the original model's capabilities. Across simulation and real robots, FlowDAgger: ๐Ÿ“ˆ Learns from only 5โ€“20 human intervention episodes ๐Ÿ† Outperforms supervised fine-tuning and latent-space reinforcement learning ๐Ÿค– Works across VLAs, diffusion policies, and world-action models โœ”๏ธ Provides reliable improvements without modifying the pretrained policy We believe this offers a practical path toward making robot foundation models improve during deployment, learning from the way humans naturally teach: through corrections. ๐Ÿ“„ Paper: ๐ŸŒ Project: ๐Ÿ’ป Code: This project was led by my amazing colleague Michael Murray with help from Daphne Chen, Simran Bagaria, Dean Fortier, Tess Hellebrekers, Harshavardhan Reddy Gajarla, Galen Mullins and Andrey Kolobov at Microsoft Research and Maya Cakmak at University of Washington
0:11

๐——๐—ผ๐—ป'๐˜ ๐—ณ๐—ถ๐—ป๐—ฒ-๐˜๐˜‚๐—ป๐—ฒ ๐—ฟ๐—ผ๐—ฏ๐—ผ๐˜ ๐—ณ๐—ผ๐˜‚๐—ป๐—ฑ๐—ฎ๐˜๐—ถ๐—ผ๐—ป ๐—บ๐—ผ๐—ฑ๐—ฒ๐—น๐˜€. ๐—ฆ๐˜๐—ฒ๐—ฒ๐—ฟ ๐˜๐—ต๐—ฒ๐—บ ๐˜„๐—ถ๐˜๐—ต ๐—ต๐˜‚๐—บ๐—ฎ๐—ป ๐—ฐ๐—ผ๐—ฟ๐—ฟ๐—ฒ๐—ฐ๐˜๐—ถ๐—ผ๐—ป๐˜€ ๐—ถ๐—ป๐˜€๐˜๐—ฒ๐—ฎ๐—ฑ, ๐˜„๐—ถ๐˜๐—ต๐—ผ๐˜‚๐˜ ๐—ฐ๐—ต๐—ฎ๐—ป๐—ด๐—ถ๐—ป๐—ด ๐˜๐—ต๐—ฒ ๐—ฏ๐—ฎ๐˜€๐—ฒ ๐—ฝ๐—ผ๐—น๐—ถ๐—ฐ๐˜† Modern VLAs and world-action models can perform impressive manipulation skills, but adapting them reliably to new robots and tasks remains challenging. A natural solution is DAgger-style online imitation learning: deploy the robot, collect human corrections, and update the policy. Yet foundation models are fragile in the low-data regime, fine-tuning on a handful of interventions can improve one behavior while degrading others. Online post-training or reinforcement learning can require costly data collection and exploration, making real-world learning expensive and potentially unsafe. In our new paper, ๐—™๐—น๐—ผ๐˜„๐——๐—”๐—ด๐—ด๐—ฒ๐—ฟ, we take a different approach: ๐—œ๐—ป๐˜€๐˜๐—ฒ๐—ฎ๐—ฑ ๐—ผ๐—ณ ๐—ฐ๐—ต๐—ฎ๐—ป๐—ด๐—ถ๐—ป๐—ด ๐˜๐—ต๐—ฒ ๐—ณ๐—ผ๐˜‚๐—ป๐—ฑ๐—ฎ๐˜๐—ถ๐—ผ๐—ป ๐—บ๐—ผ๐—ฑ๐—ฒ๐—น, ๐˜„๐—ฒ ๐—น๐—ฒ๐—ฎ๐—ฟ๐—ป ๐—ต๐—ผ๐˜„ ๐˜๐—ผ ๐˜€๐˜๐—ฒ๐—ฒ๐—ฟ ๐—ถ๐˜ ๐—ณ๐—ฟ๐—ผ๐—บ ๐—ต๐˜‚๐—บ๐—ฎ๐—ป ๐—ฐ๐—ผ๐—ฟ๐—ฟ๐—ฒ๐—ฐ๐˜๐—ถ๐—ผ๐—ป๐˜€. The key idea is ๐—ฎ๐—ฐ๐˜๐—ถ๐—ผ๐—ป ๐—ถ๐—ป๐˜ƒ๐—ฒ๐—ฟ๐˜€๐—ถ๐—ผ๐—ป: we map human corrective actions back into the latent noise space of the frozen generative policy. These latent targets train a lightweight controller that adapts the robot while preserving the original model's capabilities. Across simulation and real robots, FlowDAgger: ๐Ÿ“ˆ Learns from only 5โ€“20 human intervention episodes ๐Ÿ† Outperforms supervised fine-tuning and latent-space reinforcement learning ๐Ÿค– Works across VLAs, diffusion policies, and world-action models โœ”๏ธ Provides reliable improvements without modifying the pretrained policy We believe this offers a practical path toward making robot foundation models improve during deployment, learning from the way humans naturally teach: through corrections. ๐Ÿ“„ Paper: ๐ŸŒ Project: ๐Ÿ’ป Code: This project was led by my amazing colleague Michael Murray with help from Daphne Chen, Simran Bagaria, Dean Fortier, Tess Hellebrekers, Harshavardhan Reddy Gajarla, Galen Mullins and Andrey Kolobov at Microsoft Research and Maya Cakmak at University of Washington

Oier Mees

13,233 ๆฌก่ง‚็œ‹ โ€ข 1 ไธชๆœˆๅ‰

๐Ÿš€Thrilled to share what weโ€™ve been building at TRI over the past several months: our first Large Behavior Models (LBMs) are here! Iโ€™m proud to have been a core contributor to the multi-task policy learning and post-training efforts. At TRI, weโ€™ve been researching how LBMs can help robots learn faster, better, and more efficiently. The key takeaways: โœ… We built an evaluation pipeline to benchmark LBM performance with real ๐ฌ๐ญ๐š๐ญ๐ข๐ฌ๐ญ๐ข๐œ๐š๐ฅ ๐œ๐จ๐ง๐Ÿ๐ข๐๐ž๐ง๐œ๐ž โœ… Pre-training on hundreds of tasks makes models more robustโ€”plus, we can teach new, complex tasks with 80% ๐ฅ๐ž๐ฌ๐ฌ ๐๐š๐ญ๐š โœ… The bigger and more diverse the pre-training, the better the results Check out our overview video, webpage and paper for more details: โœจ ๐ŸŒŽ ๐Ÿ“„ We hope this work helps move the field of robotics forward!
0:18

๐Ÿš€Thrilled to share what weโ€™ve been building at TRI over the past several months: our first Large Behavior Models (LBMs) are here! Iโ€™m proud to have been a core contributor to the multi-task policy learning and post-training efforts. At TRI, weโ€™ve been researching how LBMs can help robots learn faster, better, and more efficiently. The key takeaways: โœ… We built an evaluation pipeline to benchmark LBM performance with real ๐ฌ๐ญ๐š๐ญ๐ข๐ฌ๐ญ๐ข๐œ๐š๐ฅ ๐œ๐จ๐ง๐Ÿ๐ข๐๐ž๐ง๐œ๐ž โœ… Pre-training on hundreds of tasks makes models more robustโ€”plus, we can teach new, complex tasks with 80% ๐ฅ๐ž๐ฌ๐ฌ ๐๐š๐ญ๐š โœ… The bigger and more diverse the pre-training, the better the results Check out our overview video, webpage and paper for more details: โœจ ๐ŸŒŽ ๐Ÿ“„ We hope this work helps move the field of robotics forward!

Zubair Irshad

20,377 ๆฌก่ง‚็œ‹ โ€ข 1 ๅนดๅ‰

Most imitation learning policies break when the camera moves or the robot changes. NOT THIS ONE ๐Ÿ‘‡ [๐Ÿ“ Bookmark for later ] A new 3D scene representation encoder, tackles this by enabling zero-shot generalization to unseen embodiments and viewpointsโ€ฆ And it works with any IL algorithm. The trick? โ€ขUse a 2D foundation model to extract semantic features โ€ขLift them into 3D space for localization (not semantics) โ€ขCondition the IL policy on this spatially grounded vector Across 93 simulated and 6 real tasks, Adapt3R: โœ… Maintains IL performance on LIBERO & MimicGen benchmarks โœ… Outperforms DP3 and 3D Diffuser Actor in most settings โœ… Holds >80% success on LIBERO even with large camera rotations Thanks for sharing this, Animesh Garg & Albert Wilcox! ๐Ÿ“Paper: Website: Code:
0:14

Most imitation learning policies break when the camera moves or the robot changes. NOT THIS ONE ๐Ÿ‘‡ [๐Ÿ“ Bookmark for later ] A new 3D scene representation encoder, tackles this by enabling zero-shot generalization to unseen embodiments and viewpointsโ€ฆ And it works with any IL algorithm. The trick? โ€ขUse a 2D foundation model to extract semantic features โ€ขLift them into 3D space for localization (not semantics) โ€ขCondition the IL policy on this spatially grounded vector Across 93 simulated and 6 real tasks, Adapt3R: โœ… Maintains IL performance on LIBERO & MimicGen benchmarks โœ… Outperforms DP3 and 3D Diffuser Actor in most settings โœ… Holds >80% success on LIBERO even with large camera rotations Thanks for sharing this, Animesh Garg & Albert Wilcox! ๐Ÿ“Paper: Website: Code:

Ilir Aliu

12,178 ๆฌก่ง‚็œ‹ โ€ข 1 ๅนดๅ‰

The Hidden Language of Diffusion Models paper page: tackle the challenge of understanding concept representations in text-to-image models by decomposing an input text prompt into a small set of interpretable elements. This is achieved by learning a pseudo-token that is a sparse weighted combination of tokens from the model's vocabulary, with the objective of reconstructing the images generated for the given concept. Applied over the state-of-the-art Stable Diffusion model, this decomposition reveals non-trivial and surprising structures in the representations of concepts. For example, we find that some concepts such as "a president" or "a composer" are dominated by specific instances (e.g., "Obama", "Biden") and their interpolations. Other concepts, such as "happiness" combine associated terms that can be concrete ("family", "laughter") or abstract ("friendship", "emotion"). In addition to peering into the inner workings of Stable Diffusion, our method also enables applications such as single-image decomposition to tokens, bias detection and mitigation, and semantic image manipulation
0:19

The Hidden Language of Diffusion Models paper page: tackle the challenge of understanding concept representations in text-to-image models by decomposing an input text prompt into a small set of interpretable elements. This is achieved by learning a pseudo-token that is a sparse weighted combination of tokens from the model's vocabulary, with the objective of reconstructing the images generated for the given concept. Applied over the state-of-the-art Stable Diffusion model, this decomposition reveals non-trivial and surprising structures in the representations of concepts. For example, we find that some concepts such as "a president" or "a composer" are dominated by specific instances (e.g., "Obama", "Biden") and their interpolations. Other concepts, such as "happiness" combine associated terms that can be concrete ("family", "laughter") or abstract ("friendship", "emotion"). In addition to peering into the inner workings of Stable Diffusion, our method also enables applications such as single-image decomposition to tokens, bias detection and mitigation, and semantic image manipulation

AK

41,830 ๆฌก่ง‚็œ‹ โ€ข 3 ๅนดๅ‰

What if you kept asking an LLM to "make it better"? In some recent work at FAIR, we investigate how we can efficiently use RL to fine-tune LLMs to iteratively self-improve on their previous solutions at inference-time. Training for iterated self-improvement can be costly. The naive approach to training for K self-improvement steps leads to K times the number of rollout steps per episode. We introduce Exploratory Iteration (ExIt), an RL-based automatic curriculum method that bootstraps diverse training distributions of self-improvement tasks by upcycling the LLM's own responses at previous turns as the starting points for both self-improvement and *self-divergence.* In order to decide what task to train on next, the curriculum prioritizes sampling of partial turn histories that led to higher return variance in its GRPO group (a learnability score that comes for free). This automatic curriculum over the bootstrapped task space teaches the model how to perform iterated self-improvement while only ever training the model on single-step self-improvement tasks. We look at ExIt's impact in both single-turn (contest math problems) and multi-turn (BFCLv3 multi-turn tasks), as well as MLE-bench, where the LLM is run in a search scaffold to produce solutions to real Kaggle competitions. Across these eval settings, we find ExIt produces models with greater capacity for inference-time self-improvement compared to GRPO. Notably, ExIt models can self-improve on test tasks for many more steps than the typical solution depth encountered during training, including a 22% improvement in MLE-bench performance compared to GRPO.
0:10

What if you kept asking an LLM to "make it better"? In some recent work at FAIR, we investigate how we can efficiently use RL to fine-tune LLMs to iteratively self-improve on their previous solutions at inference-time. Training for iterated self-improvement can be costly. The naive approach to training for K self-improvement steps leads to K times the number of rollout steps per episode. We introduce Exploratory Iteration (ExIt), an RL-based automatic curriculum method that bootstraps diverse training distributions of self-improvement tasks by upcycling the LLM's own responses at previous turns as the starting points for both self-improvement and *self-divergence.* In order to decide what task to train on next, the curriculum prioritizes sampling of partial turn histories that led to higher return variance in its GRPO group (a learnability score that comes for free). This automatic curriculum over the bootstrapped task space teaches the model how to perform iterated self-improvement while only ever training the model on single-step self-improvement tasks. We look at ExIt's impact in both single-turn (contest math problems) and multi-turn (BFCLv3 multi-turn tasks), as well as MLE-bench, where the LLM is run in a search scaffold to produce solutions to real Kaggle competitions. Across these eval settings, we find ExIt produces models with greater capacity for inference-time self-improvement compared to GRPO. Notably, ExIt models can self-improve on test tasks for many more steps than the typical solution depth encountered during training, including a 22% improvement in MLE-bench performance compared to GRPO.

Minqi Jiang

41,141 ๆฌก่ง‚็œ‹ โ€ข 1 ๅนดๅ‰

1/ World models are getting popular in robotics ๐Ÿค–โœจ But thereโ€™s a big problem: most are slow and break physical consistency over long horizons. 2/ Today weโ€™re releasing Interactive World Simulator: An action-conditioned world model that supports stable long-horizon interaction. 3/ Key result: โœ… 10+ minutes of interactive prediction โœ… 15 FPS โœ… on a single RTX 4090๐Ÿ”ฅ 4/ Why this matters: it unlocks two critical robotics applications: ๐Ÿš€ Scalable data generation for policy training ๐Ÿงช Faithful policy evaluation 5/ You can play with our world model NOW at NO git clone, NO pip install, NO python. Just click and play! NOTE โš ๏ธ ALL videos here are generated purely by our model in pixel space! They are **NOT** from a real camera More details coming ๐Ÿ‘‡ (1/9) #Robotics #AI #MachineLearning #WorldModels #RobotLearning #ImitationLearning
0:11

1/ World models are getting popular in robotics ๐Ÿค–โœจ But thereโ€™s a big problem: most are slow and break physical consistency over long horizons. 2/ Today weโ€™re releasing Interactive World Simulator: An action-conditioned world model that supports stable long-horizon interaction. 3/ Key result: โœ… 10+ minutes of interactive prediction โœ… 15 FPS โœ… on a single RTX 4090๐Ÿ”ฅ 4/ Why this matters: it unlocks two critical robotics applications: ๐Ÿš€ Scalable data generation for policy training ๐Ÿงช Faithful policy evaluation 5/ You can play with our world model NOW at NO git clone, NO pip install, NO python. Just click and play! NOTE โš ๏ธ ALL videos here are generated purely by our model in pixel space! They are **NOT** from a real camera More details coming ๐Ÿ‘‡ (1/9) #Robotics #AI #MachineLearning #WorldModels #RobotLearning #ImitationLearning

Yixuan Wang

129,557 ๆฌก่ง‚็œ‹ โ€ข 6 ไธชๆœˆๅ‰

The term "continual learning" has become overloaded if you see it as an ML problem. One classic thread is about memorization: regularization-based continual learning methods, such as EWC, MAS, and SI, estimate which parameters mattered for previous tasks and resist changing them too much. One modern thread is about adaptation: test-time training and inference-time learning methods, such as TTT, adapt part of the model on the incoming test stream before making predictions. These are sometimes discussed as separate threads. But in modern scalable architectures, I think they are better seen as complementary constraints: a model that learns quickly at test time also benefits from a mechanism for deciding what not to forget. In our #ECCV2026 paper, we study this in large-scale 4D reconstruction: how to build fast spatial memory that can adapt over long observation streams while reducing collapse and forgetting. Instead of using fully plastic test-time updates, we stabilize fast-weight adaptation with an elastic prior that balances adaptation and memory. Key ideas: - Elastic Test-Time Training: Fisher-weighted consolidation for fast-weight updates - EMA anchor weights that provide a moving reference for stability - Chunk-by-chunk inference for long 3D/4D observation streams We show that this scales across large 3D/4D pretraining settings, including both LRM-style and LVSM-style models, and improves reconstruction across benchmarks including Stereo4D, NVIDIA, and DL3DV-140. We release model checkpoints across different design choices: resolution, post-training curriculum, and whether the model uses an explicit 4DGS intermediate representation. - Homepage: - Paper: - Code: - Models: This work is co-led with Xueyang Yu, contributed by Haoyu Zhen Yuncong Yang, and advised by Michigan SLED Lab Chuang Gan.
0:14

The term "continual learning" has become overloaded if you see it as an ML problem. One classic thread is about memorization: regularization-based continual learning methods, such as EWC, MAS, and SI, estimate which parameters mattered for previous tasks and resist changing them too much. One modern thread is about adaptation: test-time training and inference-time learning methods, such as TTT, adapt part of the model on the incoming test stream before making predictions. These are sometimes discussed as separate threads. But in modern scalable architectures, I think they are better seen as complementary constraints: a model that learns quickly at test time also benefits from a mechanism for deciding what not to forget. In our #ECCV2026 paper, we study this in large-scale 4D reconstruction: how to build fast spatial memory that can adapt over long observation streams while reducing collapse and forgetting. Instead of using fully plastic test-time updates, we stabilize fast-weight adaptation with an elastic prior that balances adaptation and memory. Key ideas: - Elastic Test-Time Training: Fisher-weighted consolidation for fast-weight updates - EMA anchor weights that provide a moving reference for stability - Chunk-by-chunk inference for long 3D/4D observation streams We show that this scales across large 3D/4D pretraining settings, including both LRM-style and LVSM-style models, and improves reconstruction across benchmarks including Stereo4D, NVIDIA, and DL3DV-140. We release model checkpoints across different design choices: resolution, post-training curriculum, and whether the model uses an explicit 4DGS intermediate representation. - Homepage: - Paper: - Code: - Models: This work is co-led with Xueyang Yu, contributed by Haoyu Zhen Yuncong Yang, and advised by Michigan SLED Lab Chuang Gan.

Martin Ziqiao Ma

33,913 ๆฌก่ง‚็œ‹ โ€ข 2 ไธชๆœˆๅ‰

AI has had exactly two scaling axes that worked so far, and the second one is starting to look finite too the first one was pretraining: with scaling parameters and data, we got world knowledge (i.e. ChatGPT had read enough to know things), but it started saturating a while ago the second one was RL, and people had been doing RL the whole time before that: RLHF is RL but it never scaled far because it was trying to control the exact output, which tokens come out, how the text reads, but you can only push that so far before youโ€™re just polishing RLVR dropped that constraint: giving the model a task, then checking whether the final answer is right, and ignoring everything in between -- so the model does whatever it wants in the middle and only the endpoint gets graded, and thatโ€™s much closer to actual RL and itโ€™s what bought us planning and reasoning (arguably, tool use sits around 2.5 on this list -- while useful, it's not a different kind of thing) so one axis gave knowledge, the other gave reasoning, and both of them are one model working alone the next axis is how many models you can get working on the same problem, which is a different kind of axis than the previous two we know that multi-agent RL has always been the harder problem: I spent years in that literature and the gap between single-agent and multi-agent is definitely not incremental -- itโ€™s a whole different class of difficulty! which is also why the derivatives are steep at the start, nobody has picked the easy wins yet... and the thing that gates this multi-agent coordination is communication: models can only coordinate as well as they can exchange information, and right now they do that by writing sentences to each other imagine what could we possibly achieve if we properly open that third axis development by letting models to exchange information in their native "language" without loosing any computational data that they produce during inference
0:14

AI has had exactly two scaling axes that worked so far, and the second one is starting to look finite too the first one was pretraining: with scaling parameters and data, we got world knowledge (i.e. ChatGPT had read enough to know things), but it started saturating a while ago the second one was RL, and people had been doing RL the whole time before that: RLHF is RL but it never scaled far because it was trying to control the exact output, which tokens come out, how the text reads, but you can only push that so far before youโ€™re just polishing RLVR dropped that constraint: giving the model a task, then checking whether the final answer is right, and ignoring everything in between -- so the model does whatever it wants in the middle and only the endpoint gets graded, and thatโ€™s much closer to actual RL and itโ€™s what bought us planning and reasoning (arguably, tool use sits around 2.5 on this list -- while useful, it's not a different kind of thing) so one axis gave knowledge, the other gave reasoning, and both of them are one model working alone the next axis is how many models you can get working on the same problem, which is a different kind of axis than the previous two we know that multi-agent RL has always been the harder problem: I spent years in that literature and the gap between single-agent and multi-agent is definitely not incremental -- itโ€™s a whole different class of difficulty! which is also why the derivatives are steep at the start, nobody has picked the easy wins yet... and the thing that gates this multi-agent coordination is communication: models can only coordinate as well as they can exchange information, and right now they do that by writing sentences to each other imagine what could we possibly achieve if we properly open that third axis development by letting models to exchange information in their native "language" without loosing any computational data that they produce during inference

Sasha Malysheva

12,177 ๆฌก่ง‚็œ‹ โ€ข 1 ไธชๆœˆๅ‰

A viral paper "Language Model Represents Space and Time" recently claims that LLMs learn "world models". As much as I like Max Tegmark's works, I disagree with their definition of world model. World model is a core concept in AI agent and decision making. It is our mental simulation of how the world works given interventions (or lack thereof). A world model captures causality and intuitive physics, telling the agent what is likely and what is impossible. It can and should be used for counterfactual reasoning, i.e. "what ifs": what would happen if I knock over a cup of water? Where would I have been if I had not taken that bus? Yann LeCun Yann LeCun says it well in his position paper ( I quote: "Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation." The first use of the term World Model in deep policy learning is attributed to hardmaru & Jรผrgen Schmidhuber: In their seminal paper, an agent masters shooting skills in the popular game Doom (demo below) by learning in imagination, using an internal world model as a "physics simulator". To put in a simple Python math formula, world model learns a function F(s[0:t-1], a) -> s[t:], which takes as input the observed past and current action, and outputs plausible future states. Now the definition of World Model in Tegmark's paper seems to be about predicting GPS coordinates and time eras. I see this as just a classification task with no causal learning and simulation going on. You cannot make meaningful interventions against that model, nor can you optimize any decision making in a closed feedback loop. As for the "space & time neurons", I think they are most similar to the "sentiment neuron" that OpenAI published in 2017: Predicting GPS is conceptually no different from predicting sentiment in my opinion. I don't think their experimental results are wrong - just that their conclusion is on shaky grounds. I welcome any debate! Paper link:
0:15

A viral paper "Language Model Represents Space and Time" recently claims that LLMs learn "world models". As much as I like Max Tegmark's works, I disagree with their definition of world model. World model is a core concept in AI agent and decision making. It is our mental simulation of how the world works given interventions (or lack thereof). A world model captures causality and intuitive physics, telling the agent what is likely and what is impossible. It can and should be used for counterfactual reasoning, i.e. "what ifs": what would happen if I knock over a cup of water? Where would I have been if I had not taken that bus? Yann LeCun Yann LeCun says it well in his position paper ( I quote: "Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation." The first use of the term World Model in deep policy learning is attributed to hardmaru & Jรผrgen Schmidhuber: In their seminal paper, an agent masters shooting skills in the popular game Doom (demo below) by learning in imagination, using an internal world model as a "physics simulator". To put in a simple Python math formula, world model learns a function F(s[0:t-1], a) -> s[t:], which takes as input the observed past and current action, and outputs plausible future states. Now the definition of World Model in Tegmark's paper seems to be about predicting GPS coordinates and time eras. I see this as just a classification task with no causal learning and simulation going on. You cannot make meaningful interventions against that model, nor can you optimize any decision making in a closed feedback loop. As for the "space & time neurons", I think they are most similar to the "sentiment neuron" that OpenAI published in 2017: Predicting GPS is conceptually no different from predicting sentiment in my opinion. I don't think their experimental results are wrong - just that their conclusion is on shaky grounds. I welcome any debate! Paper link:

Jim Fan

594,014 ๆฌก่ง‚็œ‹ โ€ข 2 ๅนดๅ‰

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! ๐Ÿฅณ tl;dr: Faster, better RL than PPO in continuous control ๐Ÿ’ช The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)
0:10

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! ๐Ÿฅณ tl;dr: Faster, better RL than PPO in continuous control ๐Ÿ’ช The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Animesh Garg

52,308 ๆฌก่ง‚็œ‹ โ€ข 2 ๅนดๅ‰

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.
0:19

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.

AK

25,449 ๆฌก่ง‚็œ‹ โ€ข 2 ๅนดๅ‰

The ATG School One-Pager Iโ€™m not trying to reinvent schooling. There are just a handful of things I believe in which I havenโ€™t seen in any school Iโ€™ve been around as a student or parent. Policy #1: Each student gets to be responsible for growing some of their own food, no matter how small, and THROUGHOUT schooling (not just a quickie project here or there). Policy #2: Minimum 1:1 ratio of time NOT SITTING IN THE CLASSROOM. What you do with this is up to you. There are so many real world skills, sports, gardening, music, etc. The strict ratio in the school day is the key for me. Common sense and personal interests can take it from there. Policy #3: Daily time to read whatever you want to read about. The biggest barrier for my reading was INTEREST. Be there to ensure the book is at their level, and to help them if they donโ€™t understand something. Other than that, LET THEM ENJOY READING, ALL THE WAY THROUGH SCHOOL, not just in early years. Policy #4: (This is the most unusual yet the biggest reason Iโ€™m in education.) High school is a 50/50 bridge to winning in real life. Mornings are for actual work, making and SAVING UP MONEY. Afternoons are for learning finances and professional skills of YOUR INTEREST. With average work, youโ€™ll finish school with $50,000-$100,000 in the bank, more skills than the norm, and a greater chance of creating your life and work from there on out, rather than conforming to make a paycheck. Policy #5: As part of the high school 50/50 system, ensure each student learns the adult financial red tape in your state/country before youโ€™ve got bills, kids, etc.
0:12

The ATG School One-Pager Iโ€™m not trying to reinvent schooling. There are just a handful of things I believe in which I havenโ€™t seen in any school Iโ€™ve been around as a student or parent. Policy #1: Each student gets to be responsible for growing some of their own food, no matter how small, and THROUGHOUT schooling (not just a quickie project here or there). Policy #2: Minimum 1:1 ratio of time NOT SITTING IN THE CLASSROOM. What you do with this is up to you. There are so many real world skills, sports, gardening, music, etc. The strict ratio in the school day is the key for me. Common sense and personal interests can take it from there. Policy #3: Daily time to read whatever you want to read about. The biggest barrier for my reading was INTEREST. Be there to ensure the book is at their level, and to help them if they donโ€™t understand something. Other than that, LET THEM ENJOY READING, ALL THE WAY THROUGH SCHOOL, not just in early years. Policy #4: (This is the most unusual yet the biggest reason Iโ€™m in education.) High school is a 50/50 bridge to winning in real life. Mornings are for actual work, making and SAVING UP MONEY. Afternoons are for learning finances and professional skills of YOUR INTEREST. With average work, youโ€™ll finish school with $50,000-$100,000 in the bank, more skills than the norm, and a greater chance of creating your life and work from there on out, rather than conforming to make a paycheck. Policy #5: As part of the high school 50/50 system, ensure each student learns the adult financial red tape in your state/country before youโ€™ve got bills, kids, etc.

KneeOverToesGuy

31,525 ๆฌก่ง‚็œ‹ โ€ข 5 ไธชๆœˆๅ‰

Most recent diffusion language model research (that Iโ€™ve seen) seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labsโ€™ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation โ€œsettlesโ€ into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes โ€œprogressโ€ towards the final output each iteration (in addition to *potentially* giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. Iโ€™m gonna test this with larger models, since tiny-diffusion only has 10M parameters.
0:13

Most recent diffusion language model research (that Iโ€™ve seen) seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labsโ€™ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation โ€œsettlesโ€ into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes โ€œprogressโ€ towards the final output each iteration (in addition to *potentially* giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. Iโ€™m gonna test this with larger models, since tiny-diffusion only has 10M parameters.

nathan (in sf)

40,440 ๆฌก่ง‚็œ‹ โ€ข 7 ไธชๆœˆๅ‰

๐—ฅ๐—ผ๐—ฏ๐—ผ๐˜๐˜€ ๐—ฑ๐—ผ๐—ปโ€™๐˜ ๐—ป๐—ฒ๐—ฒ๐—ฑ ๐—บ๐—ผ๐—ฟ๐—ฒ ๐—ฑ๐—ฒ๐—บ๐—ผ๐—ป๐˜€๐˜๐—ฟ๐—ฎ๐˜๐—ถ๐—ผ๐—ป๐˜€. ๐—ง๐—ต๐—ฒ๐˜† ๐—ป๐—ฒ๐—ฒ๐—ฑ ๐˜๐—ผ ๐—น๐—ฒ๐—ฎ๐—ฟ๐—ป ๐—ณ๐—ฟ๐—ผ๐—บ ๐—ณ๐—ฎ๐—ถ๐—น๐˜‚๐—ฟ๐—ฒ โ€” ๐—ฎ๐—ณ๐˜๐—ฒ๐—ฟ ๐˜„๐—ฎ๐˜๐—ฐ๐—ต๐—ถ๐—ป๐—ด ๐—ต๐˜‚๐—บ๐—ฎ๐—ป๐˜€. Most robot learning systems assume failure is the end of learning. In our new work, we study whether robots can improve after deployment by learning from their own failures, without any human intervention, teleoperation, or corrective labels. The key idea is simple: human videos contain structure about how the world works. We use them to learn cross-embodiment representations of action, dynamics, and value, enabling a shared predictive space between human behavior and robot experience. This allows a new learning loop: ๐Ÿ‘‰ pretrain on human videos ๐Ÿ‘‰ deploy robot policy ๐Ÿ‘‰ observe failures ๐Ÿ‘‰ reinterpret failures using human priors ๐Ÿ‘‰ improve autonomously We evaluate this across 7 real-world manipulation tasks, showing: ๐Ÿ“ˆ 40% โ†’ 81% success rate ๐Ÿ† Strong improvements over ฯ€0.6 RECAP and RISE โœ”๏ธ Zero human intervention during post-deployment improvement ๐Ÿงฌ Generalizes across robot embodiments and policy backbones A key finding is that explicit failure repair significantly outperforms failure reweighting, yielding substantially larger gains under identical data conditions (+25 pts vs +5 pts on the same ฯ€0.5 base policy). Overall, the results suggest a shift in how we think about robot learning: Human videos are not only for pretraining policies. They can provide the structure needed for continual self-improvement after deployment. ๐Ÿ“„ Paper: ๐ŸŒ Project: I am grateful for working with the fantastic leads Hanzhi Chen and Anran Zhang, and our collaborators Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers. Special thanks to Stefan Leutenegger for co-advising this project with me. ETH Zรผrich TU Mรผnchen Microsoft Check out Hanzhi's ๐Ÿงต for more details
0:12

๐—ฅ๐—ผ๐—ฏ๐—ผ๐˜๐˜€ ๐—ฑ๐—ผ๐—ปโ€™๐˜ ๐—ป๐—ฒ๐—ฒ๐—ฑ ๐—บ๐—ผ๐—ฟ๐—ฒ ๐—ฑ๐—ฒ๐—บ๐—ผ๐—ป๐˜€๐˜๐—ฟ๐—ฎ๐˜๐—ถ๐—ผ๐—ป๐˜€. ๐—ง๐—ต๐—ฒ๐˜† ๐—ป๐—ฒ๐—ฒ๐—ฑ ๐˜๐—ผ ๐—น๐—ฒ๐—ฎ๐—ฟ๐—ป ๐—ณ๐—ฟ๐—ผ๐—บ ๐—ณ๐—ฎ๐—ถ๐—น๐˜‚๐—ฟ๐—ฒ โ€” ๐—ฎ๐—ณ๐˜๐—ฒ๐—ฟ ๐˜„๐—ฎ๐˜๐—ฐ๐—ต๐—ถ๐—ป๐—ด ๐—ต๐˜‚๐—บ๐—ฎ๐—ป๐˜€. Most robot learning systems assume failure is the end of learning. In our new work, we study whether robots can improve after deployment by learning from their own failures, without any human intervention, teleoperation, or corrective labels. The key idea is simple: human videos contain structure about how the world works. We use them to learn cross-embodiment representations of action, dynamics, and value, enabling a shared predictive space between human behavior and robot experience. This allows a new learning loop: ๐Ÿ‘‰ pretrain on human videos ๐Ÿ‘‰ deploy robot policy ๐Ÿ‘‰ observe failures ๐Ÿ‘‰ reinterpret failures using human priors ๐Ÿ‘‰ improve autonomously We evaluate this across 7 real-world manipulation tasks, showing: ๐Ÿ“ˆ 40% โ†’ 81% success rate ๐Ÿ† Strong improvements over ฯ€0.6 RECAP and RISE โœ”๏ธ Zero human intervention during post-deployment improvement ๐Ÿงฌ Generalizes across robot embodiments and policy backbones A key finding is that explicit failure repair significantly outperforms failure reweighting, yielding substantially larger gains under identical data conditions (+25 pts vs +5 pts on the same ฯ€0.5 base policy). Overall, the results suggest a shift in how we think about robot learning: Human videos are not only for pretraining policies. They can provide the structure needed for continual self-improvement after deployment. ๐Ÿ“„ Paper: ๐ŸŒ Project: I am grateful for working with the fantastic leads Hanzhi Chen and Anran Zhang, and our collaborators Simon Schaefer, Kejia Chen, Shi Chen, Daniel Cremers. Special thanks to Stefan Leutenegger for co-advising this project with me. ETH Zรผrich TU Mรผnchen Microsoft Check out Hanzhi's ๐Ÿงต for more details

Oier Mees

12,514 ๆฌก่ง‚็œ‹ โ€ข 2 ไธชๆœˆๅ‰

Whether you are a random person on X or Elon Musk, I urge you all to please read this post and stop using videos like this one to claim that Biden has an "open border policy": #1) The people in this video are already on American soil. The agents have a few options here. Detain them behind the razor wire or detain them in front of it, let the children cut themselves on razor wire or make sure they don't get cut. #2) In May President Biden issued an "asylum ban" which barred migrants from applying for humanitarian protection if they cross the border against the law or fail to first apply for safe harbor while crossing through another country on the way to the U.S. The US COURTS OVERTURNED IT. Biden also requested $3.5B for more border patrol agents and judges and lawyers. Republicans refused. #3) Let me explain to you all why the people in this video can't simply be put on a truck and driven back into Mexico and dropped off: a - They have a right to due process according to our Constitution. b - There are international laws indicating that we can't just drop people off from Honduras or Cuba or any other nation into Mexico. What if one of these people claimed they were from Canada? Should we drop them in Canada? c - How do we immediately prove that a person is not an American, or an Italian, or a Brazilian if they claim they are? Stop pretending that Biden isn't doing anything. Stop pretending that the immigration problem is simple. Stop pretending that this has not been a problem for decades. Understand that the surge of migrants isn't the fault of Biden but actually global geopolitical upheaval in Cuba, Nicaragua, Venezuela and other nations. Instead of pointing fingers and claiming that "Biden has an open border policy," when his policy is not much different than those we have had for the last 40 years, how about working together on comprehensive reform? So before you make another Tweet claiming Biden has an Open Border Policy, how about instead you provide actual solutions that are LEGAL, that should be implemented.
0:18

Whether you are a random person on X or Elon Musk, I urge you all to please read this post and stop using videos like this one to claim that Biden has an "open border policy": #1) The people in this video are already on American soil. The agents have a few options here. Detain them behind the razor wire or detain them in front of it, let the children cut themselves on razor wire or make sure they don't get cut. #2) In May President Biden issued an "asylum ban" which barred migrants from applying for humanitarian protection if they cross the border against the law or fail to first apply for safe harbor while crossing through another country on the way to the U.S. The US COURTS OVERTURNED IT. Biden also requested $3.5B for more border patrol agents and judges and lawyers. Republicans refused. #3) Let me explain to you all why the people in this video can't simply be put on a truck and driven back into Mexico and dropped off: a - They have a right to due process according to our Constitution. b - There are international laws indicating that we can't just drop people off from Honduras or Cuba or any other nation into Mexico. What if one of these people claimed they were from Canada? Should we drop them in Canada? c - How do we immediately prove that a person is not an American, or an Italian, or a Brazilian if they claim they are? Stop pretending that Biden isn't doing anything. Stop pretending that the immigration problem is simple. Stop pretending that this has not been a problem for decades. Understand that the surge of migrants isn't the fault of Biden but actually global geopolitical upheaval in Cuba, Nicaragua, Venezuela and other nations. Instead of pointing fingers and claiming that "Biden has an open border policy," when his policy is not much different than those we have had for the last 40 years, how about working together on comprehensive reform? So before you make another Tweet claiming Biden has an Open Border Policy, how about instead you provide actual solutions that are LEGAL, that should be implemented.

Brian Krassenstein

7,720,381 ๆฌก่ง‚็œ‹ โ€ข 2 ๅนดๅ‰

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.
0:13

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

MrNeRF

27,428 ๆฌก่ง‚็œ‹ โ€ข 1 ๅนดๅ‰