Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

1/ RL for improving task success has made LLMs remarkably capable. But what happens when the model fails? Turns out, training for graceful failure can be surprisingly tricky! Very excited to share our new paper: Training Language Models to Fail Gracefully 🧵

Daman Arora

1,038 subscribers

57,877 Aufrufe • vor 10 Tagen •via X (Twitter)

Gesundheit & Wellness Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Excited to share CrystalReasoner, a reasoning model for crystal structure generation with LLMs and property-conditioned generation through RL: Website: Paper: Code:

Excited to share CrystalReasoner, a reasoning model for crystal structure generation with LLMs and property-conditioned generation through RL: Website: Paper: Code:

Sherry Yang

11,531 Aufrufe • vor 5 Monaten

Reinforcement learning should be able to improve upon behaviors seen when training. In practice, RL agents often struggle to generalize to new long-horizon behaviors. Our new paper studies *horizon generalization*, the degree RL algorithms generalize to reaching distant goals. 1/

Reinforcement learning should be able to improve upon behaviors seen when training. In practice, RL agents often struggle to generalize to new long-horizon behaviors. Our new paper studies horizon generalization, the degree RL algorithms generalize to reaching distant goals. 1/

Vivek Myers

79,523 Aufrufe • vor 1 Jahr

ChatGPT uses your data for training its Large Language Model (LLM). But now you can ask ChatGPT to NOT use data for training its model. Go to "Settings." Click on "Show" in front of "Data Controls." Turn off "Chat History and Training."

ChatGPT uses your data for training its Large Language Model (LLM). But now you can ask ChatGPT to NOT use data for training its model. Go to "Settings." Click on "Show" in front of "Data Controls." Turn off "Chat History and Training."

Mushtaq Bilal, PhD

68,006 Aufrufe • vor 3 Jahren

My Melody wishes for the whole world to be origami! 🩷 But when everything turns paper-perfect, she realizes real life might be even sweeter. Find out what happens on the #HelloKittyandFriends YouTube channel:

My Melody wishes for the whole world to be origami! 🩷 But when everything turns paper-perfect, she realizes real life might be even sweeter. Find out what happens on the #HelloKittyandFriends YouTube channel:

Sanrio

16,978 Aufrufe • vor 11 Monaten

How can agents like LLMs become decision-making partners for humans? 💬 Excited to share a new paper + suite of envs for 𝘥𝘦𝘤𝘪𝘴𝘪𝘰𝘯-𝘰𝘳𝘪𝘦𝘯𝘵𝘦𝘥 𝘥𝘪𝘢𝘭𝘰𝘨𝘶𝘦𝘴, where agents + humans collab to solve hard everyday problems. [1/n] Site:

How can agents like LLMs become decision-making partners for humans? 💬 Excited to share a new paper + suite of envs for 𝘥𝘦𝘤𝘪𝘴𝘪𝘰𝘯-𝘰𝘳𝘪𝘦𝘯𝘵𝘦𝘥 𝘥𝘪𝘢𝘭𝘰𝘨𝘶𝘦𝘴, where agents + humans collab to solve hard everyday problems. [1/n] Site:

Jessy Lin

86,897 Aufrufe • vor 3 Jahren

Models love reward hacking. Maybe we should just let them! In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!

Models love reward hacking. Maybe we should just let them! In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!

Jake Ward

301,477 Aufrufe • vor 7 Tagen

As a supporter of the open weights ecosystem, we're proud to be a post-training partner for NVIDIA Nemotron. We post-train Nemotron models for customer use cases, de-risk mainline RL runs on our AC2 platform and training stack, and contribute aggregate workload statistics for inference benchmarking. This is how open models get better, and we're excited to keep working closely with NVIDIA AI.

As a supporter of the open weights ecosystem, we're proud to be a post-training partner for NVIDIA Nemotron. We post-train Nemotron models for customer use cases, de-risk mainline RL runs on our AC2 platform and training stack, and contribute aggregate workload statistics for inference benchmarking. This is how open models get better, and we're excited to keep working closely with NVIDIA AI.

Applied Compute

25,683 Aufrufe • vor 2 Monaten

We're very excited to introduce TAPNext: a model that sets a new state-of-art for Tracking Any Point in videos, by formulating the task as Next Token Prediction. For more, see: 🧵

We're very excited to introduce TAPNext: a model that sets a new state-of-art for Tracking Any Point in videos, by formulating the task as Next Token Prediction. For more, see: 🧵

Carl Doersch

45,355 Aufrufe • vor 1 Jahr

Diffusion models are great, but we can squeeze out so much more from them. The only problem is that it usually requires extra training or manual representation editing. In our new paper, we show that with the current capabilities of LLMs, it is much simpler than we thought!

Diffusion models are great, but we can squeeze out so much more from them. The only problem is that it usually requires extra training or manual representation editing. In our new paper, we show that with the current capabilities of LLMs, it is much simpler than we thought!

Yossi Gandelsman

34,925 Aufrufe • vor 7 Monaten

Excited to share GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Training, which received the Best Paper Award at the ICLR 2026 Workshop on Foundation Models for Science. Can we scale neural physics simulation without scaling expensive solver-generated labels? (1/6) (The below results are all predicted by GeoPT.)

Excited to share GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Training, which received the Best Paper Award at the ICLR 2026 Workshop on Foundation Models for Science. Can we scale neural physics simulation without scaling expensive solver-generated labels? (1/6) (The below results are all predicted by GeoPT.)

Minghao Guo

39,620 Aufrufe • vor 6 Monaten

Most of what I actually need help with, I never think to tell a model. But why is it on me to remember? Our new paper asks: what if AI could proactively specialize to individuals and the tasks they’re carrying out at this very moment? 🧵

Most of what I actually need help with, I never think to tell a model. But why is it on me to remember? Our new paper asks: what if AI could proactively specialize to individuals and the tasks they’re carrying out at this very moment? 🧵

Michelle Lam

50,971 Aufrufe • vor 6 Monaten

Really excited to launch FrontierSWE v2! In one task, we ask models to build an OpenGL engine that can render a flight simulator game. Excited to share more detailed analysis in the coming days but checkout the blog for more.

Really excited to launch FrontierSWE v2! In one task, we ask models to build an OpenGL engine that can render a flight simulator game. Excited to share more detailed analysis in the coming days but checkout the blog for more.

Calvin Chen

12,285 Aufrufe • vor 1 Monat

Wouldn’t it be nice if ChatGPT could find your missing keys for you? Our latest research from Berkeley AI Research + Google AI suggests that robots can use large language models (LLMs) to find hidden objects faster. 🧵👇

Wouldn’t it be nice if ChatGPT could find your missing keys for you? Our latest research from Berkeley AI Research + Google AI suggests that robots can use large language models (LLMs) to find hidden objects faster. 🧵👇

Kaushik Shivakumar

58,523 Aufrufe • vor 3 Jahren

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!

Tongzhou Mu 🤖🦾🦿

17,077 Aufrufe • vor 1 Jahr

Why am I working on RL for LLMs, when I can go work on RL for Olaf... #humanoidssummit

Why am I working on RL for LLMs, when I can go work on RL for Olaf... #humanoidssummit

Shane Gu

12,503 Aufrufe • vor 10 Monaten

🚀Thrilled to share what we’ve been building at TRI over the past several months: our first Large Behavior Models (LBMs) are here! I’m proud to have been a core contributor to the multi-task policy learning and post-training efforts. At TRI, we’ve been researching how LBMs can help robots learn faster, better, and more efficiently. The key takeaways: ✅ We built an evaluation pipeline to benchmark LBM performance with real 𝐬𝐭𝐚𝐭𝐢𝐬𝐭𝐢𝐜𝐚𝐥 𝐜𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐜𝐞 ✅ Pre-training on hundreds of tasks makes models more robust—plus, we can teach new, complex tasks with 80% 𝐥𝐞𝐬𝐬 𝐝𝐚𝐭𝐚 ✅ The bigger and more diverse the pre-training, the better the results Check out our overview video, webpage and paper for more details: ✨ 🌎 📄 We hope this work helps move the field of robotics forward!

🚀Thrilled to share what we’ve been building at TRI over the past several months: our first Large Behavior Models (LBMs) are here! I’m proud to have been a core contributor to the multi-task policy learning and post-training efforts. At TRI, we’ve been researching how LBMs can help robots learn faster, better, and more efficiently. The key takeaways: ✅ We built an evaluation pipeline to benchmark LBM performance with real 𝐬𝐭𝐚𝐭𝐢𝐬𝐭𝐢𝐜𝐚𝐥 𝐜𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐜𝐞 ✅ Pre-training on hundreds of tasks makes models more robust—plus, we can teach new, complex tasks with 80% 𝐥𝐞𝐬𝐬 𝐝𝐚𝐭𝐚 ✅ The bigger and more diverse the pre-training, the better the results Check out our overview video, webpage and paper for more details: ✨ 🌎 📄 We hope this work helps move the field of robotics forward!

Zubair Irshad

20,518 Aufrufe • vor 1 Jahr

What happens when AI agents put thousands of GPUs to work improving AI itself? Meet OpenRSI Index. 👇 The goal: measure whether AI research agents can improve on human-designed training recipes. Real model-development tasks: 🔹 Pre-training 🔹 Post-training 🔹 Vision generation ⚡ Up to 20,864 H100-hours per run in the featured examples. Can agents discover better methods? Let’s put them to the test. Great work by OpenRSI Foundation and the research community! 🚀

What happens when AI agents put thousands of GPUs to work improving AI itself? Meet OpenRSI Index. 👇 The goal: measure whether AI research agents can improve on human-designed training recipes. Real model-development tasks: 🔹 Pre-training 🔹 Post-training 🔹 Vision generation ⚡ Up to 20,864 H100-hours per run in the featured examples. Can agents discover better methods? Let’s put them to the test. Great work by OpenRSI Foundation and the research community! 🚀

Pan Lu

45,084 Aufrufe • vor 1 Monat

What if you kept asking an LLM to "make it better"? In some recent work at FAIR, we investigate how we can efficiently use RL to fine-tune LLMs to iteratively self-improve on their previous solutions at inference-time. Training for iterated self-improvement can be costly. The naive approach to training for K self-improvement steps leads to K times the number of rollout steps per episode. We introduce Exploratory Iteration (ExIt), an RL-based automatic curriculum method that bootstraps diverse training distributions of self-improvement tasks by upcycling the LLM's own responses at previous turns as the starting points for both self-improvement and *self-divergence.* In order to decide what task to train on next, the curriculum prioritizes sampling of partial turn histories that led to higher return variance in its GRPO group (a learnability score that comes for free). This automatic curriculum over the bootstrapped task space teaches the model how to perform iterated self-improvement while only ever training the model on single-step self-improvement tasks. We look at ExIt's impact in both single-turn (contest math problems) and multi-turn (BFCLv3 multi-turn tasks), as well as MLE-bench, where the LLM is run in a search scaffold to produce solutions to real Kaggle competitions. Across these eval settings, we find ExIt produces models with greater capacity for inference-time self-improvement compared to GRPO. Notably, ExIt models can self-improve on test tasks for many more steps than the typical solution depth encountered during training, including a 22% improvement in MLE-bench performance compared to GRPO.

What if you kept asking an LLM to "make it better"? In some recent work at FAIR, we investigate how we can efficiently use RL to fine-tune LLMs to iteratively self-improve on their previous solutions at inference-time. Training for iterated self-improvement can be costly. The naive approach to training for K self-improvement steps leads to K times the number of rollout steps per episode. We introduce Exploratory Iteration (ExIt), an RL-based automatic curriculum method that bootstraps diverse training distributions of self-improvement tasks by upcycling the LLM's own responses at previous turns as the starting points for both self-improvement and self-divergence. In order to decide what task to train on next, the curriculum prioritizes sampling of partial turn histories that led to higher return variance in its GRPO group (a learnability score that comes for free). This automatic curriculum over the bootstrapped task space teaches the model how to perform iterated self-improvement while only ever training the model on single-step self-improvement tasks. We look at ExIt's impact in both single-turn (contest math problems) and multi-turn (BFCLv3 multi-turn tasks), as well as MLE-bench, where the LLM is run in a search scaffold to produce solutions to real Kaggle competitions. Across these eval settings, we find ExIt produces models with greater capacity for inference-time self-improvement compared to GRPO. Notably, ExIt models can self-improve on test tasks for many more steps than the typical solution depth encountered during training, including a 22% improvement in MLE-bench performance compared to GRPO.

Minqi Jiang

41,154 Aufrufe • vor 1 Jahr

Ever wondered how training dynamics differ between LLMs 🖋️ and Vision 👁️ models? We explore this and close the gap between VMs and LLMs in our #NeurIPS2024 paper "TrAct: Making First-layer Pre-Activations Trainable". Paper📜 Video🎥

Ever wondered how training dynamics differ between LLMs 🖋️ and Vision 👁️ models? We explore this and close the gap between VMs and LLMs in our #NeurIPS2024 paper "TrAct: Making First-layer Pre-Activations Trainable". Paper📜 Video🎥

Felix Petersen

20,993 Aufrufe • vor 1 Jahr