Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Does LLM RL post-training need to be on-policy?

xkianteb's profile picture

Kianté Brantley

1,907 subscribers

113,263 görüntüleme • 3 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

LLM training on RTX 5090
0:12

LLM training on RTX 5090

ℏεsam

144,051 görüntüleme • 1 yıl önce

Zombie robot RL policy
0:16

Zombie robot RL policy

Simon Kalouche

164,319 görüntüleme • 11 ay önce

Does off-policy value-based RL scale? In LLMs, larger scale predictably improves performance. Value-based RL learns from arbitrary data and is sample-efficient, but folk wisdom says it doesn't scale 🧵⬇️We show predictability for scaling value-based RL!
0:16

Does off-policy value-based RL scale? In LLMs, larger scale predictably improves performance. Value-based RL learns from arbitrary data and is sample-efficient, but folk wisdom says it doesn't scale 🧵⬇️We show predictability for scaling value-based RL!

Oleg Rybkin

23,968 görüntüleme • 1 yıl önce

Tutorial Time: Run any open-source LLM locally. Now we will run an LLM on your M1/2 Mac. And its fast. All you need is moved to: @lmstudio let's get started. Good to be back. A thread
0:17

Tutorial Time: Run any open-source LLM locally. Now we will run an LLM on your M1/2 Mac. And its fast. All you need is moved to: @lmstudio let's get started. Good to be back. A thread

Linus ✦ Ekenstam

915,753 görüntüleme • 2 yıl önce

Reinforcement learning should be able to improve upon behaviors seen when training. In practice, RL agents often struggle to generalize to new long-horizon behaviors. Our new paper studies *horizon generalization*, the degree RL algorithms generalize to reaching distant goals. 1/
0:15

Reinforcement learning should be able to improve upon behaviors seen when training. In practice, RL agents often struggle to generalize to new long-horizon behaviors. Our new paper studies *horizon generalization*, the degree RL algorithms generalize to reaching distant goals. 1/

Vivek Myers

79,514 görüntüleme • 1 yıl önce

New project! Flow Policy Gradients for Robot Control tldr; a simple online RL recipe for training and fine-tuning flow policies for robots co-led w/ Hongsuk Benjamin Choi:
0:18

New project! Flow Policy Gradients for Robot Control tldr; a simple online RL recipe for training and fine-tuning flow policies for robots co-led w/ Hongsuk Benjamin Choi:

Brent Yi

73,790 görüntüleme • 4 ay önce

How thick does ice need to be before you can walk on it? 😭
0:10

How thick does ice need to be before you can walk on it? 😭

AlphaFox

43,053 görüntüleme • 4 ay önce

Does your bed board need to be replaced 😏
0:18

Does your bed board need to be replaced 😏

sytoys-us1

49,674 görüntüleme • 8 ay önce

Young man does not say "Thank you Sir" will need further training
0:15

Young man does not say "Thank you Sir" will need further training

newbeginning

12,898 görüntüleme • 1 yıl önce

How it feels to be an LLM
0:14

How it feels to be an LLM

Beff (e/acc)

36,669 görüntüleme • 1 yıl önce

How does high-fidelity tactile simulation help robots nail the last millimeter? We’re releasing VT-Refine, accepted to CoRL: a real-to-sim-to-real visuo-tactile policy using a GPU-parallel tactile sim for our piezoresistive skin FlexiTac. Then fine-tuning a diffusion policy with large-scale RL in simulation. Website: #CoRL2025 #RobotLearning #Sim2Real
0:17

How does high-fidelity tactile simulation help robots nail the last millimeter? We’re releasing VT-Refine, accepted to CoRL: a real-to-sim-to-real visuo-tactile policy using a GPU-parallel tactile sim for our piezoresistive skin FlexiTac. Then fine-tuning a diffusion policy with large-scale RL in simulation. Website: #CoRL2025 #RobotLearning #Sim2Real

Binghao Huang

46,967 görüntüleme • 7 ay önce

i need to post on here more oops
0:12

i need to post on here more oops

haley ⋆✧.*

14,443 görüntüleme • 1 yıl önce

i need to post more on here 😩
0:15

i need to post more on here 😩

MS. F!NEE $HITT

124,925 görüntüleme • 1 yıl önce

🚨Current scalable RL algos train a policy w/o value func, which is limiting with learning in open-ended, non-stationary, dynamic environments. But, how to scale value-based RL with more data/compute is unclear... Not anymore: presenting scaling laws for value-based RL 🧵⬇️
0:10

🚨Current scalable RL algos train a policy w/o value func, which is limiting with learning in open-ended, non-stationary, dynamic environments. But, how to scale value-based RL with more data/compute is unclear... Not anymore: presenting scaling laws for value-based RL 🧵⬇️

Aviral Kumar

37,284 görüntüleme • 1 yıl önce

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!
0:16

🤔 How to fine-tune an Imitation Learning policy (e.g., Diffusion Policy, ACT) with RL? As an RL practitioner, I’ve been struggling with this problem for a while. Here’s why it’s tough: 1️⃣ Special designs (usually for multimodal action distributions) in modern IL models make them non-trivial to fine-tune by RL. 2️⃣ Large policy models + RL's poor sample efficiency = a nightmare But finally, we figured out a simple solution that works for any model architecture! 🌟 Check out our #ICLR2025 paper: “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Models”, led by my amazing mentee Xiu Yuan. 🔗 🧵 Read more below!

Tongzhou Mu 🤖🦾🦿

16,923 görüntüleme • 1 yıl önce

Why do kids need this training? Be honest
0:17

Why do kids need this training? Be honest

Firearm Videos

19,276 görüntüleme • 4 ay önce

I love this guy. He does not need to be doing all that
0:16

I love this guy. He does not need to be doing all that

Dex

35,821 görüntüleme • 1 yıl önce

Introducing RL Environment Creator Skill Now any one can create RL environments $ npx skills add adithya-s-k/RL_Envs_101 > You can create environments across multiple frameworks like OpenEnv, OpenReward, Verifiers, NemoGym ... > the repo has live working examples of environments that your coding agent can reference > The skill is design to first understand what type of model you are training and create an environment while keeping that in mind ps. There’s a lot more to building RL environments that can be used for training. One major aspect is the data, which this skill can’t directly solve. However, the skill will help with implementing tools, rewards, and other components of an RL environment, making it easier to go from idea to implementation quickly across different frameworks. Let me know if you’d be interested in a detailed, end-to-end blog/tutorial on building an environment and actually training a model for a useful use case.
0:18

Introducing RL Environment Creator Skill Now any one can create RL environments $ npx skills add adithya-s-k/RL_Envs_101 > You can create environments across multiple frameworks like OpenEnv, OpenReward, Verifiers, NemoGym ... > the repo has live working examples of environments that your coding agent can reference > The skill is design to first understand what type of model you are training and create an environment while keeping that in mind ps. There’s a lot more to building RL environments that can be used for training. One major aspect is the data, which this skill can’t directly solve. However, the skill will help with implementing tools, rewards, and other components of an RL environment, making it easier to go from idea to implementation quickly across different frameworks. Let me know if you’d be interested in a detailed, end-to-end blog/tutorial on building an environment and actually training a model for a useful use case.

Adithya S K

46,445 görüntüleme • 1 ay önce

POST THIS ON WV…I NEED TO SEE TAEHYUNG’S REACTION 😭
0:19

POST THIS ON WV…I NEED TO SEE TAEHYUNG’S REACTION 😭

taehyꪜng

23,053 görüntüleme • 1 yıl önce