正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Introducing RL Visualizer See PPO and GRPO mentioned everywhere but don't know what actually makes them different? Visualize and compare these algorithms in a simple online maze environment! 🚀

alphaXiv

41,033 subscribers

78,947 次观看 • 7 个月前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Want to scale RL with your shiny new GPU? 🚀 In our ICML24 Oral we find that RL algorithms hit a barrier when data is scaled up. Our new algorithm, SAPG, proposes a simple fix. It scales to 25k envs and solves hard tasks where PPO makes no progress. 1/n

Want to scale RL with your shiny new GPU? 🚀 In our ICML24 Oral we find that RL algorithms hit a barrier when data is scaled up. Our new algorithm, SAPG, proposes a simple fix. It scales to 25k envs and solves hard tasks where PPO makes no progress. 1/n

Ananye Agarwal

64,403 次观看 • 1 年前

Let’s talk about our new orthodontic pacifiers and what makes them different! In this video I wanted to highlight all the features of our new design and what makes these unique~

Let’s talk about our new orthodontic pacifiers and what makes them different! In this video I wanted to highlight all the features of our new design and what makes these unique~

Lil Comforts

47,233 次观看 • 6 个月前

I don't know but all in all we now understand what and why all these happened .

I don't know but all in all we now understand what and why all these happened .

𝗩𝗶𝗴𝗶𝗹𝗮𝗻𝘁𝗲 𝗩𝗮𝗻𝗴𝘂𝗮𝗿𝗱

164,825 次观看 • 1 年前

When I say Ødegaard lacks conviction in his actions, this is what I mean. Do you see the urgency, creativity, and spontaneity here? That’s what makes a big mover. Now compare that to Ødegaard it's a different level of intent and directness.

When I say Ødegaard lacks conviction in his actions, this is what I mean. Do you see the urgency, creativity, and spontaneity here? That’s what makes a big mover. Now compare that to Ødegaard it's a different level of intent and directness.

Arsenal X-tra

473,389 次观看 • 7 个月前

What did Peter Obi say that was different from what's coming from the mouth of these Pastors? We a know the truth and the simple message PO was trying to pass, but let's keep pretending and forming religious voltrons in a dying economy.

What did Peter Obi say that was different from what's coming from the mouth of these Pastors? We a know the truth and the simple message PO was trying to pass, but let's keep pretending and forming religious voltrons in a dying economy.

WiII E. 🏝️

341,439 次观看 • 1 年前

They know they’re doing fraud but you will see them floating luxury cars and money online. When will these boys learn sense? Arrangement and Lanchester paaa 💔💔💔💔

They know they’re doing fraud but you will see them floating luxury cars and money online. When will these boys learn sense? Arrangement and Lanchester paaa 💔💔💔💔

Gh Durk 🇬🇭🌎

43,152 次观看 • 2 个月前

They’re calling it a “dragon”… but nobody can prove what it actually is. That hasn’t stopped it from spreading everywhere online. In Romania, a man claims to have found and excavated this large skull, which some are claiming is a dragon. What do you see, discovery or deception? 🐉

They’re calling it a “dragon”… but nobody can prove what it actually is. That hasn’t stopped it from spreading everywhere online. In Romania, a man claims to have found and excavated this large skull, which some are claiming is a dragon. What do you see, discovery or deception? 🐉

𝙲𝚘𝚗𝚜𝚎𝚛𝚟𝚊𝚝𝚒𝚟𝚎 👑 𝙳𝚒𝚟𝚊™

25,683 次观看 • 2 个月前

They put these same lessons over and over everywhere under the cover of entertainment, but really it's a school. Run by who I really don't know.

They put these same lessons over and over everywhere under the cover of entertainment, but really it's a school. Run by who I really don't know.

Invisidon

54,063 次观看 • 1 年前

I'm a curator of a museum of art. I think these watches tell a story in every case, and they're all different. You know, they're striking enough that you see something different in every dial, and that's exactly what I'm trying to achieve.

I'm a curator of a museum of art. I think these watches tell a story in every case, and they're all different. You know, they're striking enough that you see something different in every dial, and that's exactly what I'm trying to achieve.

Kevin O'Leary aka Mr. Wonderful

25,767 次观看 • 1 年前

Sungjae was mentioned in Han Chaeah's latest content HCA is trying to get these high school students to know her. They know YSJ and The Golden Spoon, but they don't believe HCA plays his mother there. Well her style as a poor mother is so different from her style irl😂

Sungjae was mentioned in Han Chaeah's latest content HCA is trying to get these high school students to know her. They know YSJ and The Golden Spoon, but they don't believe HCA plays his mother there. Well her style as a poor mother is so different from her style irl😂

fy🍀🎶

15,774 次观看 • 2 个月前

But what they really want is a forever family ❤️ Meet these perfect dogs and see what makes each of them unforgettable! To learn more about these precious pups, check out Franklin County Dog Shelter & Adoption Center on Instagram:

But what they really want is a forever family ❤️ Meet these perfect dogs and see what makes each of them unforgettable! To learn more about these precious pups, check out Franklin County Dog Shelter & Adoption Center on Instagram:

The Dodo

46,936 次观看 • 6 个月前

I don't even know where we begin fixing society. Send them back and see what happens is a start I guess.

I don't even know where we begin fixing society. Send them back and see what happens is a start I guess.

Russian Garbage Human

101,659 次观看 • 3 个月前

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."

TBPN

85,358 次观看 • 5 个月前

“What is an intifada?” Blank stares. These people live in an environment where they just learn a bunch of slogans and it violates the norms of the community to expect them to know what they’re talking about.

“What is an intifada?” Blank stares. These people live in an environment where they just learn a bunch of slogans and it violates the norms of the community to expect them to know what they’re talking about.

Richard Hanania

1,061,039 次观看 • 2 年前

🚨 🇺🇸 SEN. FETTERMAN: DON’T EVER COMPARE ANYONE TO HITLER. IT INCITES VIOLENCE "Do not ever, ever, ever compare anyone to Hitler. If not, go online and read up on exactly what he's responsible for. You know, 75 to 80 million lives were lost in World War II, and you don't compare him to anyone." Source: Bret Baier

🚨 🇺🇸 SEN. FETTERMAN: DON’T EVER COMPARE ANYONE TO HITLER. IT INCITES VIOLENCE "Do not ever, ever, ever compare anyone to Hitler. If not, go online and read up on exactly what he's responsible for. You know, 75 to 80 million lives were lost in World War II, and you don't compare him to anyone." Source: Bret Baier

Mario Nawfal

149,031 次观看 • 10 个月前

This clip from a Disney vacation planning DVD from 2008 makes Disney World look so easy and affordable. Things look much different now. I know prices rise, and I don’t really have a problem with that. But a few things mentioned in this clip that aren’t there now makes the price harder to swallow…

This clip from a Disney vacation planning DVD from 2008 makes Disney World look so easy and affordable. Things look much different now. I know prices rise, and I don’t really have a problem with that. But a few things mentioned in this clip that aren’t there now makes the price harder to swallow…

Nick Chappell

751,438 次观看 • 20 天前

Karpathy's prediction about RL is coming true now! He called reward functions unreliable and argued that a single reward number is too low-dimensional to teach an agent what "good" means for complex tasks. To solve this, Agents need a knowledge-guided review as a higher-dimensional feedback channel. Every major AI lab trains models with RL today (OpenAI, Anthropic, DeepSeek). And their key bottleneck has always been the reward functions. GRPO by DeepSeek worked well for math and code because the environment gave a binary signal. But for real agent tasks, someone still has to hand-code the scoring function. That takes days and breaks every time the pipeline changes. RULER (implemented in OpenPipe ART, 10k stars) addresses the exact problem Karpathy identified. The reward criteria are defined in plain English, and an LLM evaluates each trajectory against that description to provide feedback for training. I trained a Qwen3 1.4B agent that plays 2048 using GRPO with this exact workflow. In this case, the agent saw the board, picked a direction, and RULER evaluated the outcome, all from this natural language definition. You can see the full implementation on GitHub and try it yourself. Here's the ART Repo: (don't forget to star it ⭐ ) Just like RLHF replaced manual rankings and GRPO replaced the critic model, natural language rewards are replacing hand-coded scoring functions. RL reward engineering is now prompt engineering. I wrote a full walkthrough covering RL for LLM agents, from RLHF to GRPO to RULER, in the article below.

Karpathy's prediction about RL is coming true now! He called reward functions unreliable and argued that a single reward number is too low-dimensional to teach an agent what "good" means for complex tasks. To solve this, Agents need a knowledge-guided review as a higher-dimensional feedback channel. Every major AI lab trains models with RL today (OpenAI, Anthropic, DeepSeek). And their key bottleneck has always been the reward functions. GRPO by DeepSeek worked well for math and code because the environment gave a binary signal. But for real agent tasks, someone still has to hand-code the scoring function. That takes days and breaks every time the pipeline changes. RULER (implemented in OpenPipe ART, 10k stars) addresses the exact problem Karpathy identified. The reward criteria are defined in plain English, and an LLM evaluates each trajectory against that description to provide feedback for training. I trained a Qwen3 1.4B agent that plays 2048 using GRPO with this exact workflow. In this case, the agent saw the board, picked a direction, and RULER evaluated the outcome, all from this natural language definition. You can see the full implementation on GitHub and try it yourself. Here's the ART Repo: (don't forget to star it ⭐ ) Just like RLHF replaced manual rankings and GRPO replaced the critic model, natural language rewards are replacing hand-coded scoring functions. RL reward engineering is now prompt engineering. I wrote a full walkthrough covering RL for LLM agents, from RLHF to GRPO to RULER, in the article below.

Avi Chawla

349,743 次观看 • 1 个月前

Chuck Schumer brands the SAVE Act 'Jim Crow 2.0' — implying Black Americans don't know how to get a simple legal ID. Dave Rubin nails it: "Do you see what these people are doing? Chuck Schumer's a racist, activist, and buffoon."

Chuck Schumer brands the SAVE Act 'Jim Crow 2.0' — implying Black Americans don't know how to get a simple legal ID. Dave Rubin nails it: "Do you see what these people are doing? Chuck Schumer's a racist, activist, and buffoon."

The Rubin Report

88,228 次观看 • 5 个月前

These people keep preying on the desperation and naivety of Nigerians. Egungun, I don't want to believe you actually know exactly what is going on here, but regardless, it is your responsibility to do your due diligence before advertising anything or anyone to your audience. Different tactics, but the same old format. Don't fall for the hype and gimmicks.

These people keep preying on the desperation and naivety of Nigerians. Egungun, I don't want to believe you actually know exactly what is going on here, but regardless, it is your responsibility to do your due diligence before advertising anything or anyone to your audience. Different tactics, but the same old format. Don't fall for the hype and gimmicks.

Jaro4x

404,319 次观看 • 5 个月前

This is what you don't know sir... Chance and Brandon go way way back and are actually very close friends.. I don't have to prove anything but had a little extra time today.. 🤦🏽‍♀️ You might actually recognize a few young ballers in the pic..💪🏽 #familyforlife #bloodbrothers

This is what you don't know sir... Chance and Brandon go way way back and are actually very close friends.. I don't have to prove anything but had a little extra time today.. 🤦🏽‍♀️ You might actually recognize a few young ballers in the pic..💪🏽 #familyforlife #bloodbrothers

Demetria Jennings

20,080 次观看 • 1 年前