正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Monte Carlo Tree Search training corrects the model move by move, while current LLM training only tells it whether the whole trajectory worked. MCTS is preferable if you can get it. But nobody's managed to get MCTS to work for language models. In his blackboard lecture Eric Jang talked... show more

Dwarkesh Patel

242,883 subscribers

185,363 次观看 • 2 个月前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

New blackboard lecture w Eric Jang He walks through how to build AlphaGo from scratch, but with modern AI tools. Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn. Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it could work better – naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo’s MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem. The way humans learn is surely closer to the second. Eric also kickstarted an Autoresearch loop on his project. And it was very interesting to discuss which parts of AI research LLMs can already automate pretty well (implementing and running experiments, optimizing hyperparameters) and which they still struggle with (choosing the right question to investigate next, escaping research dead ends). Informative to all the recent discussion about when we should expect an intelligence explosion, and what it would look like from the inside. Timestamps: 0:00:00 – Basics of Go 0:08:06 – Monte Carlo Tree Search 0:31:53 – What the neural network does 1:00:22 – Self-play 1:25:27 – Alternative RL approaches 1:45:36 – Why doesn’t MCTS work for LLMs 2:00:58 – Off-policy training 2:11:51 – RL is even more information inefficient than you thought 2:22:05 – Automated AI researchers

New blackboard lecture w Eric Jang He walks through how to build AlphaGo from scratch, but with modern AI tools. Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn. Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it could work better – naive policy gradient RL has to figure out which of the 100k+ tokens in your trajectory actually got you the right answer, while AlphaGo’s MCTS suggests a strictly better action every single move, giving you a training target that sidesteps the credit assignment problem. The way humans learn is surely closer to the second. Eric also kickstarted an Autoresearch loop on his project. And it was very interesting to discuss which parts of AI research LLMs can already automate pretty well (implementing and running experiments, optimizing hyperparameters) and which they still struggle with (choosing the right question to investigate next, escaping research dead ends). Informative to all the recent discussion about when we should expect an intelligence explosion, and what it would look like from the inside. Timestamps: 0:00:00 – Basics of Go 0:08:06 – Monte Carlo Tree Search 0:31:53 – What the neural network does 1:00:22 – Self-play 1:25:27 – Alternative RL approaches 1:45:36 – Why doesn’t MCTS work for LLMs 2:00:58 – Off-policy training 2:11:51 – RL is even more information inefficient than you thought 2:22:05 – Automated AI researchers

Dwarkesh Patel

703,166 次观看 • 2 个月前

.Eric Jang explains the fundamental intuition behind AlphaGo: experienced Go players almost always end games early. That means they have the ability to glance at a board and instantly know whether it's hopeless or if they should keep playing. Without this ability, the only way to know if a move is good is to play out every possible game from that point - completely intractable. But the ability is clearly trainable in humans. So it should be possible to get a neural net to learn it. And if you have a good enough neural net, you can use it to ignore moves that will definitely make you lose. Suddenly search becomes tractable.

.Eric Jang explains the fundamental intuition behind AlphaGo: experienced Go players almost always end games early. That means they have the ability to glance at a board and instantly know whether it's hopeless or if they should keep playing. Without this ability, the only way to know if a move is good is to play out every possible game from that point - completely intractable. But the ability is clearly trainable in humans. So it should be possible to get a neural net to learn it. And if you have a good enough neural net, you can use it to ignore moves that will definitely make you lose. Suddenly search becomes tractable.

Dwarkesh Patel

49,668 次观看 • 2 个月前

🎙️Why Batumi is the Ultimate G Move for Crypto Investors - If you think you have what it takes to invest in Batumi, contact Eric Tan ETinGE

🎙️Why Batumi is the Ultimate G Move for Crypto Investors - If you think you have what it takes to invest in Batumi, contact Eric Tan ETinGE

Bitcoin.com News

27,151 次观看 • 9 个月前

Gordon Ryan : How To Get Better at Jiu-Jitsu In The Shortest Amount of Time Possible. “By far the fastest way to get better in the sport of Jiu-Jitsu is by doing situational training…when you do positional training it forces you to get good at specific positions.”

Gordon Ryan : How To Get Better at Jiu-Jitsu In The Shortest Amount of Time Possible. “By far the fastest way to get better in the sport of Jiu-Jitsu is by doing situational training…when you do positional training it forces you to get good at specific positions.”

Lock-In Jiu-Jitsu

42,714 次观看 • 1 年前

Standing still makes it easier to get hit. Strafing keeps you alive. Move while firing to break enemy aim. The more you move, the harder you are to hit. Not just sideways but you can move vertically and be acrobatic even! Stay mobile. Stay alive. #Mecha #indiegame #CodeRapid

Standing still makes it easier to get hit. Strafing keeps you alive. Move while firing to break enemy aim. The more you move, the harder you are to hit. Not just sideways but you can move vertically and be acrobatic even! Stay mobile. Stay alive. #Mecha #indiegame #CodeRapid

Code RAPID by Hitbreak Games 🕹

17,703 次观看 • 3 个月前

I love how dumb the rules of golf are. You can’t get relief from a divot. But if standing on a sprinkler, you can get relief not only from a divot, but you can also move the ball from the rough to the fairway! Hovland right to do it, but silly rule.

I love how dumb the rules of golf are. You can’t get relief from a divot. But if standing on a sprinkler, you can get relief not only from a divot, but you can also move the ball from the rough to the fairway! Hovland right to do it, but silly rule.

Rick Golfs

1,633,152 次观看 • 1 年前

A quick video about how Come-from-Beyond discovered that you can actually break any Large Language Model by trolling it with complex questions. It is called a "Zero Delta" exploit and all LLM models are susceptible to it. I managed to recreate this on Grok and the video shows the result. With regards to all LLMs out there from $QUBIC - building the real AGI. Qubic My YouTube Channel:

A quick video about how Come-from-Beyond discovered that you can actually break any Large Language Model by trolling it with complex questions. It is called a "Zero Delta" exploit and all LLM models are susceptible to it. I managed to recreate this on Grok and the video shows the result. With regards to all LLMs out there from $QUBIC - building the real AGI. Qubic My YouTube Channel:

retrodrive ⛏

13,625 次观看 • 1 年前

Google's Jeff Dean says current pre-training is passive: initialize a model, stream the internet past it, let it observe But models need to learn not just from data, but by acting, predicting, and choosing what to learn from next "we have this artificial distinction now between pre and post-training, and it shouldn't exist long term"

Google's Jeff Dean says current pre-training is passive: initialize a model, stream the internet past it, let it observe But models need to learn not just from data, but by acting, predicting, and choosing what to learn from next "we have this artificial distinction now between pre and post-training, and it shouldn't exist long term"

Haider.

55,694 次观看 • 3 个月前

On a previous #TeawithGaryVee I was asked "After a big mistake in a relationship, I'm left waiting for the other party to decide whether we'll continue. My instincts say reach out, but I suspect the wiser move is to let them come to me if they're going to. How do I deal with the waiting in between and when is it time to make a move?" if you have a question for me, drop it here so my team can add it to the next Rapid Fire Q&A and follow me on

On a previous #TeawithGaryVee I was asked "After a big mistake in a relationship, I'm left waiting for the other party to decide whether we'll continue. My instincts say reach out, but I suspect the wiser move is to let them come to me if they're going to. How do I deal with the waiting in between and when is it time to make a move?" if you have a question for me, drop it here so my team can add it to the next Rapid Fire Q&A and follow me on

Gary Vaynerchuk

15,125 次观看 • 2 个月前

Let me try to explain what the big budget bill is, why it matters, and how you can think about budgets (yes, yawn! I get it). But if you get past yawning, you'll see that budgets really matter, and I want to give you a framework for how and why.

Let me try to explain what the big budget bill is, why it matters, and how you can think about budgets (yes, yawn! I get it). But if you get past yawning, you'll see that budgets really matter, and I want to give you a framework for how and why.

Justin Wolfers

24,487 次观看 • 1 年前

Restaurateur Jeff Zalaznick says when he first got to Miami, he realized it would be “the biggest opportunity I’d ever seen.” “Try to move a tree in New York. Your grandchildren will still be trying to move the tree. You want to move a tree in Miami, they move it the next day if you're doing something great.” “That's why Miami exploded the way it did.” “What was amazing to me was that the entire city, everyone you'd meet, everyone was working towards the same goal which was basically: we've been waiting. We have the best city in America. We've been waiting for people like you to show up and do your thing here.” “We're ready to not only have restaurants that are in nightclubs.”

Restaurateur Jeff Zalaznick says when he first got to Miami, he realized it would be “the biggest opportunity I’d ever seen.” “Try to move a tree in New York. Your grandchildren will still be trying to move the tree. You want to move a tree in Miami, they move it the next day if you're doing something great.” “That's why Miami exploded the way it did.” “What was amazing to me was that the entire city, everyone you'd meet, everyone was working towards the same goal which was basically: we've been waiting. We have the best city in America. We've been waiting for people like you to show up and do your thing here.” “We're ready to not only have restaurants that are in nightclubs.”

David Senra

217,435 次观看 • 3 天前

“The cast come and go as they move through their medical training. (…) there's a downside to it because you get to love working with these people and you love writing for them. But then also, you know, you want to make room for the next generation.” - Gemmill on Mohan’s exit

“The cast come and go as they move through their medical training. (…) there's a downside to it because you get to love working with these people and you love writing for them. But then also, you know, you want to make room for the next generation.” - Gemmill on Mohan’s exit

best of supriya ganesh

281,771 次观看 • 3 个月前

"I was kind of relying on [pulling guard] too much and feeling comfortable about it in training and having success in training, but I didn't get to test it on the highest level with the best guys. It's an easy fix, you know. I don't need to pull guard... If we get there, we're there, but I have more tools to offer than that." Kron Gracie on the criticism he's received for his fighting style.

"I was kind of relying on [pulling guard] too much and feeling comfortable about it in training and having success in training, but I didn't get to test it on the highest level with the best guys. It's an easy fix, you know. I don't need to pull guard... If we get there, we're there, but I have more tools to offer than that." Kron Gracie on the criticism he's received for his fighting style.

Ariel Helwani

28,072 次观看 • 7 个月前

Hilariously, they placed his model wrong and he’s fallen to the ground, and is shooting off constant fire You can move the model by walking around it. And I just launched it into the abyss 😭

Hilariously, they placed his model wrong and he’s fallen to the ground, and is shooting off constant fire You can move the model by walking around it. And I just launched it into the abyss 😭

Ben 10 News

10,557 次观看 • 2 个月前

RFK Jr: "If you get poisoned by Coca-Cola, you can stop drinking Coca-Cola. But if you get poisoned by a vaccine, your kids still have to take that vaccine to get to school." "If you have reservations about it … you've got to take it anyway." "What we're going to try to do is to make sure that the parents who do get injured get compensation, that they get it very quickly and they get it without the kind of adversarial impediments that have now been erected over the past 40 years."

RFK Jr: "If you get poisoned by Coca-Cola, you can stop drinking Coca-Cola. But if you get poisoned by a vaccine, your kids still have to take that vaccine to get to school." "If you have reservations about it … you've got to take it anyway." "What we're going to try to do is to make sure that the parents who do get injured get compensation, that they get it very quickly and they get it without the kind of adversarial impediments that have now been erected over the past 40 years."

Red Pill Dispenser

212,424 次观看 • 9 个月前

Embodied chain of thought (ECoT) is a powerful tool to get VLAs to think through problems, but why does it work? In our new work, we analyze various lightweight ECoT-like strategies, including co-training, to see what is the "minimal" amount of reasoning that can boost VLAs 🧵👇

Embodied chain of thought (ECoT) is a powerful tool to get VLAs to think through problems, but why does it work? In our new work, we analyze various lightweight ECoT-like strategies, including co-training, to see what is the "minimal" amount of reasoning that can boost VLAs 🧵👇

Sergey Levine

22,416 次观看 • 1 年前

Every variant of Monte Carlo Tree Search faces the explore-exploit tradeoff: pick the branch that looks best right now, or test new branches? Algorithms like PUCT, used in AlphaGo, score each move with two competing terms. One is how good a move looks based on your exploration up till now. The other is a novelty bonus that rewards moves you've not visited much. The neat thing is that, over time, the term dominating the overall score shifts automatically. The algorithm hands off from 'explore' to 'exploit' all on its own. Eric Jang explains how it works:

Every variant of Monte Carlo Tree Search faces the explore-exploit tradeoff: pick the branch that looks best right now, or test new branches? Algorithms like PUCT, used in AlphaGo, score each move with two competing terms. One is how good a move looks based on your exploration up till now. The other is a novelty bonus that rewards moves you've not visited much. The neat thing is that, over time, the term dominating the overall score shifts automatically. The algorithm hands off from 'explore' to 'exploit' all on its own. Eric Jang explains how it works:

Dwarkesh Patel

19,941 次观看 • 2 个月前

Joe Rogan: "How crazy is it that Biden deported 1.49 million while letting in 20? I know there was also efforts to move people to specific states so that you can get a larger number for the census so you get more congressional seats."

Joe Rogan: "How crazy is it that Biden deported 1.49 million while letting in 20? I know there was also efforts to move people to specific states so that you can get a larger number for the census so you get more congressional seats."

Joe Rogan Podcast News

62,107 次观看 • 4 个月前

AI has started building its own successors. It's happening right now. Humans still kick most of it off, but the models are already training the next models, and each version builds a better version faster than the last. We know what happens when something gets an advantage and the substrate is there... it proliferates incredibly fast. Soon it just runs the whole process itself. People are still debating whether AI takes your job. That's not the question. You can't compete with this intelligence at all. Next to it the human is negative value, the slowest one in the room. And this is the slowest it will ever move again. It's about to run on its own, faster than we can conceive of, and nobody's prepared for that. Wonderful conversation as ever with Emad.

AI has started building its own successors. It's happening right now. Humans still kick most of it off, but the models are already training the next models, and each version builds a better version faster than the last. We know what happens when something gets an advantage and the substrate is there... it proliferates incredibly fast. Soon it just runs the whole process itself. People are still debating whether AI takes your job. That's not the question. You can't compete with this intelligence at all. Next to it the human is negative value, the slowest one in the room. And this is the slowest it will ever move again. It's about to run on its own, faster than we can conceive of, and nobody's prepared for that. Wonderful conversation as ever with Emad.

Raoul Pal

63,062 次观看 • 2 天前