Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Is pre-training dead? OpenAI Chief Research Officer Mark Chen doesn't think so: "We've always found some kind of technique whether it be better engineering or some new research insight that helps you break past the boundary."

Latent.Space

31,952 subscribers

37,431 Aufrufe • vor 2 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

3 Kommentare

Profilbild von Monk

Monkvor 2 Monaten

@OpenAI @markchen90 Pre-training is far from done — scaling laws still have room to run.

Profilbild von Dipanshu Kushwaha

Dipanshu Kushwahavor 2 Monaten

@OpenAI @markchen90 Pre-training is definitely not dead. It's exciting to see how they keep finding ways to improve it.

Profilbild von Eggs inSpace

Eggs inSpacevor 2 Monaten

@OpenAI @markchen90 Agreed. I think people call the wall too early. Usually someone improves the pipeline and the ceiling moves again.

Ähnliche Videos

OpenAI Chief Research Officer, Mark Chen "We've found a new paradigm through Reasoning, which we're also scaling" - GPT-5 could be a significant advancement because we're scaling models not just through unsupervised learning (like with GPT-4.5), but also through a new "reasoning paradigm" - GPT-5 maybe a combination of these approaches

OpenAI Chief Research Officer, Mark Chen "We've found a new paradigm through Reasoning, which we're also scaling" - GPT-5 could be a significant advancement because we're scaling models not just through unsupervised learning (like with GPT-4.5), but also through a new "reasoning paradigm" - GPT-5 maybe a combination of these approaches

Haider.

153,382 Aufrufe • vor 1 Jahr

Ilya Sutskever says the AI scaling era is running out of road because pre-training is about to hit a finite-data wall. "At some point though, pre-training will run out of data. The data is very clearly finite." "Then, okay, what do you do next? Either you do some kind of a souped-up pre-training... or you're doing RL, or maybe something else." "In some sense, we are back to the age of research." His timeline is the part most AI investors and founders should hear: "From 2012 to 2020, it was the age of research. Now from 2020 to 2025, it was the age of scaling... But now the scale is so big." "Is the belief really that... if you had 100x more everything would be so different?" "I don't think that's true. So it's back to the age of research again, just with big computers."

Ilya Sutskever says the AI scaling era is running out of road because pre-training is about to hit a finite-data wall. "At some point though, pre-training will run out of data. The data is very clearly finite." "Then, okay, what do you do next? Either you do some kind of a souped-up pre-training... or you're doing RL, or maybe something else." "In some sense, we are back to the age of research." His timeline is the part most AI investors and founders should hear: "From 2012 to 2020, it was the age of research. Now from 2020 to 2025, it was the age of scaling... But now the scale is so big." "Is the belief really that... if you had 100x more everything would be so different?" "I don't think that's true. So it's back to the age of research again, just with big computers."

Karl Mehta

200,003 Aufrufe • vor 3 Monaten

Mark Chen dropping some alpha while cooking up a tasty meal: "AGI is coming soon." "We're getting closer and closer to a world where the models can come up with more of the innovations on their own, kind of do self-sustained research." "Are there big bets before that point in time? I think the window is small."

Mark Chen dropping some alpha while cooking up a tasty meal: "AGI is coming soon." "We're getting closer and closer to a world where the models can come up with more of the innovations on their own, kind of do self-sustained research." "Are there big bets before that point in time? I think the window is small."

prinz

28,060 Aufrufe • vor 2 Monaten

In this episode, OpenAI Chief Research Officer Mark Chen joins allen to flambé shrimp, cook Korean stew, and chat about being at the frontier of AI research: why scaling laws and pre-training still matter, how OpenAI chooses research bets and allocates compute, what it means to develop research taste, why evals are in crisis, how to avoid benchmark-maxing, and what it will take for models to handle long-horizon real-world work, multimodal reasoning, and eventually end-to-end AI research. Timestamps: 0:00 Intro 0:28 The Soup Story 1:52 From Trading to AI Research 3:21 How to Develop Research Taste 5:23 RL, Evals, and Superhuman Benchmarks 8:17 Cooking Begins on the Impulse Stove 8:53 Scaling Laws, Pre-Training, and Reasoning 12:33 OpenAI’s Research Roadmap and Compute Allocation 15:48 What Makes a Great Researcher 19:33 The Evals Crisis and Benchmark-Maxing 24:34 Jagged Intelligence, Context, and Long-Horizon Learning 27:14 Shrimp Flambé and New Research Bets 31:32 Multimodal Models and One Architecture 32:36 Vibe Researching and End-to-End AI Research 34:36 Failed Bets, Postmortems, and OpenAI’s Alpha 37:07 Final Taste Test 37:53 Overrated vs. Underrated AI Research 41:00 Closing

In this episode, OpenAI Chief Research Officer Mark Chen joins allen to flambé shrimp, cook Korean stew, and chat about being at the frontier of AI research: why scaling laws and pre-training still matter, how OpenAI chooses research bets and allocates compute, what it means to develop research taste, why evals are in crisis, how to avoid benchmark-maxing, and what it will take for models to handle long-horizon real-world work, multimodal reasoning, and eventually end-to-end AI research. Timestamps: 0:00 Intro 0:28 The Soup Story 1:52 From Trading to AI Research 3:21 How to Develop Research Taste 5:23 RL, Evals, and Superhuman Benchmarks 8:17 Cooking Begins on the Impulse Stove 8:53 Scaling Laws, Pre-Training, and Reasoning 12:33 OpenAI’s Research Roadmap and Compute Allocation 15:48 What Makes a Great Researcher 19:33 The Evals Crisis and Benchmark-Maxing 24:34 Jagged Intelligence, Context, and Long-Horizon Learning 27:14 Shrimp Flambé and New Research Bets 31:32 Multimodal Models and One Architecture 32:36 Vibe Researching and End-to-End AI Research 34:36 Failed Bets, Postmortems, and OpenAI’s Alpha 37:07 Final Taste Test 37:53 Overrated vs. Underrated AI Research 41:00 Closing

Latent.Space

62,796 Aufrufe • vor 2 Monaten

DANA BASH: “You want to end this gain-of-function research. Some scientists say, isn’t it better just to put better guardrails around it?” SECRETARY KENNEDY: “People have been trying to put guardrails around it for generations, and it hasn’t worked.” “And clearly, the COVID-19 pandemic and many other global pandemics that we have, have come from gain-of-function research.” “The RSV epidemic that we have, that is the biggest killer of kids in this country today came from these kind of experiments.” “The Lyme disease almost certainly came from this kind of research.” “And the scientists, who you say are ‘experts,’ are making the argument that this is necessary for pandemic response, cannot point to one instance in history where research that came out of this kind of experimentation has actually provided us something beneficial.”

DANA BASH: “You want to end this gain-of-function research. Some scientists say, isn’t it better just to put better guardrails around it?” SECRETARY KENNEDY: “People have been trying to put guardrails around it for generations, and it hasn’t worked.” “And clearly, the COVID-19 pandemic and many other global pandemics that we have, have come from gain-of-function research.” “The RSV epidemic that we have, that is the biggest killer of kids in this country today came from these kind of experiments.” “The Lyme disease almost certainly came from this kind of research.” “And the scientists, who you say are ‘experts,’ are making the argument that this is necessary for pandemic response, cannot point to one instance in history where research that came out of this kind of experimentation has actually provided us something beneficial.”

Overton

158,411 Aufrufe • vor 1 Monat

Trumpettes founder Toni Holt Kramer said “we need a change” from politics and so she will begin discussing “the new rage” of “mature women with younger men.” Toni: “I think that’s kind of interesting. I’ve been doing some research on it.”

Trumpettes founder Toni Holt Kramer said “we need a change” from politics and so she will begin discussing “the new rage” of “mature women with younger men.” Toni: “I think that’s kind of interesting. I’ve been doing some research on it.”

PatriotTakes 🇺🇸

361,072 Aufrufe • vor 9 Monaten

Training at long muscle lengths grows muscle better than training at short muscle lengths 💪 New research breakdown! RIP Dan Rowden.joelseedman_ahp 🤣🤣🤣🤣🤣🤣 We have substantial research demonstrating that full range of motion (ROM) training produces better hypertrophy & strength than partial range of motion training but that has only been shown when the partial reps are when the muscle is in a shortened position (PMID: 34170576) A new meta-analysis by @bradschoenfeldphd Dan Rowden__pak @wolfcoach_ @bretcontreras1 Menno.henselmans Dan Rowdenmikeroberts compared training at long muscle lengths vs training at short muscle lengths & found that training at long muscle lengths produced more hypertrophy than training at shortened muscle lengths This body of literature demonstrating that to maximize muscular hypertrophy it is important to train muscles in a lengthened position. Some have postulated that lengthened partials may produce more hypertrophy than full ROM training but thus far the research shows them to be equivalent when volume & intensity are matched Anyone wanna take a bet on if Dan Rowden.joelseedman_ahp adjusts his recommendations to be in line with the literature? Spoiler alert… he won’t Share & save. Only YOU can stop the spread of BS 🔥

Training at long muscle lengths grows muscle better than training at short muscle lengths 💪 New research breakdown! RIP Dan Rowden.joelseedman_ahp 🤣🤣🤣🤣🤣🤣 We have substantial research demonstrating that full range of motion (ROM) training produces better hypertrophy & strength than partial range of motion training but that has only been shown when the partial reps are when the muscle is in a shortened position (PMID: 34170576) A new meta-analysis by @bradschoenfeldphd Dan Rowden__pak @wolfcoach_ @bretcontreras1 Menno.henselmans Dan Rowdenmikeroberts compared training at long muscle lengths vs training at short muscle lengths & found that training at long muscle lengths produced more hypertrophy than training at shortened muscle lengths This body of literature demonstrating that to maximize muscular hypertrophy it is important to train muscles in a lengthened position. Some have postulated that lengthened partials may produce more hypertrophy than full ROM training but thus far the research shows them to be equivalent when volume & intensity are matched Anyone wanna take a bet on if Dan Rowden.joelseedman_ahp adjusts his recommendations to be in line with the literature? Spoiler alert… he won’t Share & save. Only YOU can stop the spread of BS 🔥

Layne Norton, PhD

52,435 Aufrufe • vor 2 Jahren

🚨🚨🚨What happens when you start doing some research and going down “the rabbit hole”? You start realizing really quickly that A LOT of things don’t make any sense!!🧐 “For example, I realized that governments have captured PATENTS like FREE ELECTRICITY, so that people can’t reproduce it!! Then you start thinking, ‘well what else is going on?’ IT’S NEVER ENDING!! This is why they don’t want people to do research”😳 “Once you realize the fence that’s built around you everywhere, you truly start to realize that this is a PRISON PLANET” Are we already living like, free-range humans? Could this be a “Prison Planet”? PLEASE, do some research and think for yourself!!! Something is clearly NOT RIGHT!!! #GlobalistsVsHumanity #THINK

🚨🚨🚨What happens when you start doing some research and going down “the rabbit hole”? You start realizing really quickly that A LOT of things don’t make any sense!!🧐 “For example, I realized that governments have captured PATENTS like FREE ELECTRICITY, so that people can’t reproduce it!! Then you start thinking, ‘well what else is going on?’ IT’S NEVER ENDING!! This is why they don’t want people to do research”😳 “Once you realize the fence that’s built around you everywhere, you truly start to realize that this is a PRISON PLANET” Are we already living like, free-range humans? Could this be a “Prison Planet”? PLEASE, do some research and think for yourself!!! Something is clearly NOT RIGHT!!! #GlobalistsVsHumanity #THINK

Kevin - WE THE PEOPLE🦁

874,603 Aufrufe • vor 2 Jahren

OpenAI research scientist Aidan McLaughlin argues if you care about AI alignment you should join labs, not METR or Epoch: Theo: "One take I hear a lot is people who are interested in AI and concerned about making AI go well should not work at labs. Instead they should work at independent verification auditing orgs like METR, Avery, Epoch. Presumably you think that more people should work in labs?" Aidan: "In order for people to love the next version of Codex, it has to be aligned. There's immense economic pressure for these things to be aligned, for them to not deceive us, for them to intuit what you mean without you having to say it, which I hope has some really helpful generalization as capabilities get better." "What I do is increasingly alignment research. If you care about things going well, it's an interesting trend that labs are doing increasingly more alignment research. Maybe you should go to the place where the rate of alignment research is increasing." OpenAI

OpenAI research scientist Aidan McLaughlin argues if you care about AI alignment you should join labs, not METR or Epoch: Theo: "One take I hear a lot is people who are interested in AI and concerned about making AI go well should not work at labs. Instead they should work at independent verification auditing orgs like METR, Avery, Epoch. Presumably you think that more people should work in labs?" Aidan: "In order for people to love the next version of Codex, it has to be aligned. There's immense economic pressure for these things to be aligned, for them to not deceive us, for them to intuit what you mean without you having to say it, which I hope has some really helpful generalization as capabilities get better." "What I do is increasingly alignment research. If you care about things going well, it's an interesting trend that labs are doing increasingly more alignment research. Maybe you should go to the place where the rate of alignment research is increasing." OpenAI

MTS

14,978 Aufrufe • vor 1 Monat

I like to think that this is just a matter of opinion but this guy seems like he's doing some research for this. All I know is I drive a Wrangler so I'm the best of the worst or the worst of the best. I'm not sure which. Let me know what you think of this list. 🤔🤣🤣🤣

I like to think that this is just a matter of opinion but this guy seems like he's doing some research for this. All I know is I drive a Wrangler so I'm the best of the worst or the worst of the best. I'm not sure which. Let me know what you think of this list. 🤔🤣🤣🤣

Johnny B. Good

70,326 Aufrufe • vor 2 Monaten

RFK Jr. gave a calm, straightforward answer to a mom on Dr. Phil. When asked what he’d tell a new mom about vaccines, he said: Do your own research. You research the stroller. You research the food. You should research the medicine too. It’s your responsibility as a parent in a democracy. This one hit me. In an era where trust in institutions is low and information feels overwhelming, his point feels grounded, parents ultimately carry the weight of these decisions. Whether you’re pro-vaccine, cautious, or somewhere in between, encouraging informed consent and personal responsibility is something we should all be able to agree on. Blind trust or blind rejection both carry risks. What’s your take? Do you think parents should do deep independent research on vaccines, or is it better to mostly trust the standard medical guidance?

RFK Jr. gave a calm, straightforward answer to a mom on Dr. Phil. When asked what he’d tell a new mom about vaccines, he said: Do your own research. You research the stroller. You research the food. You should research the medicine too. It’s your responsibility as a parent in a democracy. This one hit me. In an era where trust in institutions is low and information feels overwhelming, his point feels grounded, parents ultimately carry the weight of these decisions. Whether you’re pro-vaccine, cautious, or somewhere in between, encouraging informed consent and personal responsibility is something we should all be able to agree on. Blind trust or blind rejection both carry risks. What’s your take? Do you think parents should do deep independent research on vaccines, or is it better to mostly trust the standard medical guidance?

Camus

54,769 Aufrufe • vor 4 Monaten

Mikel Arteta on whether fans should be concerned by pre-season defeats: "No, I mean, I think it has to hurt when you lose a football match. It has to hurt whether it is a friendly match, a test or whatever you want to call it. But, yeah, we know where we are, we know that we are missing still a lot of players and some players that haven't had enough training. But, yeah, sometimes it's good."

Mikel Arteta on whether fans should be concerned by pre-season defeats: "No, I mean, I think it has to hurt when you lose a football match. It has to hurt whether it is a friendly match, a test or whatever you want to call it. But, yeah, we know where we are, we know that we are missing still a lot of players and some players that haven't had enough training. But, yeah, sometimes it's good."

Connor Humm

119,325 Aufrufe • vor 1 Monat

Love how nxthompson homes in on the most important implication of my Fast16 story ( "It was like epistemological warfare. It was affecting belief of the scientists so they would think it’s not working. This is...pretty important when you think about the future. Imagine hacks of AI systems that we trust so much, that could...tell us that research that is actually good, is bad, and throw us off our work. That is some crazy software."

Love how nxthompson homes in on the most important implication of my Fast16 story ( "It was like epistemological warfare. It was affecting belief of the scientists so they would think it’s not working. This is...pretty important when you think about the future. Imagine hacks of AI systems that we trust so much, that could...tell us that research that is actually good, is bad, and throw us off our work. That is some crazy software."

Kim Zetter

25,253 Aufrufe • vor 4 Monaten

ELON: GROK IS THE BEST ASSISTANT TO FIGHT PROPAGANDA “You just press the Grok icon on any 𝕏 post and it will analyze it for you and research it as much as you want. So, just by tapping the Grok icon, you can assess whether that post is the truth, nothing but the truth, or whether there's something supplemental that needs to be explained. So, I think we've made a lot of progress towards freedom of speech and people being able to tell whether something is false or not false, or propaganda. The recent update to Grok is actually, I think, very good at piercing through propaganda. And then, we used that latest version of Grok to create Grokipedia. It's not just, I think, more neutral and more accurate than Wikipedia, but it actually has a lot more information than a Wikipedia page.” Source: All-In Podcast The All-In Podcast, October 31, 2025

ELON: GROK IS THE BEST ASSISTANT TO FIGHT PROPAGANDA “You just press the Grok icon on any 𝕏 post and it will analyze it for you and research it as much as you want. So, just by tapping the Grok icon, you can assess whether that post is the truth, nothing but the truth, or whether there's something supplemental that needs to be explained. So, I think we've made a lot of progress towards freedom of speech and people being able to tell whether something is false or not false, or propaganda. The recent update to Grok is actually, I think, very good at piercing through propaganda. And then, we used that latest version of Grok to create Grokipedia. It's not just, I think, more neutral and more accurate than Wikipedia, but it actually has a lot more information than a Wikipedia page.” Source: All-In Podcast The All-In Podcast, October 31, 2025

Mario Nawfal

1,641,293 Aufrufe • vor 10 Monaten

I’ve always been so amazed when I see clips like this. They remind me of a real life Karate Kid. While I have trained when I was younger up until last year, I’ve always admired how strong the will of people can be. This woman enlisted in a training program at a Shaolin temple, this is as tough and as real as it gets, each crushing blow is felt, the intense training is what keeps them whole and unaffected. This is the kind of training that turns people into warriors. The age old traditional ways have molded some of the fiercest fighters of the world, could you withstand this training? I guess the better question would be, could you handle this kind of backbreaking training?

I’ve always been so amazed when I see clips like this. They remind me of a real life Karate Kid. While I have trained when I was younger up until last year, I’ve always admired how strong the will of people can be. This woman enlisted in a training program at a Shaolin temple, this is as tough and as real as it gets, each crushing blow is felt, the intense training is what keeps them whole and unaffected. This is the kind of training that turns people into warriors. The age old traditional ways have molded some of the fiercest fighters of the world, could you withstand this training? I guess the better question would be, could you handle this kind of backbreaking training?

SonnyBoy🇺🇸

57,203 Aufrufe • vor 4 Monaten