正在加载视频...

视频加载失败

返回首页

How does GPT-5 become GPT-6? Everything on the internet was already trained on a year, year and a half ago. New data is being added but it's marginal and a lot of it is AI slop. There is interesting private data (like JP Morgan's data) but nobody is opening...

himanshustwts's profile picture

himanshu

28,170 subscribers

10,684 次观看 • 3 个月前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

The Delta of Intelligence is Human Data! Curtis G. Northcutt is Director of AI Research at Handshake AI. In this episode, Curtis shares his unique perspectives on human data, RL envs, four quadrants of the AI market, journey of Handshake and much more. 0:00:00 - INTRO 0:02:08 - The Snowman Effect: From Rural Kentucky to 7 Years at MIT 0:09:50 - Confident Learning, Two Systems of Intelligence, Classical ML Has to Prevail 0:20:37 - Origin of CleanLab and Handshake acquisition 0:28:00 - How does GPT-5 become GPT-6? 0:31:20 - The Fastest growing Data Lab: What makes Handshake different? 0:35:32 - Difference between Good Data and Bad Data 0:38:17 - New kind of Data, IA and AI 0:42:04 - Scaling Coding Benchmarks and Efficiency in Long Horizon Tasks 0:49:12 - On Pre-Training, Misconception around Human Data 0:57:13 - Open Question: Taste and Personality, Quick Fire 01:05:15 - Advice to 20yo : Skill and Obsession
1:08:18

The Delta of Intelligence is Human Data! Curtis G. Northcutt is Director of AI Research at Handshake AI. In this episode, Curtis shares his unique perspectives on human data, RL envs, four quadrants of the AI market, journey of Handshake and much more. 0:00:00 - INTRO 0:02:08 - The Snowman Effect: From Rural Kentucky to 7 Years at MIT 0:09:50 - Confident Learning, Two Systems of Intelligence, Classical ML Has to Prevail 0:20:37 - Origin of CleanLab and Handshake acquisition 0:28:00 - How does GPT-5 become GPT-6? 0:31:20 - The Fastest growing Data Lab: What makes Handshake different? 0:35:32 - Difference between Good Data and Bad Data 0:38:17 - New kind of Data, IA and AI 0:42:04 - Scaling Coding Benchmarks and Efficiency in Long Horizon Tasks 0:49:12 - On Pre-Training, Misconception around Human Data 0:57:13 - Open Question: Taste and Personality, Quick Fire 01:05:15 - Advice to 20yo : Skill and Obsession

himanshu

22,264 次观看 • 3 个月前

Ryan believes we could get 3-6 years of AI progress at the current pace, but in a single year, once we automated AI R&D. So a jump as big as GPT 3.5 to Mythos in a single year. A big crux is whether you'd end up bottlenecked by data. It seems to that a big source of AI progress currently is human expert data, codified into RL environments and SFT data. Ryan doesn't think human expert data is a primary driver of AI progress, and this ended up being a fruitful debate during the conversation.
2:29

Ryan believes we could get 3-6 years of AI progress at the current pace, but in a single year, once we automated AI R&D. So a jump as big as GPT 3.5 to Mythos in a single year. A big crux is whether you'd end up bottlenecked by data. It seems to that a big source of AI progress currently is human expert data, codified into RL environments and SFT data. Ryan doesn't think human expert data is a primary driver of AI progress, and this ended up being a fruitful debate during the conversation.

Dwarkesh Patel

91,786 次观看 • 3 天前

This workflow combining Loom’s AI features + a custom ChatGPT GPT is saving me hours. Instead of creating onboarding Docs for new team members, I film a Loom → generate SOP → train a GPT to answer questions Game changer for businesses to delegate faster. Here's how to do it: First, I record a video of whatever task I want to delegate to the new team member with Loom. The more in-depth, the better, but I just used a 7-minute video. Then, I use Loom's new AI 'Write a document' feature. Upgrading to Loom AI from the standard Loom plan cost me $2. Loom AI can generate an entire SOP, PR description, step-by-step guide, QA, and more from a simple Loom video in <5 seconds. In the past, I’ve spent 2 hours+ hand-writing each one of these docs to onboard new team members, so Loom AI is already a massive timesaver. But it gets even better! Next, we can take that data from the SOP document, and we use it as 'Knowledge' to train a Custom GPT that can answer the new team members' questions. The more SOP docs/Knowledge you feed the GPT, the better. But one is fine if that's all you have because the GPT will pull any unknown answers from the web or its training data. Here are the prompt Instructions you want to put into the Custom GPT (copy and paste this): You are an expert CEO, specialized in onboarding and training new team members. Using the Knowledge provided, you will help new team members with any questions or stipulations they may have about their new role. Stick as true to the data provided as possible, but if= they ask any questions that the Knowledge base does not have a specific answer for, you are permitted to use your pre-trained data and/or web browsing capabilities. That's it! It can't replace you entirely, but it'll save you 90% of the time you would've wasted on writing an SOP doc and answering questions. Simple AI workflows here and there really add up. There's also a workflow to help with the job screening process, but I'll save that for another day :^)
0:54

This workflow combining Loom’s AI features + a custom ChatGPT GPT is saving me hours. Instead of creating onboarding Docs for new team members, I film a Loom → generate SOP → train a GPT to answer questions Game changer for businesses to delegate faster. Here's how to do it: First, I record a video of whatever task I want to delegate to the new team member with Loom. The more in-depth, the better, but I just used a 7-minute video. Then, I use Loom's new AI 'Write a document' feature. Upgrading to Loom AI from the standard Loom plan cost me $2. Loom AI can generate an entire SOP, PR description, step-by-step guide, QA, and more from a simple Loom video in <5 seconds. In the past, I’ve spent 2 hours+ hand-writing each one of these docs to onboard new team members, so Loom AI is already a massive timesaver. But it gets even better! Next, we can take that data from the SOP document, and we use it as 'Knowledge' to train a Custom GPT that can answer the new team members' questions. The more SOP docs/Knowledge you feed the GPT, the better. But one is fine if that's all you have because the GPT will pull any unknown answers from the web or its training data. Here are the prompt Instructions you want to put into the Custom GPT (copy and paste this): You are an expert CEO, specialized in onboarding and training new team members. Using the Knowledge provided, you will help new team members with any questions or stipulations they may have about their new role. Stick as true to the data provided as possible, but if= they ask any questions that the Knowledge base does not have a specific answer for, you are permitted to use your pre-trained data and/or web browsing capabilities. That's it! It can't replace you entirely, but it'll save you 90% of the time you would've wasted on writing an SOP doc and answering questions. Simple AI workflows here and there really add up. There's also a workflow to help with the job screening process, but I'll save that for another day :^)

Rowan Cheung

129,424 次观看 • 2 年前

A lot of people shat on GPT-5 when it came out All of those people are wrong. GPT-5 is actually insanely good. It just requires a different style of prompting than any other model If you prompt it correctly, it's the best AI model out there Here's exactly how to do that:
21:52

A lot of people shat on GPT-5 when it came out All of those people are wrong. GPT-5 is actually insanely good. It just requires a different style of prompting than any other model If you prompt it correctly, it's the best AI model out there Here's exactly how to do that:

Alex Finn

19,286 次观看 • 11 个月前

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua
0:42

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua

Chris

250,405 次观看 • 1 个月前

Narrative violation from Dylan on Dwarkesh: H100s are worth *more* today than they were 3 years ago. There's a sentiment that data center buildouts are priced into the risk of rapidly depreciating GPUs. But the models want to learn. Token prices are falling so fast that you can get more intelligence out of GPT-5.4 on a 3-year old H100 than GPT-4 on a brand new H100. Dylan Patel dwarkeshpatel
1:22

Narrative violation from Dylan on Dwarkesh: H100s are worth *more* today than they were 3 years ago. There's a sentiment that data center buildouts are priced into the risk of rapidly depreciating GPUs. But the models want to learn. Token prices are falling so fast that you can get more intelligence out of GPT-5.4 on a 3-year old H100 than GPT-4 on a brand new H100. Dylan Patel dwarkeshpatel

Sarah Wang

159,395 次观看 • 5 个月前

Openai giving $50,000 in FREE API credits and it's hidden in your account settings 😳 most people have no idea this exists what you get every single day: - 250K tokens/day for GPT-5.5 and GPT-5.2 (powerful models) - 2.5M tokens/day for GPT-5.1-Mini and GPT-5-Nano (lighter models) - up to 10M tokens/day if you're on tier 3-5 not a one time lump sum, it resets daily it's called the data sharing program and here's how it works: > go to > click data controls → sharing > opt in to both options > done, free daily tokens activate immediately important note: - your data gets used by openai for training and improvements - don't use it for client work or anything sensitive - need a positive account balance to activate perfect for: - learning and experimenting - side projects and personal builds - testing new models without paying you need a positive account balance to activate it but after that it's free every day save this and take free 50k credits before they close it
0:43

Openai giving $50,000 in FREE API credits and it's hidden in your account settings 😳 most people have no idea this exists what you get every single day: - 250K tokens/day for GPT-5.5 and GPT-5.2 (powerful models) - 2.5M tokens/day for GPT-5.1-Mini and GPT-5-Nano (lighter models) - up to 10M tokens/day if you're on tier 3-5 not a one time lump sum, it resets daily it's called the data sharing program and here's how it works: > go to > click data controls → sharing > opt in to both options > done, free daily tokens activate immediately important note: - your data gets used by openai for training and improvements - don't use it for client work or anything sensitive - need a positive account balance to activate perfect for: - learning and experimenting - side projects and personal builds - testing new models without paying you need a positive account balance to activate it but after that it's free every day save this and take free 50k credits before they close it

Isra

168,671 次观看 • 2 个月前

Elon Musk: The probability of AI going bad is not zero. “I saw GPT-1, GPT-2, GPT-3, GPT-4, the whole lead-up to that. So it was easy for me to kind of see where it's going. If you just extrapolate the points on a curve and assume that trend will continue, then we will have profound artificial intelligence, and obviously at a level that far exceeds human intelligence. So I'm glad to see at this point that people are taking safety seriously. And I do think overall that the potential is there for artificial intelligence, AI, to have most likely a positive effect and to create a future of abundance where there is no scarcity of goods and services. But it is somewhat of the magic genie problem, where if you have a magic genie that can grant all the wishes, usually those stories don't end well. Be careful what you wish for.” In Conversation with Rishi Sunak, November 2, 2023.
1:12

Elon Musk: The probability of AI going bad is not zero. “I saw GPT-1, GPT-2, GPT-3, GPT-4, the whole lead-up to that. So it was easy for me to kind of see where it's going. If you just extrapolate the points on a curve and assume that trend will continue, then we will have profound artificial intelligence, and obviously at a level that far exceeds human intelligence. So I'm glad to see at this point that people are taking safety seriously. And I do think overall that the potential is there for artificial intelligence, AI, to have most likely a positive effect and to create a future of abundance where there is no scarcity of goods and services. But it is somewhat of the magic genie problem, where if you have a magic genie that can grant all the wishes, usually those stories don't end well. Be careful what you wish for.” In Conversation with Rishi Sunak, November 2, 2023.

ELON CLIPS

22,095 次观看 • 1 个月前

Repeating the same experiment but one year later. GPT 5.6 Pro cannot be used vanilla for this prompt because it immediately fires the right CAS command to compute the Galois group. So instead I used a very restrictive prompt. One year has passed since GPT 5 Pro premiere, GPT Pro 5.6 is now able to compute the result from first principles, including the resolvent construction. This is like seeing a model building a whole Galois theory course live. It's so nice to learn from this output, very detailed! Prompt: Compute the Galois of the polynomial x^8-x^7+3*x^6-3*x^5+2*x^4-2*x^3+5*x^2+5*x+1 Don't look up the result in the internet nor any specialized database. Don't use ready made commands from SymPy or any other program. Develop a full resolvent computation. Do it all the way down to the definitions, no excuses, like there is no tomorrow.
1:13

Repeating the same experiment but one year later. GPT 5.6 Pro cannot be used vanilla for this prompt because it immediately fires the right CAS command to compute the Galois group. So instead I used a very restrictive prompt. One year has passed since GPT 5 Pro premiere, GPT Pro 5.6 is now able to compute the result from first principles, including the resolvent construction. This is like seeing a model building a whole Galois theory course live. It's so nice to learn from this output, very detailed! Prompt: Compute the Galois of the polynomial x^8-x^7+3*x^6-3*x^5+2*x^4-2*x^3+5*x^2+5*x+1 Don't look up the result in the internet nor any specialized database. Don't use ready made commands from SymPy or any other program. Develop a full resolvent computation. Do it all the way down to the definitions, no excuses, like there is no tomorrow.

Bartosz Naskręcki

76,288 次观看 • 6 天前

Sam Altman said moving from GPT-5 back to older models feels painful. He described GPT-5 as fluent, deep, and clearly more intelligent—emphasizing that it is a single integrated model, no switching required. He compared it to having experts in every field available anytime. GPT-5 can build full software programs from scratch, write research reports on complex topics, and can even plan real-world events autonomously.
0:50

Sam Altman said moving from GPT-5 back to older models feels painful. He described GPT-5 as fluent, deep, and clearly more intelligent—emphasizing that it is a single integrated model, no switching required. He compared it to having experts in every field available anytime. GPT-5 can build full software programs from scratch, write research reports on complex topics, and can even plan real-world events autonomously.

Wes Roth

219,869 次观看 • 11 个月前

Ex-OpenAI chief research officer Bob McGrew says each AI model generation uses 100x more compute and because it takes time to build new data centers it appears like nothing is happening from the outside, while inside companies the view is very different; and before we see GPT-5 we will see a 10x lift with a half-generation release
1:49

Ex-OpenAI chief research officer Bob McGrew says each AI model generation uses 100x more compute and because it takes time to build new data centers it appears like nothing is happening from the outside, while inside companies the view is very different; and before we see GPT-5 we will see a 10x lift with a half-generation release

Tsarathustra

51,769 次观看 • 1 年前

Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)
1:07

Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)

Rohan Paul

441,195 次观看 • 21 天前

ELON: AI HAS DRAINED HUMAN KNOWLEDGE DRY - NOW DRINKING FROM ITS OWN WELL "You take the entire internet, all books ever written, all the interesting videos, and you distill that down into tokens, essentially bits of information. We've now exhausted all of that; the cumulative sum of human knowledge has been exhausted in AI training. That happened last year. So the only way to supplement that is with synthetic data, where the AI creates. It'll write an essay, or it'll come up with a thesis, and then it will grade itself. It'll go through this process of self-learning with synthetic data." Source: Elon Musk, Mark Penn, January 9th 2025
0:37

ELON: AI HAS DRAINED HUMAN KNOWLEDGE DRY - NOW DRINKING FROM ITS OWN WELL "You take the entire internet, all books ever written, all the interesting videos, and you distill that down into tokens, essentially bits of information. We've now exhausted all of that; the cumulative sum of human knowledge has been exhausted in AI training. That happened last year. So the only way to supplement that is with synthetic data, where the AI creates. It'll write an essay, or it'll come up with a thesis, and then it will grade itself. It'll go through this process of self-learning with synthetic data." Source: Elon Musk, Mark Penn, January 9th 2025

Mario Nawfal

1,938,792 次观看 • 10 个月前

ChatGPT warns before opening 𝕏.com, but not before Facebook, Instagram, LinkedIn, Reddit, or YouTube. • 𝕏.com is the 6th most visited site in the world. • OpenAI even trained GPT on Twitter data illegally. Now that 𝕏 is a competitor, they make it harder for users to visit.
0:26

ChatGPT warns before opening 𝕏.com, but not before Facebook, Instagram, LinkedIn, Reddit, or YouTube. • 𝕏.com is the 6th most visited site in the world. • OpenAI even trained GPT on Twitter data illegally. Now that 𝕏 is a competitor, they make it harder for users to visit.

DogeDesigner

929,230 次观看 • 1 年前

Anyone sharing age demographic data from Thomas Massie’s primary last is promoting the “GOY-SLOP”narrative. These is zero accurate information coming directly from the tabulator. The information you are being fed comes exit polling. Stop promoting their slop narrative that some how Ed Gallrein beat Thomas Masse because a massive about of boomers who watch Fox News showed up and voted for him.
3:49

Anyone sharing age demographic data from Thomas Massie’s primary last is promoting the “GOY-SLOP”narrative. These is zero accurate information coming directly from the tabulator. The information you are being fed comes exit polling. Stop promoting their slop narrative that some how Ed Gallrein beat Thomas Masse because a massive about of boomers who watch Fox News showed up and voted for him.

RyanMatta 🇺🇸 🦅

80,792 次观看 • 2 个月前

Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)
1:00

Over the last few months, we’ve been thinking about how to learn from “off-domain” data - data from non-robot sources like video or simulation. These data sources are not quite good enough to learn policies (even monolithic VLA models) directly, but they still contain lots of information that can be useful for generalizable robot control. How can we develop robot learning models that are able to make use of this type of data for generalizable control? In new work, that we call HAMSTER, we show that VLMs can be useful for enabling robotic learning from off-domain data, but specifically when used through hierarchical VLA architectures. We show that this class of models can learn generalizable robot policies for the real world from large-scale, off-domain data. A 🧵 (1/10)

Abhishek Gupta

11,994 次观看 • 1 年前

FRIEDBERG: VIDEO DATA WILL POWER THE NEXT GENERATION OF AI Friedberg broke down the scale shift coming to artificial intelligence, arguing that text based models like GPT are just the beginning, and that the real revolution will come from video-trained systems: “The internet and all these LLMs are language models trained on text from the internet, around 50 billion words total, maybe one to five terabytes of data in their training sets. But if you look at the video data out there, there are hundreds of billions of hours, much of it on YouTube. By some estimates, there’s a thousand exabytes of video data on the internet, about a billion times more than text data. I think we just saw that play out with the new video model that launched yesterday. Google has all this YouTube data, whether or not they’re using it to train, I don’t know. I’ve heard from insiders they’re not allowed to yet and would have to redo the terms of service.” Source: AIFinInsights david friedberg
2:12

FRIEDBERG: VIDEO DATA WILL POWER THE NEXT GENERATION OF AI Friedberg broke down the scale shift coming to artificial intelligence, arguing that text based models like GPT are just the beginning, and that the real revolution will come from video-trained systems: “The internet and all these LLMs are language models trained on text from the internet, around 50 billion words total, maybe one to five terabytes of data in their training sets. But if you look at the video data out there, there are hundreds of billions of hours, much of it on YouTube. By some estimates, there’s a thousand exabytes of video data on the internet, about a billion times more than text data. I think we just saw that play out with the new video model that launched yesterday. Google has all this YouTube data, whether or not they’re using it to train, I don’t know. I’ve heard from insiders they’re not allowed to yet and would have to redo the terms of service.” Source: AIFinInsights david friedberg

Mario Nawfal

15,855 次观看 • 8 个月前