Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Dylan Patel says GPT-5 is not a bigger model and did not gain intelligence through scale but through optimization. He highlights how OpenAI improved efficiency by cutting down wasted tokens compared to earlier o-series models. He adds that GPT-5’s new router system decides whether to use the base model,... show more

Wes Roth

35,782 subscribers

87,388 görüntüleme • 11 ay önce •via X (Twitter)

Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

OpenAI researcher, Michelle Pokrass: the challenge in building GPT‑5 is finding the right balance between reasoning and conversation. o3 thinks hard, but it isn't ideal for casual chat. GPT‑4.1 improved at coding by sacrificing some chit-chat quality. now, the goal is to train a model that knows when to think hard and when to talk

OpenAI researcher, Michelle Pokrass: the challenge in building GPT‑5 is finding the right balance between reasoning and conversation. o3 thinks hard, but it isn't ideal for casual chat. GPT‑4.1 improved at coding by sacrificing some chit-chat quality. now, the goal is to train a model that knows when to think hard and when to talk

Haider.

195,733 görüntüleme • 1 yıl önce

.The Information Reporter Stephanie Palazzolo on OpenAI's GPT-5—a major step up for coding and what it means for competitors. "GPT-5 is a step up on a lot of different domains...one area that really stood out to [sources] was with coding." "Not only is GPT-5 better on more academic...tasks, but it's also better on the more practical programming tasks...working with very large and complex code bases." "If GPT-5 is going to be significantly better at these more practical everyday programming tasks, that could prove to be bad news for Anthropic." Watch the full episode on

.The Information Reporter Stephanie Palazzolo on OpenAI's GPT-5—a major step up for coding and what it means for competitors. "GPT-5 is a step up on a lot of different domains...one area that really stood out to [sources] was with coding." "Not only is GPT-5 better on more academic...tasks, but it's also better on the more practical programming tasks...working with very large and complex code bases." "If GPT-5 is going to be significantly better at these more practical everyday programming tasks, that could prove to be bad news for Anthropic." Watch the full episode on

The Information

52,067 görüntüleme • 11 ay önce

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua

Chris

249,587 görüntüleme • 29 gün önce

You'd think the race to AGI would mean training the biggest possible model. But parameter scaling had stalled for a long time after GPT-4's trillion+ parameters, and only now are models getting bigger again. What gives? Partially it’s RL scaling, as Dylan Patel explains. A 5T parameter model takes 5x longer to generate RL rollouts than a 1T model. Even if the bigger model is 2x more sample-efficient, the smaller model finishes RL faster, gets deployed to research sooner, and starts helping build the next model before the big one is even done training.

You'd think the race to AGI would mean training the biggest possible model. But parameter scaling had stalled for a long time after GPT-4's trillion+ parameters, and only now are models getting bigger again. What gives? Partially it’s RL scaling, as Dylan Patel explains. A 5T parameter model takes 5x longer to generate RL rollouts than a 1T model. Even if the bigger model is 2x more sample-efficient, the smaller model finishes RL faster, gets deployed to research sooner, and starts helping build the next model before the big one is even done training.

Dwarkesh Patel

65,123 görüntüleme • 3 ay önce

Openai giving $50,000 in FREE API credits and it's hidden in your account settings 😳 most people have no idea this exists what you get every single day: - 250K tokens/day for GPT-5.5 and GPT-5.2 (powerful models) - 2.5M tokens/day for GPT-5.1-Mini and GPT-5-Nano (lighter models) - up to 10M tokens/day if you're on tier 3-5 not a one time lump sum, it resets daily it's called the data sharing program and here's how it works: > go to > click data controls → sharing > opt in to both options > done, free daily tokens activate immediately important note: - your data gets used by openai for training and improvements - don't use it for client work or anything sensitive - need a positive account balance to activate perfect for: - learning and experimenting - side projects and personal builds - testing new models without paying you need a positive account balance to activate it but after that it's free every day save this and take free 50k credits before they close it

Openai giving $50,000 in FREE API credits and it's hidden in your account settings 😳 most people have no idea this exists what you get every single day: - 250K tokens/day for GPT-5.5 and GPT-5.2 (powerful models) - 2.5M tokens/day for GPT-5.1-Mini and GPT-5-Nano (lighter models) - up to 10M tokens/day if you're on tier 3-5 not a one time lump sum, it resets daily it's called the data sharing program and here's how it works: > go to > click data controls → sharing > opt in to both options > done, free daily tokens activate immediately important note: - your data gets used by openai for training and improvements - don't use it for client work or anything sensitive - need a positive account balance to activate perfect for: - learning and experimenting - side projects and personal builds - testing new models without paying you need a positive account balance to activate it but after that it's free every day save this and take free 50k credits before they close it

Isra

164,530 görüntüleme • 1 ay önce

"Projects like the New Deal, the Apollo program pale in comparison to what we're doing right now." 🆕 Greg Brockman (Greg Brockman) joins us to talk GPT-5, GPT-OSS, and what's next on OpenAI's road to crystallizing all of human intelligence! “Energy turns into compute, turns into intelligence… crystallizing compute into potential energy you can release again and again.” 0:00:04 - Introductions 0:01:04 - The Evolution of Reasoning at OpenAI 0:04:01 - Online vs Offline Learning in Language Models 0:06:44 - Sample Efficiency and Human Curation in Reinforcement Learning 0:08:16 - Scaling Compute and Supercritical Learning 0:13:21 - Wall clock time limitations in RL and real-world interactions 0:16:34 - Experience with ARC Institute and DNA neural networks 0:19:33 - Defining the GPT-5 Era 0:22:46 - Evaluating Model Intelligence and Task Difficulty 0:25:06 - Practical Advice for Developers Using GPT-5 0:31:48 - Model Specs 0:37:21 - Challenges in RL Preferences (e.g., try/catch) 0:39:13 - Model Routing and Hybrid Architectures in GPT-5 0:43:58 - GPT-5 pricing and compute efficiency improvements 0:46:04 - Self-Improving Coding Agents and Tool Usage 0:49:11 - On-Device Models and Local vs Remote Agent Systems 0:51:34 - Engineering at OpenAI and Leveraging LLMs 0:54:16 - Structuring Codebases and Teams for AI Optimization 0:55:27 - The Value of Engineers in the Age of AGI 0:58:42 - Current state of AI research and lab diversity 1:01:11 - OpenAI’s Prioritization and Focus Areas 1:03:05 - Advice for Founders - It's Not Too Late 1:04:20 - Future outlook and closing thoughts 1:04:33 - Time Capsule to 2045 - Future of Compute and Abundance 1:07:07 - Time Capsule to 2005 - More Problems Will Emerge

"Projects like the New Deal, the Apollo program pale in comparison to what we're doing right now." 🆕 Greg Brockman (Greg Brockman) joins us to talk GPT-5, GPT-OSS, and what's next on OpenAI's road to crystallizing all of human intelligence! “Energy turns into compute, turns into intelligence… crystallizing compute into potential energy you can release again and again.” 0:00:04 - Introductions 0:01:04 - The Evolution of Reasoning at OpenAI 0:04:01 - Online vs Offline Learning in Language Models 0:06:44 - Sample Efficiency and Human Curation in Reinforcement Learning 0:08:16 - Scaling Compute and Supercritical Learning 0:13:21 - Wall clock time limitations in RL and real-world interactions 0:16:34 - Experience with ARC Institute and DNA neural networks 0:19:33 - Defining the GPT-5 Era 0:22:46 - Evaluating Model Intelligence and Task Difficulty 0:25:06 - Practical Advice for Developers Using GPT-5 0:31:48 - Model Specs 0:37:21 - Challenges in RL Preferences (e.g., try/catch) 0:39:13 - Model Routing and Hybrid Architectures in GPT-5 0:43:58 - GPT-5 pricing and compute efficiency improvements 0:46:04 - Self-Improving Coding Agents and Tool Usage 0:49:11 - On-Device Models and Local vs Remote Agent Systems 0:51:34 - Engineering at OpenAI and Leveraging LLMs 0:54:16 - Structuring Codebases and Teams for AI Optimization 0:55:27 - The Value of Engineers in the Age of AGI 0:58:42 - Current state of AI research and lab diversity 1:01:11 - OpenAI’s Prioritization and Focus Areas 1:03:05 - Advice for Founders - It's Not Too Late 1:04:20 - Future outlook and closing thoughts 1:04:33 - Time Capsule to 2045 - Future of Compute and Abundance 1:07:07 - Time Capsule to 2005 - More Problems Will Emerge

Latent.Space

305,090 görüntüleme • 11 ay önce

$Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.$

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,189,304 görüntüleme • 2 ay önce

ELON: GROK’S A BEAST - IT CAN'T BE CRAMMED INTO A CAR “Grok is a giant model; you could not possibly squeeze Grok onto a car, that’s for sure. With Grok, it’s trying to solve for artificial intelligence, with a massive amount of AI training, compute, and inference compute. Grok 5 will only run effectively on a GV300 [AI training cluster]; that’s how much of a beast Grok 5 is. Whereas Tesla’s models are less than 10% the size, maybe closer to 5% the size of Grok.” Source: Elon Musk, Tesla Podcast, October 22, 2025

ELON: GROK’S A BEAST - IT CAN'T BE CRAMMED INTO A CAR “Grok is a giant model; you could not possibly squeeze Grok onto a car, that’s for sure. With Grok, it’s trying to solve for artificial intelligence, with a massive amount of AI training, compute, and inference compute. Grok 5 will only run effectively on a GV300 [AI training cluster]; that’s how much of a beast Grok 5 is. Whereas Tesla’s models are less than 10% the size, maybe closer to 5% the size of Grok.” Source: Elon Musk, Tesla Podcast, October 22, 2025

Mario Nawfal

33,523 görüntüleme • 8 ay önce

I've made a ton of money helping companies implement LLM-as-a-judge evaluations. LLM Judges provide a ton of value. But the hard part is choosing the model to implement the judge. • The family of GPT-5 models is very good, but slow and expensive. • Models like Gemma and Phi are fast and cheap, but not that good. Most of the time, you can only run a percentage of your traffic through the model (otherwise it would be too expensive and slow). But now, there's a better strategy.

I've made a ton of money helping companies implement LLM-as-a-judge evaluations. LLM Judges provide a ton of value. But the hard part is choosing the model to implement the judge. • The family of GPT-5 models is very good, but slow and expensive. • Models like Gemma and Phi are fast and cheap, but not that good. Most of the time, you can only run a percentage of your traffic through the model (otherwise it would be too expensive and slow). But now, there's a better strategy.

Santiago

29,970 görüntüleme • 2 ay önce

I know your timeline is flooded now with word salads of "insane, HER, 10 features you missed, we're so back". Sit down. Chill. Take a deep breath like Mark does in the demo . Let's think step by step: - Technique-wise, OpenAI has figured out a way to map audio to audio directly as first-class modality, and stream videos to a transformer in real-time. These require some new research on tokenization and architecture, but overall it's a data and system optimization problem (as most things are). High-quality data can come from at least 2 sources: 1) Naturally occurring dialogues on YouTube, podcasts, TV series, movies, etc. Whisper can be trained to identify speaker turns in a dialogue or separate overlapping speeches for automated annotation. 2) Synthetic data. Run the slow 3-stage pipeline using the most powerful models: speech1->text1 (ASR), text1->text2 (LLM), text2->speech2 (TTS). The middle LLM can decide when to stop and also simulate how to resume from interruption. It could output additional "thought traces" that are not verbalized to help generate better reply. Then GPT-4o distills directly from speech1->speech2, with optional auxiliary loss functions based on the 3-stage data. After distillation, these behaviors are now baked into the model without emitting intermediate texts. On the system side: the latency would not meet real-time threshold if every video frame is decompressed into an RGB image. OpenAI has likely developed their own neural-first, streaming video codec to transmit the motion deltas as tokens. The communication protocol and NN inference must be co-optimized. For example, there could be a small and energy-efficient NN running on the edge device that decides to transmit more tokens if the video is interesting, and fewer otherwise. - I didn't expect GPT-4o to be closer to GPT-5, the rumored "Arrakis" model that takes multimodal in and out. In fact, it's likely an early checkpoint of GPT-5 that hasn't finished training yet. The branding betrays a certain insecurity. Ahead of Google I/O, OpenAI would rather beat our mental projection of GPT-4.5 than disappoint by missing the sky-high expectation for GPT-5. A smart move to buy more time. - Notably, the assistant is much more lively and even a bit flirty. GPT-4o is trying (perhaps a bit too hard) to sound like HER. OpenAI is eating Character AI's lunch, with almost 100% overlap in form factor and huge distribution channels. It's a pivot towards more emotional AI with strong personality, which OpenAI seemed to actively suppress in the past. - Whoever wins Apple first wins big time. I see 3 levels of integration with iOS: 1) Ditch Siri. OpenAI distills a smaller-tier, purely on-device GPT-4o for iOS, with optional paid upgrade to use the cloud. 2) Native features to stream the camera or screen into the model. Chip-level support for neural audio/video codec. 3) Integrate with iOS system-level action API and smart home APIs. No one uses Siri Shortcuts, but it's time to resurrect. This could become the AI agent product with a billion users from the get-go. The FSD for smartphones with a Tesla-scale data flywheel.

I know your timeline is flooded now with word salads of "insane, HER, 10 features you missed, we're so back". Sit down. Chill. Take a deep breath like Mark does in the demo . Let's think step by step: - Technique-wise, OpenAI has figured out a way to map audio to audio directly as first-class modality, and stream videos to a transformer in real-time. These require some new research on tokenization and architecture, but overall it's a data and system optimization problem (as most things are). High-quality data can come from at least 2 sources: 1) Naturally occurring dialogues on YouTube, podcasts, TV series, movies, etc. Whisper can be trained to identify speaker turns in a dialogue or separate overlapping speeches for automated annotation. 2) Synthetic data. Run the slow 3-stage pipeline using the most powerful models: speech1->text1 (ASR), text1->text2 (LLM), text2->speech2 (TTS). The middle LLM can decide when to stop and also simulate how to resume from interruption. It could output additional "thought traces" that are not verbalized to help generate better reply. Then GPT-4o distills directly from speech1->speech2, with optional auxiliary loss functions based on the 3-stage data. After distillation, these behaviors are now baked into the model without emitting intermediate texts. On the system side: the latency would not meet real-time threshold if every video frame is decompressed into an RGB image. OpenAI has likely developed their own neural-first, streaming video codec to transmit the motion deltas as tokens. The communication protocol and NN inference must be co-optimized. For example, there could be a small and energy-efficient NN running on the edge device that decides to transmit more tokens if the video is interesting, and fewer otherwise. - I didn't expect GPT-4o to be closer to GPT-5, the rumored "Arrakis" model that takes multimodal in and out. In fact, it's likely an early checkpoint of GPT-5 that hasn't finished training yet. The branding betrays a certain insecurity. Ahead of Google I/O, OpenAI would rather beat our mental projection of GPT-4.5 than disappoint by missing the sky-high expectation for GPT-5. A smart move to buy more time. - Notably, the assistant is much more lively and even a bit flirty. GPT-4o is trying (perhaps a bit too hard) to sound like HER. OpenAI is eating Character AI's lunch, with almost 100% overlap in form factor and huge distribution channels. It's a pivot towards more emotional AI with strong personality, which OpenAI seemed to actively suppress in the past. - Whoever wins Apple first wins big time. I see 3 levels of integration with iOS: 1) Ditch Siri. OpenAI distills a smaller-tier, purely on-device GPT-4o for iOS, with optional paid upgrade to use the cloud. 2) Native features to stream the camera or screen into the model. Chip-level support for neural audio/video codec. 3) Integrate with iOS system-level action API and smart home APIs. No one uses Siri Shortcuts, but it's time to resurrect. This could become the AI agent product with a billion users from the get-go. The FSD for smartphones with a Tesla-scale data flywheel.

Jim Fan

991,628 görüntüleme • 2 yıl önce

GPT-5 is live in Cline. We've been working with OpenAI to get this model ready, and here's our take: it's disciplined, persistent, & highly competent. It's collaborative in planning & and a diligent operator while acting. It plans thoroughly, asks optioned follow-ups when needed, & then gets out of the way and ships code. On long tasks it keeps going before pausing to check in. It follows instructions to the letter. And most importantly -- it writes good code. GPT-5 is like "The Wolf" from Pulp Fiction. Comes in, assesses the situation, then executes. Here's what you can expect from GPT-5 in Cline: > verbose while planning; terse while executing > asks a lot of good clarification questions, & frequently provides options when appropriate > strong context retention and persistence over long horizons (256k context window) > good at diff-style edits and multi-file changes (we'll monitor as more usage data comes in) > quiet in Act mode -- writes code without yapping Metaprompting is another strength. We tested early with OpenAI and used GPT-5 to tune our own prompt for GPT-5. Here's a pattern we like: “Answer from your own perspective: what changes or additions would help you better follow this prompt? Here is the prompt (or snippet): [snippet]. Users have complained about X and Y. What minimal edits would you make while keeping the rest intact?” Do you need to change any of your existing patterns in Cline? No -- it's good out of the box. Give a clear goal and constraints, let it plan, then let it cook. Expect more clarifying questions than most models. Pricing: $1.25/M input tokens (+90% cache), $10/M output. Roughly half of Sonnet 4 ($3/$15). Want to try GPT-5? Use it in Cline today for pure, unfiltered inference via the OpenAI, Cline, or OpenRouter providers. (fyi -- GPT-5 one-shotted this browser DAW below on the prompt "build something impressive to show me what you're capable of")

GPT-5 is live in Cline. We've been working with OpenAI to get this model ready, and here's our take: it's disciplined, persistent, & highly competent. It's collaborative in planning & and a diligent operator while acting. It plans thoroughly, asks optioned follow-ups when needed, & then gets out of the way and ships code. On long tasks it keeps going before pausing to check in. It follows instructions to the letter. And most importantly -- it writes good code. GPT-5 is like "The Wolf" from Pulp Fiction. Comes in, assesses the situation, then executes. Here's what you can expect from GPT-5 in Cline: > verbose while planning; terse while executing > asks a lot of good clarification questions, & frequently provides options when appropriate > strong context retention and persistence over long horizons (256k context window) > good at diff-style edits and multi-file changes (we'll monitor as more usage data comes in) > quiet in Act mode -- writes code without yapping Metaprompting is another strength. We tested early with OpenAI and used GPT-5 to tune our own prompt for GPT-5. Here's a pattern we like: “Answer from your own perspective: what changes or additions would help you better follow this prompt? Here is the prompt (or snippet): [snippet]. Users have complained about X and Y. What minimal edits would you make while keeping the rest intact?” Do you need to change any of your existing patterns in Cline? No -- it's good out of the box. Give a clear goal and constraints, let it plan, then let it cook. Expect more clarifying questions than most models. Pricing: $1.25/M input tokens (+90% cache), $10/M output. Roughly half of Sonnet 4 ($3/$15). Want to try GPT-5? Use it in Cline today for pure, unfiltered inference via the OpenAI, Cline, or OpenRouter providers. (fyi -- GPT-5 one-shotted this browser DAW below on the prompt "build something impressive to show me what you're capable of")

Cline

63,496 görüntüleme • 11 ay önce

Just listened to Bill gates podcast with sam, and it's very clear that OpenAI is purposely sandbagging Even Bill - who initially did not expect GPT-5 to be much better than GPT-4, mentioned how amazed he was by the demos hes seen. Right now, everyone thinks chatGPT sucks (including Sam himself) but remember, OpenAI hasn't released a new class of models in ~1.5 years. This is creating the illusion that models have hit their upper limit, and that progress will continue to be slow But if you listen to the podcast , its extremely clear how quickly we're going to accelerate from models doing basic, tasks to actually being able run companies. But personally i think 2024 will be the an even bigger year than the "chatGPT" moment we had. Literally no one (myself included) is ready for the rate of change we're about to experience as humanity. Most companies will be eaten by an agi-like software, and yes that includes most apps you use today. Definitely scary, but there is a sense of calm, knowing that if they're right, all of humanity is in this together. but i am excited to see what I can build with such technology

Just listened to Bill gates podcast with sam, and it's very clear that OpenAI is purposely sandbagging Even Bill - who initially did not expect GPT-5 to be much better than GPT-4, mentioned how amazed he was by the demos hes seen. Right now, everyone thinks chatGPT sucks (including Sam himself) but remember, OpenAI hasn't released a new class of models in ~1.5 years. This is creating the illusion that models have hit their upper limit, and that progress will continue to be slow But if you listen to the podcast , its extremely clear how quickly we're going to accelerate from models doing basic, tasks to actually being able run companies. But personally i think 2024 will be the an even bigger year than the "chatGPT" moment we had. Literally no one (myself included) is ready for the rate of change we're about to experience as humanity. Most companies will be eaten by an agi-like software, and yes that includes most apps you use today. Definitely scary, but there is a sense of calm, knowing that if they're right, all of humanity is in this together. but i am excited to see what I can build with such technology

Sully

613,768 görüntüleme • 2 yıl önce

"This is the first time that OpenAI has released a new model that was not decisively the best." @jason and Gavin Baker discuss OpenAI's GPT-5 release: Jason: " Sam did this tweet, it kind of built a lot of expectation that this would be otherworldly. It wasn't." "Are we hitting either the trough of despair or maybe diminishing returns in the LLM space, where maybe it's feeling incremental, not groundbreaking, when we release these iterations?" Gavin: "Maybe for OpenAI." "This is the first time that OpenAI has released a new model that was not decisively the best." " The one thing about OpenAI is they're very good at product. Just really, really good." "And I just always go back to the statement from Eric Vishria, you know, it might be that if OpenAI never releases another model, they're still a really valuable company."

"This is the first time that OpenAI has released a new model that was not decisively the best." @jason and Gavin Baker discuss OpenAI's GPT-5 release: Jason: " Sam did this tweet, it kind of built a lot of expectation that this would be otherworldly. It wasn't." "Are we hitting either the trough of despair or maybe diminishing returns in the LLM space, where maybe it's feeling incremental, not groundbreaking, when we release these iterations?" Gavin: "Maybe for OpenAI." "This is the first time that OpenAI has released a new model that was not decisively the best." " The one thing about OpenAI is they're very good at product. Just really, really good." "And I just always go back to the statement from Eric Vishria, you know, it might be that if OpenAI never releases another model, they're still a really valuable company."

The All-In Podcast

49,179 görüntüleme • 11 ay önce

Thanksgiving-week treat: an epic conversation on Frontier AI with Lukasz Kaiser -co-author of “Attention Is All You Need” (Transformers) and leading research scientist at OpenAI working on GPT-5.1-era reasoning models. 00:00 – Cold open and intro 01:29 – “AI slowdown” vs a wild week of new frontier models 08:03 – Low-hanging fruit, infra, RL training and better data 11:39 – What is a reasoning model, in plain language 17:02 – Chain-of-thought and training the thinking process with RL 21:39 – Łukasz’s path: from logic and France to Google and Kurzweil 24:20 – Inside the Transformer story and what “attention” really means 28:42 – From Google Brain to OpenAI: culture, scale and GPUs 32:49 – What’s next for pre-training, GPUs and distillation 37:29 – Can we still understand these models? Circuits, sparsity and black boxes 39:42 – GPT-4 → GPT-5 → GPT-5.1: what actually changed 42:40 – Post-training, safety and teaching GPT-5.1 different tones 46:16 – How long should GPT-5.1 think? Reasoning tokens and jagged abilities 47:43 – The five-year-old’s dot puzzle that still breaks frontier models 52:22 – Generalization, child-like learning and whether reasoning is enough 53:48 – Beyond Transformers: ARC, LeCun’s ideas and multimodal bottlenecks 56:10 – GPT-5.1 Codex Max, long-running agents and compaction 1:00:06 – Will foundation models eat most apps? The translation analogy and trust 1:02:34 – What still needs to be solved, and where AI might go next

Thanksgiving-week treat: an epic conversation on Frontier AI with Lukasz Kaiser -co-author of “Attention Is All You Need” (Transformers) and leading research scientist at OpenAI working on GPT-5.1-era reasoning models. 00:00 – Cold open and intro 01:29 – “AI slowdown” vs a wild week of new frontier models 08:03 – Low-hanging fruit, infra, RL training and better data 11:39 – What is a reasoning model, in plain language 17:02 – Chain-of-thought and training the thinking process with RL 21:39 – Łukasz’s path: from logic and France to Google and Kurzweil 24:20 – Inside the Transformer story and what “attention” really means 28:42 – From Google Brain to OpenAI: culture, scale and GPUs 32:49 – What’s next for pre-training, GPUs and distillation 37:29 – Can we still understand these models? Circuits, sparsity and black boxes 39:42 – GPT-4 → GPT-5 → GPT-5.1: what actually changed 42:40 – Post-training, safety and teaching GPT-5.1 different tones 46:16 – How long should GPT-5.1 think? Reasoning tokens and jagged abilities 47:43 – The five-year-old’s dot puzzle that still breaks frontier models 52:22 – Generalization, child-like learning and whether reasoning is enough 53:48 – Beyond Transformers: ARC, LeCun’s ideas and multimodal bottlenecks 56:10 – GPT-5.1 Codex Max, long-running agents and compaction 1:00:06 – Will foundation models eat most apps? The translation analogy and trust 1:02:34 – What still needs to be solved, and where AI might go next

Matt Turck

167,926 görüntüleme • 7 ay önce

If you hold $NVDA, here is the long-running agents argument that made Gavin Baker more bullish on compute. Fable 5 (Claude Opus 4.8) and ChatGPT 5.5 both demonstrated long-running agent capability - models that sustain deep reasoning over hours or days, not single queries. Baker's framing on BG2 Pod: imagine if Einstein had thought about fundamental physics 24 hours a day for a year. A model never gets distracted, never sleeps, never ages. The compute implication: more tokens per query, compounding inference revenue. Under 0.2% of people on Earth use AI agentically today. One such user runs 500 CPU cores and 5 GPUs, 24/7. Scale that to any meaningful population and the compute shortage persists indefinitely via Brad Gerstner: Source: BG2 Pod -

If you hold $NVDA, here is the long-running agents argument that made Gavin Baker more bullish on compute. Fable 5 (Claude Opus 4.8) and ChatGPT 5.5 both demonstrated long-running agent capability - models that sustain deep reasoning over hours or days, not single queries. Baker's framing on BG2 Pod: imagine if Einstein had thought about fundamental physics 24 hours a day for a year. A model never gets distracted, never sleeps, never ages. The compute implication: more tokens per query, compounding inference revenue. Under 0.2% of people on Earth use AI agentically today. One such user runs 500 CPU cores and 5 GPUs, 24/7. Scale that to any meaningful population and the compute shortage persists indefinitely via Brad Gerstner: Source: BG2 Pod -

Podcast Alpha

13,790 görüntüleme • 1 ay önce

Sam Altman says the AI infrastructure race is not about bigger chatbots. It is about shifting the planet's intellectual output from human brains to AI brains. "If people knew what we could do with compute, they would want way, way more." "The thing I am personally most excited about is to use AI and lots of compute to discover new science." "A recent cool example here is we built the Sora Android app using Codex... in less than a month." "You can imagine that way much further, where entire companies can build their products using lots of compute." Then he gives the scale: "Let's say that an AI company today might be generating something on the order of 10 trillion tokens a day out of frontier models." "We're going to have these models at a company be outputting more tokens per day than all of humanity put together, and then 10 times that, and then 100 times that." The bottleneck is moving from software talent to compute supply, inference speed, product integration, and where all that machine intelligence gets pointed.

Sam Altman says the AI infrastructure race is not about bigger chatbots. It is about shifting the planet's intellectual output from human brains to AI brains. "If people knew what we could do with compute, they would want way, way more." "The thing I am personally most excited about is to use AI and lots of compute to discover new science." "A recent cool example here is we built the Sora Android app using Codex... in less than a month." "You can imagine that way much further, where entire companies can build their products using lots of compute." Then he gives the scale: "Let's say that an AI company today might be generating something on the order of 10 trillion tokens a day out of frontier models." "We're going to have these models at a company be outputting more tokens per day than all of humanity put together, and then 10 times that, and then 100 times that." The bottleneck is moving from software talent to compute supply, inference speed, product integration, and where all that machine intelligence gets pointed.

Karl Mehta

16,606 görüntüleme • 20 gün önce

Sam Altman recently said: “GPT-OSS has strong real-world performance comparable to o4-mini—and you can run it locally on your phone.” Many believed running a 20B-parameter model on mobile devices was still years away. At Nexa AI, we’ve built our foundation on deep on-device AI technology—turning that vision into reality. Today, GPT-OSS is running fully local on mobile devices through our app, Nexa Studio. Real performance on Snapdragon Gen 5: - 17 tokens/sec decoding speed - < 3 seconds Time-to-First-Token Developers can now use NexaSDK to build their own local AI apps powered by GPT-OSS. What this unlocks: - Real reasoning models running locally - The next wave of AI Agent use cases—without cloud limits - True data privacy: your data never leaves your device Appreciation to OpenAI Sam Altman Greg Brockman and the entire leadership team for pushing the boundaries of the open-source AI community. Thanks to Qualcomm for the partnership, and to Manoj Khilnani and Chun-Po (Jerry) Chang for the incredible collaboration.

Sam Altman recently said: “GPT-OSS has strong real-world performance comparable to o4-mini—and you can run it locally on your phone.” Many believed running a 20B-parameter model on mobile devices was still years away. At Nexa AI, we’ve built our foundation on deep on-device AI technology—turning that vision into reality. Today, GPT-OSS is running fully local on mobile devices through our app, Nexa Studio. Real performance on Snapdragon Gen 5: - 17 tokens/sec decoding speed - < 3 seconds Time-to-First-Token Developers can now use NexaSDK to build their own local AI apps powered by GPT-OSS. What this unlocks: - Real reasoning models running locally - The next wave of AI Agent use cases—without cloud limits - True data privacy: your data never leaves your device Appreciation to OpenAI Sam Altman Greg Brockman and the entire leadership team for pushing the boundaries of the open-source AI community. Thanks to Qualcomm for the partnership, and to Manoj Khilnani and Chun-Po (Jerry) Chang for the incredible collaboration.

NEXA AI

197,453 görüntüleme • 9 ay önce

Andrej Karpathy just made one of the most interesting arguments about AI model design that most people are completely missing. His take is that frontier AI models are not too big because the technology is complex and too big because the training data is garbage. When you or I think of the internet, we picture Wall Street Journal articles, Wikipedia entries, serious writing. That is not what a pretraining dataset looks like. When researchers at frontier labs look at random documents from the actual training corpus, it is stock ticker symbols, broken HTML, spam, gibberish. One estimate puts Llama 3's information compression at just 0.07 bits per token meaning the model has only a hazy recollection of most of what it trained on. So we build trillion parameter models not because we need a trillion parameter brain but because we need a trillion-parameter compression engine to squeeze some intelligence out of a firehose of noise. Most of those parameters are doing memory work, not cognitive work. Karpathy's prediction is separate the two entirely. Build a cognitive core, a model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization and pair it with external memory that it can query when it needs facts. He thinks a cognitive core trained on high-quality data could hit genuine intelligence at around one billion parameters. For reference, today's flagship models run between 200 billion and 1.8 trillion parameters with most of that weight dedicated to remembering the internet's slop. The trend is already moving his direction. GPT-4o operates at roughly 200 billion parameters and outperforms the original 1.8 trillion-parameter GPT-4. Inference costs for GPT-3.5-level performance dropped 280-fold between 2022 and 2024 driven almost entirely by smaller, cleaner, better-architected models. The real bottleneck in AI right now is not compute but rather data quality.

Andrej Karpathy just made one of the most interesting arguments about AI model design that most people are completely missing. His take is that frontier AI models are not too big because the technology is complex and too big because the training data is garbage. When you or I think of the internet, we picture Wall Street Journal articles, Wikipedia entries, serious writing. That is not what a pretraining dataset looks like. When researchers at frontier labs look at random documents from the actual training corpus, it is stock ticker symbols, broken HTML, spam, gibberish. One estimate puts Llama 3's information compression at just 0.07 bits per token meaning the model has only a hazy recollection of most of what it trained on. So we build trillion parameter models not because we need a trillion parameter brain but because we need a trillion-parameter compression engine to squeeze some intelligence out of a firehose of noise. Most of those parameters are doing memory work, not cognitive work. Karpathy's prediction is separate the two entirely. Build a cognitive core, a model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization and pair it with external memory that it can query when it needs facts. He thinks a cognitive core trained on high-quality data could hit genuine intelligence at around one billion parameters. For reference, today's flagship models run between 200 billion and 1.8 trillion parameters with most of that weight dedicated to remembering the internet's slop. The trend is already moving his direction. GPT-4o operates at roughly 200 billion parameters and outperforms the original 1.8 trillion-parameter GPT-4. Inference costs for GPT-3.5-level performance dropped 280-fold between 2022 and 2024 driven almost entirely by smaller, cleaner, better-architected models. The real bottleneck in AI right now is not compute but rather data quality.

Milk Road AI

200,276 görüntüleme • 3 ay önce

$Vinod Khosla fractured his wrist. Sent an X-ray to ChatGPT. Got a treatment plan. Forwarded it to a surgeon he had never met. The surgeon said it was exactly what he would have recommended. Khosla skipped the consultation entirely. The catch: general LLMs have a 20-30% triage error rate. His son's company layers domain-specific AI on GPT-5 and brings that to near-zero. That gap - base model vs. domain-layered AI - is the investable signal. The opportunity is not in OpenAI. It is in the companies building the health-specific layer on top. Full breakdown: Source: SparX by Mukesh Bansal -$

Vinod Khosla fractured his wrist. Sent an X-ray to ChatGPT. Got a treatment plan. Forwarded it to a surgeon he had never met. The surgeon said it was exactly what he would have recommended. Khosla skipped the consultation entirely. The catch: general LLMs have a 20-30% triage error rate. His son's company layers domain-specific AI on GPT-5 and brings that to near-zero. That gap - base model vs. domain-layered AI - is the investable signal. The opportunity is not in OpenAI. It is in the companies building the health-specific layer on top. Full breakdown: Source: SparX by Mukesh Bansal -

Podcast Alpha

66,898 görüntüleme • 1 ay önce