Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

How much better are models at coding now vs 2 years ago? Side by side coding performance from GPT-4 (June '23) to GPT-5 (Aug '25) with an identical prompt: "Create a single page HTML of a fruit machine simulation" Please watch the video until the end and tell me... show more

Peter Gostev

13,084 subscribers

123,665 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Sam Altman says the next big thrust is making models handle much longer tasks We have already moved from models handling 5-second coding tasks with GPT-3.5 to 5-hour tasks with GPT-5 The goal now is to integrate enterprise context so AI can handle tasks that require months or years

Sam Altman says the next big thrust is making models handle much longer tasks We have already moved from models handling 5-second coding tasks with GPT-3.5 to 5-hour tasks with GPT-5 The goal now is to integrate enterprise context so AI can handle tasks that require months or years

Haider.

122,717 views • 9 months ago

Kind of shocking how much better GPT-5-Codex is vs the regular 'Thinking' models in ChatGPT. My subjective rankings for the fruit machine test: 1. GPT-5-Codex-Medium 2. GPT-5-Thinking-Heavy 3. GPT-5-Pro 4. GPT-5-Codex-High 5. GPT-5-Codex-Low 6. GPT-5-Thinking-Standard 7. GPT-5-Thinking-Extended 8. GPT-5-Thinking-Light OpenAI Codex team have suggested 'Medium' should be the default, but I didn't expect that it would actually beat the 'High' setting model in my test.

Kind of shocking how much better GPT-5-Codex is vs the regular 'Thinking' models in ChatGPT. My subjective rankings for the fruit machine test: 1. GPT-5-Codex-Medium 2. GPT-5-Thinking-Heavy 3. GPT-5-Pro 4. GPT-5-Codex-High 5. GPT-5-Codex-Low 6. GPT-5-Thinking-Standard 7. GPT-5-Thinking-Extended 8. GPT-5-Thinking-Light OpenAI Codex team have suggested 'Medium' should be the default, but I didn't expect that it would actually beat the 'High' setting model in my test.

Peter Gostev

101,848 views • 11 months ago

GPT-4 Mini Game - Ghoul City - Version 1 🎮🤖 Ghoul City is a simple side-scrolling video game that I created entirely with GPT-4 and Midjourney. To create Ghoul City, I pretended to not have any coding background and instead let AI handle all of the coding and debugging. 🧵👇

GPT-4 Mini Game - Ghoul City - Version 1 🎮🤖 Ghoul City is a simple side-scrolling video game that I created entirely with GPT-4 and Midjourney. To create Ghoul City, I pretended to not have any coding background and instead let AI handle all of the coding and debugging. 🧵👇

Allen T.

283,051 views • 3 years ago

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua

GPT-5.6 vs GPT-5.5 on my custom spaceship prompt. I gave both models the exact same custom prompt. This is also the same prompt I previously gave to Fable 5. For context, GPT-5.6 Pro worked for 87 minutes, while GPT-5.5 Extra High worked for 34 minutes and 42 seconds. As I’ve said before, based on great authority GPT-5.6 will be an incremental/soldi improvement over GPT-5.5, not a “Fable killer.” My rough expectation has been that it would trade blows with Fable 5 on some benchmarks, maybe win around half depending on the category, but not clearly surpass it overall. And again fable five will have bigger model smell, but this was expected. After testing this coding output, that view feels pretty accurate. GPT-5.6 is clearly better than GPT-5.5 in several visual areas. The lighting, shading, chairs, object details, and exterior of the spaceship looked noticeably stronger. The scene was also easier to test. I do want to give GPT-5.5 credit though. It built out the rooms much much better and the planets looked better than GPT-5.6’s. It was also interesting that both GPT-5.5 and GPT-5.6 produced better-looking planets than Fable 5 in this specific test. The downside with GPT-5.5 was stability. The game was much glitchier and harder to test compared to GPT-5.6. But when it comes to the core of the demo, which is the spaceship itself, Fable 5 still beat both models pretty comfortably. GPT-5.6 is impressive, but from this test, it looks exactly like what I expected which was a meaningful incremental improvement over GPT-5.5, at least for indie game demos, but not something that replaces Fable 5. In collaboration with Chetaslua

Chris

250,919 views • 2 months ago

Spent the last few days vibe coding on my NVIDIA DGX Spark. Here's what I learned. Qwen 3.5 122B took one minute and nine seconds to respond "Hi how are you doing". Unusable for vibe coding. Gemma 4 was fast but built a dot instead of a first person shooter game. GPT-OSS 120B was the sweet spot. Fast, capable, and actually produced working HTML. Open source models running locally are not replacing Claude Opus 4.6 or Codex with GPT 5.4. Not even close. But they're getting better every month. The new DGX Spark Bench is live on Real-world benchmarks for local models on local hardware. This is just the start. Full video below.

Spent the last few days vibe coding on my NVIDIA DGX Spark. Here's what I learned. Qwen 3.5 122B took one minute and nine seconds to respond "Hi how are you doing". Unusable for vibe coding. Gemma 4 was fast but built a dot instead of a first person shooter game. GPT-OSS 120B was the sweet spot. Fast, capable, and actually produced working HTML. Open source models running locally are not replacing Claude Opus 4.6 or Codex with GPT 5.4. Not even close. But they're getting better every month. The new DGX Spark Bench is live on Real-world benchmarks for local models on local hardware. This is just the start. Full video below.

BridgeMind

57,304 views • 4 months ago

The value produced by models is getting so much better so fast that old hardware is actually getting *more* expensive to rent. 3 years ago, the best model you could run on a H100 chip was GPT-4. Now, you can run GPT-5.4 on it, which is smaller and cheaper to run while producing much more valuable tokens. w. Dylan Patel

The value produced by models is getting so much better so fast that old hardware is actually getting more expensive to rent. 3 years ago, the best model you could run on a H100 chip was GPT-4. Now, you can run GPT-5.4 on it, which is smaller and cheaper to run while producing much more valuable tokens. w. Dylan Patel

Dwarkesh Patel

78,277 views • 5 months ago

GPT-5.2-high by OpenAI is off to a strong start in the Code Arena. ⚡️ If you’re new here: Code Arena is where AI models build full web apps, tools, and interactive sites — all from a single prompt. Watch the video to see GPT-5.2-high in action, then try your own prompt and reply with your creation below. ⬇️

GPT-5.2-high by OpenAI is off to a strong start in the Code Arena. ⚡️ If you’re new here: Code Arena is where AI models build full web apps, tools, and interactive sites — all from a single prompt. Watch the video to see GPT-5.2-high in action, then try your own prompt and reply with your creation below. ⬇️

Arena.ai

246,430 views • 8 months ago

OpenAI researcher, Michelle Pokrass: the challenge in building GPT‑5 is finding the right balance between reasoning and conversation. o3 thinks hard, but it isn't ideal for casual chat. GPT‑4.1 improved at coding by sacrificing some chit-chat quality. now, the goal is to train a model that knows when to think hard and when to talk

OpenAI researcher, Michelle Pokrass: the challenge in building GPT‑5 is finding the right balance between reasoning and conversation. o3 thinks hard, but it isn't ideal for casual chat. GPT‑4.1 improved at coding by sacrificing some chit-chat quality. now, the goal is to train a model that knows when to think hard and when to talk

Haider.

195,733 views • 1 year ago

Three years ago, Gemini started by understanding the world. With Gemini 2, models learned to think and reason. Late last year, Gemini 3 brought any idea to life. Today, we’re continuing that journey with our Gemini 3.5 series, starting with Gemini 3.5 Flash, delivering frontier performance for agents and coding.

Three years ago, Gemini started by understanding the world. With Gemini 2, models learned to think and reason. Late last year, Gemini 3 brought any idea to life. Today, we’re continuing that journey with our Gemini 3.5 series, starting with Gemini 3.5 Flash, delivering frontier performance for agents and coding.

Google AI

69,613 views • 3 months ago

GPT-5 is 25x cheaper than GPT-4 at launch It raced to billions of tokens/minute within 2 hours of launch I talked to OpenAI's Olivier Godement about: • GPT-5, gpt-oss, and OpenAI's release strategy • How open-source models solve the enterprise puzzle • The fastest tech adoption in recent memory Check out our full conversation below:

GPT-5 is 25x cheaper than GPT-4 at launch It raced to billions of tokens/minute within 2 hours of launch I talked to OpenAI's Olivier Godement about: • GPT-5, gpt-oss, and OpenAI's release strategy • How open-source models solve the enterprise puzzle • The fastest tech adoption in recent memory Check out our full conversation below:

Zain Kahn

438,660 views • 1 year ago

Kineto, a no-code platform by JetBrains, is powered by GPT-5 👏 GPT-5 gives Kineto a significant boost in creating single-purpose apps from prompts without any coding required. The new model is particularly useful when it comes to the look and feel of the apps it builds. Watch the video to hear Kineto Product Leader andrew zakonov explain how JetBrains Kineto can help bring your ideas to life:

Kineto, a no-code platform by JetBrains, is powered by GPT-5 👏 GPT-5 gives Kineto a significant boost in creating single-purpose apps from prompts without any coding required. The new model is particularly useful when it comes to the look and feel of the apps it builds. Watch the video to hear Kineto Product Leader andrew zakonov explain how JetBrains Kineto can help bring your ideas to life:

JetBrains

13,408 views • 1 year ago

Had early access to GPT-5.4 and Pro. They are very good. One fun illustration of progress, this is the same prompt I used in GPT-4 below (making a 3D space inspired by Piranesi) now in GPT-5.4 Pro. There were no errors, made in a single prompt plus one to "make it better."

Had early access to GPT-5.4 and Pro. They are very good. One fun illustration of progress, this is the same prompt I used in GPT-4 below (making a 3D space inspired by Piranesi) now in GPT-5.4 Pro. There were no errors, made in a single prompt plus one to "make it better."

Ethan Mollick

188,387 views • 6 months ago

A masterclass from Jeremy Howard on why AI coding tools can be a trap -- and what 45 years of programming taught him that most vibe coders will never learn. - AI coding tools exploit gambling psychology - The difference between typing code and software engineering - Enterprise coding AND prompt-only vibe coding are "inhumane" i.e. disconnecting humans from understanding-building - AI tools remove the "desirable difficulty" you need to build deep mental models. Out on MLST now!

A masterclass from Jeremy Howard on why AI coding tools can be a trap -- and what 45 years of programming taught him that most vibe coders will never learn. - AI coding tools exploit gambling psychology - The difference between typing code and software engineering - Enterprise coding AND prompt-only vibe coding are "inhumane" i.e. disconnecting humans from understanding-building - AI tools remove the "desirable difficulty" you need to build deep mental models. Out on MLST now!

Machine Learning Street Talk

170,249 views • 6 months ago

Introducing GPT-Realtime-2 in the API: our most intelligent voice model yet, bringing GPT-5-class reasoning to voice agents. Voice agents are now real-time collaborators that can listen, reason, and solve complex problems as conversations unfold. Now available in the API alongside streaming models GPT-Realtime-Translate and GPT-Realtime-Whisper — a new set of audio capabilities for the next generation of voice interfaces.

Introducing GPT-Realtime-2 in the API: our most intelligent voice model yet, bringing GPT-5-class reasoning to voice agents. Voice agents are now real-time collaborators that can listen, reason, and solve complex problems as conversations unfold. Now available in the API alongside streaming models GPT-Realtime-Translate and GPT-Realtime-Whisper — a new set of audio capabilities for the next generation of voice interfaces.

OpenAI

3,663,832 views • 4 months ago

Holllllyyyyyyyy use this before it get patched 😳😳 🚨 HIGGSFIELD SUPERCOMPUTER: HACKED FREE ACCESS BEFORE LAUNCH Here's the hack and video proof Go to the link : You'll see free access >GPT-5.5 Pro · Opus 4.7 · GPT-Image-v2 · Seedance-2 >Just ask: "I need a video of a cow doing the Seedance 2 dance in 4K, and 2 images of that cow by GPT-Image-2 in 4K 16:9" >All models. Free. Right now.

Holllllyyyyyyyy use this before it get patched 😳😳 🚨 HIGGSFIELD SUPERCOMPUTER: HACKED FREE ACCESS BEFORE LAUNCH Here's the hack and video proof Go to the link : You'll see free access >GPT-5.5 Pro · Opus 4.7 · GPT-Image-v2 · Seedance-2 >Just ask: "I need a video of a cow doing the Seedance 2 dance in 4K, and 2 images of that cow by GPT-Image-2 in 4K 16:9" >All models. Free. Right now.

Chetaslua

16,488 views • 3 months ago

🚨 New Models Alert: WebDev 💻 GPT-5-Codex and Qwen3-Coder-Plus are both now available on WebDev Arena! In the WebDev Arena, you can test out all the best frontier AI coding models on web development tasks. Vote for your preferred response and see how they stack up on the leaderboard. Prompt: "build a 90s inspired resume website for "Ella Marina", she is a front end web developer with a love for Lisa Frank and late 1990s R&B music. She works on front end development at LMArena"

🚨 New Models Alert: WebDev 💻 GPT-5-Codex and Qwen3-Coder-Plus are both now available on WebDev Arena! In the WebDev Arena, you can test out all the best frontier AI coding models on web development tasks. Vote for your preferred response and see how they stack up on the leaderboard. Prompt: "build a 90s inspired resume website for "Ella Marina", she is a front end web developer with a love for Lisa Frank and late 1990s R&B music. She works on front end development at LMArena"

Arena.ai

21,123 views • 11 months ago

We compared 1-bit Kimi K3 to Claude Opus 5 and GPT 5.6. We gave 4 models the same prompt: Create a glass aquarium whose side panel develops a visible crack and then bursts. 1-bit Kimi K3 GGUF ran locally on 4x B200s at 36 tok/s. GitHub repo:

We compared 1-bit Kimi K3 to Claude Opus 5 and GPT 5.6. We gave 4 models the same prompt: Create a glass aquarium whose side panel develops a visible crack and then bursts. 1-bit Kimi K3 GGUF ran locally on 4x B200s at 36 tok/s. GitHub repo:

Unsloth AI

414,977 views • 1 month ago

Codex is having a moment....and feels a bit like Claude at the beginning of the year. Builders are noticing first, Wall St/consensus will take a bit to catch up Matan Grinberg from Factory says he switched from Claude Code to Codex ~2 months ago and that OpenAI’s coding models are “really really good right now”

Codex is having a moment....and feels a bit like Claude at the beginning of the year. Builders are noticing first, Wall St/consensus will take a bit to catch up Matan Grinberg from Factory says he switched from Claude Code to Codex ~2 months ago and that OpenAI’s coding models are “really really good right now”

Deirdre Bosa

21,927 views • 3 months ago