Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

How much better are models at coding now vs 2 years ago? Side by side coding performance from GPT-4 (June '23) to GPT-5 (Aug '25) with an identical prompt: "Create a single page HTML of a fruit machine simulation" Please watch the video until the end and tell me... show more

Peter Gostev (SF: 29 Mar - 3 Apr)

9,669 subscribers

123,593 views • 9 months ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Sam Altman says the next big thrust is making models handle much longer tasks We have already moved from models handling 5-second coding tasks with GPT-3.5 to 5-hour tasks with GPT-5 The goal now is to integrate enterprise context so AI can handle tasks that require months or years

Sam Altman says the next big thrust is making models handle much longer tasks We have already moved from models handling 5-second coding tasks with GPT-3.5 to 5-hour tasks with GPT-5 The goal now is to integrate enterprise context so AI can handle tasks that require months or years

Haider.

122,717 views • 6 months ago

Kind of shocking how much better GPT-5-Codex is vs the regular 'Thinking' models in ChatGPT. My subjective rankings for the fruit machine test: 1. GPT-5-Codex-Medium 2. GPT-5-Thinking-Heavy 3. GPT-5-Pro 4. GPT-5-Codex-High 5. GPT-5-Codex-Low 6. GPT-5-Thinking-Standard 7. GPT-5-Thinking-Extended 8. GPT-5-Thinking-Light OpenAI Codex team have suggested 'Medium' should be the default, but I didn't expect that it would actually beat the 'High' setting model in my test.

Kind of shocking how much better GPT-5-Codex is vs the regular 'Thinking' models in ChatGPT. My subjective rankings for the fruit machine test: 1. GPT-5-Codex-Medium 2. GPT-5-Thinking-Heavy 3. GPT-5-Pro 4. GPT-5-Codex-High 5. GPT-5-Codex-Low 6. GPT-5-Thinking-Standard 7. GPT-5-Thinking-Extended 8. GPT-5-Thinking-Light OpenAI Codex team have suggested 'Medium' should be the default, but I didn't expect that it would actually beat the 'High' setting model in my test.

Peter Gostev

101,848 views • 8 months ago

GPT-4 Mini Game - Ghoul City - Version 1 🎮🤖 Ghoul City is a simple side-scrolling video game that I created entirely with GPT-4 and Midjourney. To create Ghoul City, I pretended to not have any coding background and instead let AI handle all of the coding and debugging. 🧵👇

GPT-4 Mini Game - Ghoul City - Version 1 🎮🤖 Ghoul City is a simple side-scrolling video game that I created entirely with GPT-4 and Midjourney. To create Ghoul City, I pretended to not have any coding background and instead let AI handle all of the coding and debugging. 🧵👇

Allen T.

282,905 views • 3 years ago

GPT-5 Thinking I like revisiting this prompt every now and then to see how much better models have gotten over time

GPT-5 Thinking I like revisiting this prompt every now and then to see how much better models have gotten over time

Angel Bogado 🌻

145,322 views • 9 months ago

Sam Altman says the leap from GPT-4 to GPT-5 will be as big as that of GPT-3 to 4 and the plan is to integrate the GPT and o series of models into one model that can do everything

Sam Altman says the leap from GPT-4 to GPT-5 will be as big as that of GPT-3 to 4 and the plan is to integrate the GPT and o series of models into one model that can do everything

Tsarathustra

224,102 views • 1 year ago

.The Information Reporter Stephanie Palazzolo on OpenAI's GPT-5—a major step up for coding and what it means for competitors. "GPT-5 is a step up on a lot of different domains...one area that really stood out to [sources] was with coding." "Not only is GPT-5 better on more academic...tasks, but it's also better on the more practical programming tasks...working with very large and complex code bases." "If GPT-5 is going to be significantly better at these more practical everyday programming tasks, that could prove to be bad news for Anthropic." Watch the full episode on

.The Information Reporter Stephanie Palazzolo on OpenAI's GPT-5—a major step up for coding and what it means for competitors. "GPT-5 is a step up on a lot of different domains...one area that really stood out to [sources] was with coding." "Not only is GPT-5 better on more academic...tasks, but it's also better on the more practical programming tasks...working with very large and complex code bases." "If GPT-5 is going to be significantly better at these more practical everyday programming tasks, that could prove to be bad news for Anthropic." Watch the full episode on

The Information

52,067 views • 10 months ago

Just tried vibecoding with ThreeJS again (after more than 6 months) and I have to say that the models have gotten much better at coding with ThreeJS and creating 3D models from scratch.

Just tried vibecoding with ThreeJS again (after more than 6 months) and I have to say that the models have gotten much better at coding with ThreeJS and creating 3D models from scratch.

Danny Limanseta

203,654 views • 2 months ago

Sam Altman says the leap from GPT-4 to GPT-5 will be as big as that of GPT-3 to 4 and the plan is to integrate the GPT and o series of models into one model that can do everything - that’s the AGI

Sam Altman says the leap from GPT-4 to GPT-5 will be as big as that of GPT-3 to 4 and the plan is to integrate the GPT and o series of models into one model that can do everything - that’s the AGI

Chubby♨️

231,778 views • 1 year ago

Spent the last few days vibe coding on my NVIDIA DGX Spark. Here's what I learned. Qwen 3.5 122B took one minute and nine seconds to respond "Hi how are you doing". Unusable for vibe coding. Gemma 4 was fast but built a dot instead of a first person shooter game. GPT-OSS 120B was the sweet spot. Fast, capable, and actually produced working HTML. Open source models running locally are not replacing Claude Opus 4.6 or Codex with GPT 5.4. Not even close. But they're getting better every month. The new DGX Spark Bench is live on Real-world benchmarks for local models on local hardware. This is just the start. Full video below.

Spent the last few days vibe coding on my NVIDIA DGX Spark. Here's what I learned. Qwen 3.5 122B took one minute and nine seconds to respond "Hi how are you doing". Unusable for vibe coding. Gemma 4 was fast but built a dot instead of a first person shooter game. GPT-OSS 120B was the sweet spot. Fast, capable, and actually produced working HTML. Open source models running locally are not replacing Claude Opus 4.6 or Codex with GPT 5.4. Not even close. But they're getting better every month. The new DGX Spark Bench is live on Real-world benchmarks for local models on local hardware. This is just the start. Full video below.

BridgeMind

55,982 views • 2 months ago

The value produced by models is getting so much better so fast that old hardware is actually getting *more* expensive to rent. 3 years ago, the best model you could run on a H100 chip was GPT-4. Now, you can run GPT-5.4 on it, which is smaller and cheaper to run while producing much more valuable tokens. w. Dylan Patel

The value produced by models is getting so much better so fast that old hardware is actually getting more expensive to rent. 3 years ago, the best model you could run on a H100 chip was GPT-4. Now, you can run GPT-5.4 on it, which is smaller and cheaper to run while producing much more valuable tokens. w. Dylan Patel

Dwarkesh Patel

77,908 views • 2 months ago

GPT-5.2-high by OpenAI is off to a strong start in the Code Arena. ⚡️ If you’re new here: Code Arena is where AI models build full web apps, tools, and interactive sites — all from a single prompt. Watch the video to see GPT-5.2-high in action, then try your own prompt and reply with your creation below. ⬇️

GPT-5.2-high by OpenAI is off to a strong start in the Code Arena. ⚡️ If you’re new here: Code Arena is where AI models build full web apps, tools, and interactive sites — all from a single prompt. Watch the video to see GPT-5.2-high in action, then try your own prompt and reply with your creation below. ⬇️

Arena.ai

246,068 views • 6 months ago

here's a side by side comparison of a ray-tracing simulator by gpt-5 and claude 4.1 (opus) gpt-5 one-shotted this. claude genuinely blew me away with how badly it failed (and you all know how much i like claude)

here's a side by side comparison of a ray-tracing simulator by gpt-5 and claude 4.1 (opus) gpt-5 one-shotted this. claude genuinely blew me away with how badly it failed (and you all know how much i like claude)

Riley Coyote

42,687 views • 10 months ago

OpenAI researcher, Michelle Pokrass: the challenge in building GPT‑5 is finding the right balance between reasoning and conversation. o3 thinks hard, but it isn't ideal for casual chat. GPT‑4.1 improved at coding by sacrificing some chit-chat quality. now, the goal is to train a model that knows when to think hard and when to talk

OpenAI researcher, Michelle Pokrass: the challenge in building GPT‑5 is finding the right balance between reasoning and conversation. o3 thinks hard, but it isn't ideal for casual chat. GPT‑4.1 improved at coding by sacrificing some chit-chat quality. now, the goal is to train a model that knows when to think hard and when to talk

Haider.

195,733 views • 1 year ago

GPT-5 is 25x cheaper than GPT-4 at launch It raced to billions of tokens/minute within 2 hours of launch I talked to OpenAI's Olivier Godement about: • GPT-5, gpt-oss, and OpenAI's release strategy • How open-source models solve the enterprise puzzle • The fastest tech adoption in recent memory Check out our full conversation below:

GPT-5 is 25x cheaper than GPT-4 at launch It raced to billions of tokens/minute within 2 hours of launch I talked to OpenAI's Olivier Godement about: • GPT-5, gpt-oss, and OpenAI's release strategy • How open-source models solve the enterprise puzzle • The fastest tech adoption in recent memory Check out our full conversation below:

Zain Kahn

438,660 views • 9 months ago

Three years ago, Gemini started by understanding the world. With Gemini 2, models learned to think and reason. Late last year, Gemini 3 brought any idea to life. Today, we’re continuing that journey with our Gemini 3.5 series, starting with Gemini 3.5 Flash, delivering frontier performance for agents and coding.

Three years ago, Gemini started by understanding the world. With Gemini 2, models learned to think and reason. Late last year, Gemini 3 brought any idea to life. Today, we’re continuing that journey with our Gemini 3.5 series, starting with Gemini 3.5 Flash, delivering frontier performance for agents and coding.

Google AI

68,274 views • 24 days ago

Had early access to GPT-5.4 and Pro. They are very good. One fun illustration of progress, this is the same prompt I used in GPT-4 below (making a 3D space inspired by Piranesi) now in GPT-5.4 Pro. There were no errors, made in a single prompt plus one to "make it better."

Had early access to GPT-5.4 and Pro. They are very good. One fun illustration of progress, this is the same prompt I used in GPT-4 below (making a 3D space inspired by Piranesi) now in GPT-5.4 Pro. There were no errors, made in a single prompt plus one to "make it better."

Ethan Mollick

188,306 views • 3 months ago

🚀Introducing Code Arena: the next generation of live coding evals for frontier AI models. Built to test how models plan, scaffold, debug, and build real web apps step-by-step. Try Claude, GPT-5, GLM-4.6 and Gemini in Code Arena today!

🚀Introducing Code Arena: the next generation of live coding evals for frontier AI models. Built to test how models plan, scaffold, debug, and build real web apps step-by-step. Try Claude, GPT-5, GLM-4.6 and Gemini in Code Arena today!

Arena.ai

327,258 views • 7 months ago

GPT-5 thinking has been incredible at coding so far i'm not a big fan of vibe coding, but just tried a "browser OS" test in one shot - gemini 2.5 pro failed - claude 4.1 handled flow better, but not as good overall - gpt-5 nailed it, and added all the standard features DEMO 👇

GPT-5 thinking has been incredible at coding so far i'm not a big fan of vibe coding, but just tried a "browser OS" test in one shot - gemini 2.5 pro failed - claude 4.1 handled flow better, but not as good overall - gpt-5 nailed it, and added all the standard features DEMO 👇

Haider.

43,656 views • 10 months ago

Free GPT-5.3-Codex in Orchids! One of the best coding models in the world right now. > Connect your ChatGPT subscription. > Select GPT-5.3-Codex. > Done, no Orchids credits will be consumed. The best way to build any app at orchids [dot] app.

Free GPT-5.3-Codex in Orchids! One of the best coding models in the world right now. > Connect your ChatGPT subscription. > Select GPT-5.3-Codex. > Done, no Orchids credits will be consumed. The best way to build any app at orchids [dot] app.

Bud

24,048 views • 3 months ago

AI has its PhD and now it’s on the job market. Introducing the AI Productivity Index (APEX), a benchmark that measures how well we’ve automated the most valuable industries in the world. Most benchmarks study abstract capabilities. APEX evaluates model performance on real deliverables across law, finance, consulting, and medicine. The models most capable of doing work today, according to APEX: 🥇 GPT 5 🥈 Grok 4 🥉 Gemini 2.5 Flash Other findings: - GPT 5 demonstrates the strongest performance across all 4 domains - Some cheaper models outperform more expensive models from the same provider (e.g. Gemini 2.5 Flash vs. Gemini 2.5 Pro) - The best open source model, Qwen (7th), performs only 2% behind Grok 4 overall

AI has its PhD and now it’s on the job market. Introducing the AI Productivity Index (APEX), a benchmark that measures how well we’ve automated the most valuable industries in the world. Most benchmarks study abstract capabilities. APEX evaluates model performance on real deliverables across law, finance, consulting, and medicine. The models most capable of doing work today, according to APEX: 🥇 GPT 5 🥈 Grok 4 🥉 Gemini 2.5 Flash Other findings: - GPT 5 demonstrates the strongest performance across all 4 domains - Some cheaper models outperform more expensive models from the same provider (e.g. Gemini 2.5 Flash vs. Gemini 2.5 Pro) - The best open source model, Qwen (7th), performs only 2% behind Grok 4 overall

Brendan (can/do)

451,298 views • 8 months ago