Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🚨 New Models Alert: WebDev 💻 GPT-5-Codex and Qwen3-Coder-Plus are both now available on WebDev Arena! In the WebDev Arena, you can test out all the best frontier AI coding models on web development tasks. Vote for your preferred response and see how they stack up on the leaderboard.... show more

Arena.ai

165,198 subscribers

21,096 views • 10 months ago •via X (Twitter)

Science & Technology Gaming Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Very excited to share the best coding model we’ve ever built! Today we’re launching Gemini 2.5 Pro Preview 'I/O edition' with massively improved coding capabilities. Ranks no.1 on LMArena in Coding and no.1 on the WebDev Arena Leaderboard. It’s especially good at building interactive web apps - this demo shows how it can be helpful for prototyping ideas. Try it in Google Gemini, Vertex AI, and AI Studio Enjoy the pre-I/O goodies !

Very excited to share the best coding model we’ve ever built! Today we’re launching Gemini 2.5 Pro Preview 'I/O edition' with massively improved coding capabilities. Ranks no.1 on LMArena in Coding and no.1 on the WebDev Arena Leaderboard. It’s especially good at building interactive web apps - this demo shows how it can be helpful for prototyping ideas. Try it in Google Gemini, Vertex AI, and AI Studio Enjoy the pre-I/O goodies !

Demis Hassabis

995,839 views • 1 year ago

👨‍💻Have you tested models in the new Code Arena yet? In this thread, we’re showcasing real Gemini 3 Pro by Google DeepMind creations and the exact prompts used, all of them built inside the Code Arena. You can directly compare Gemini’s outputs against other top frontier models on real web development tasks. Build, compare, vote, and share your own creations directly from the chat. See examples in the thread. 🧵

👨‍💻Have you tested models in the new Code Arena yet? In this thread, we’re showcasing real Gemini 3 Pro by Google DeepMind creations and the exact prompts used, all of them built inside the Code Arena. You can directly compare Gemini’s outputs against other top frontier models on real web development tasks. Build, compare, vote, and share your own creations directly from the chat. See examples in the thread. 🧵

Arena.ai

25,369 views • 8 months ago

Introducing Agent Mode: Agentic AI is now measured in the Arena. Agent Mode can do deep research, create reports, generate images, build websites, debug code, and more. It completes more complex tasks by using tools like web search, bash in a sandbox environment, image generation, file writing, and asking follow-up questions. Frontier models are waiting for you in Agent Mode to take on real-world tasks. GPT-5.5, Claude Opus 4.7, Gemini 3.1 Pro, and top open models. Test them yourself.

Introducing Agent Mode: Agentic AI is now measured in the Arena. Agent Mode can do deep research, create reports, generate images, build websites, debug code, and more. It completes more complex tasks by using tools like web search, bash in a sandbox environment, image generation, file writing, and asking follow-up questions. Frontier models are waiting for you in Agent Mode to take on real-world tasks. GPT-5.5, Claude Opus 4.7, Gemini 3.1 Pro, and top open models. Test them yourself.

Arena.ai

360,320 views • 1 month ago

🚨BIG NEWS: 🎬 Video Arena is now live on the web! Test out Veo 3.1, Sora 2, Seedance v1.5 Pro, Kling 2.6 Pro, Wan 2.5 & more. What started last summer as a small Discord bot experiment has grown into a rigorous way to measure and understand how frontier video models perform with real-world use. Thank you to our wonderful community for all the feedback! Today, we’re opening up access by making it available on the web. 🎥 Generate videos with 15 different frontier AI models and compare them head-to-head. 📊 Vote for the best output to power the leaderboards.

🚨BIG NEWS: 🎬 Video Arena is now live on the web! Test out Veo 3.1, Sora 2, Seedance v1.5 Pro, Kling 2.6 Pro, Wan 2.5 & more. What started last summer as a small Discord bot experiment has grown into a rigorous way to measure and understand how frontier video models perform with real-world use. Thank you to our wonderful community for all the feedback! Today, we’re opening up access by making it available on the web. 🎥 Generate videos with 15 different frontier AI models and compare them head-to-head. 📊 Vote for the best output to power the leaderboards.

Arena.ai

61,930 views • 6 months ago

Today's update of Gemini 2.5 Pro Preview is amazing at code web dev, editing, and transformation -- it is now #1 on the WebDev arena, with an ELO score of 1420. That means you can build super cool, visually stunning web apps, with just a single prompt! Check out this example, where we explore the world using this app we created in Gemini's Canvas using the Maps API.

Today's update of Gemini 2.5 Pro Preview is amazing at code web dev, editing, and transformation -- it is now #1 on the WebDev arena, with an ELO score of 1420. That means you can build super cool, visually stunning web apps, with just a single prompt! Check out this example, where we explore the world using this app we created in Gemini's Canvas using the Maps API.

Tulsee Doshi

115,545 views • 1 year ago

This release is fucking huge. It's one of the biggest updates to LMArena this year! Code Arena is our next generation of coding evaluations, beginning with web development tasks. Here you can use models to build interactive websites and share them with your friends. The links are persistent, so you can e.g. build a game and play it whenever you want. Here watch two models -- Claude Haiku and Grok-Code-Fast -- compete to build a galaxy. In this case, I liked the "star-wars" effect of Grok!

This release is fucking huge. It's one of the biggest updates to LMArena this year! Code Arena is our next generation of coding evaluations, beginning with web development tasks. Here you can use models to build interactive websites and share them with your friends. The links are persistent, so you can e.g. build a game and play it whenever you want. Here watch two models -- Claude Haiku and Grok-Code-Fast -- compete to build a galaxy. In this case, I liked the "star-wars" effect of Grok!

Anastasios Nikolas Angelopoulos

38,032 views • 8 months ago

OpenAI are testing a new model on the Web Dev Arena Arena under the name 'Anonymous Chatbot 0717'. I can't believe I'm gonna say this, but it is genuinely at a completely different level of front end coding - far better than Sonnet, o3, Gemini 2.5 Pro, or Grok 4. To test it, I ran a great prompt borrowed from the amazing The Feature Crew YouTube channel, asking models to create a procedurally generated planet with Three.js. Take a look yourselves, but I'm pretty astonished by how big the jump is. I have featured the new model twice just because its implementations have been so interesting. Of course, this is only one test, but OpenAI models have always been a bit 'meh' at front-end work, and they seem to have finally overtaken everyone else on that front. We'll see when it comes out. Credit to Chetaslua for discovering the model

OpenAI are testing a new model on the Web Dev Arena Arena under the name 'Anonymous Chatbot 0717'. I can't believe I'm gonna say this, but it is genuinely at a completely different level of front end coding - far better than Sonnet, o3, Gemini 2.5 Pro, or Grok 4. To test it, I ran a great prompt borrowed from the amazing The Feature Crew YouTube channel, asking models to create a procedurally generated planet with Three.js. Take a look yourselves, but I'm pretty astonished by how big the jump is. I have featured the new model twice just because its implementations have been so interesting. Of course, this is only one test, but OpenAI models have always been a bit 'meh' at front-end work, and they seem to have finally overtaken everyone else on that front. We'll see when it comes out. Credit to Chetaslua for discovering the model

Peter Gostev

495,815 views • 1 year ago

Introducing Orchids, the world's first vibe coding IDE. Orchids can build, watch, and listen on par with a human developer. Orchids ranks #1 on App Bench, the most rigorous benchmark for end-to-end software development. An agent, IDE, built-in browser, Supabase, and Stripe all in a single tool. Local, no lock in, no browser limitations - the next step forward in building with AI. Comment for 100k free credits at orchids [dot] app.

Introducing Orchids, the world's first vibe coding IDE. Orchids can build, watch, and listen on par with a human developer. Orchids ranks #1 on App Bench, the most rigorous benchmark for end-to-end software development. An agent, IDE, built-in browser, Supabase, and Stripe all in a single tool. Local, no lock in, no browser limitations - the next step forward in building with AI. Comment for 100k free credits at orchids [dot] app.

Bud

850,367 views • 7 months ago

a16z GP David George: "You need to adapt to the AI era or die. "On the front end, you need to incorporate AI into your product natively, and not just attach a chatbot app into your existing workflow." "Reimagine what it can mean with AI and be aggressive about disrupting yourself and changing." "And then on the back end... you need to be fully rolled out with the latest coding models for all of your developers." David George Jen Kha

a16z GP David George: "You need to adapt to the AI era or die. "On the front end, you need to incorporate AI into your product natively, and not just attach a chatbot app into your existing workflow." "Reimagine what it can mean with AI and be aggressive about disrupting yourself and changing." "And then on the back end... you need to be fully rolled out with the latest coding models for all of your developers." David George Jen Kha

a16z

28,037 views • 5 months ago

PhoenixLLM for Coding & Engineering is a specialized vertical LLM running on SkyNet custom-tailored for automating the development of applications in 80+ programming languages, with output quality on-par with GPT4 and GPT-4o. One of its use cases include rapid development of smart contracts and dApps on various public blockchains by using a simple prompt. Try out and test the PhoenixLLM Coder to generate smart contract code and utilize with Ethereum Remix tool ( to compile and deploy. A few examples of smart contracts include DeFi collateralized lending, DeFi liquidity pool, and staking contract. Stay tuned for a smart contract development hackathon using PhoenixLLM Coder + Remix to win some $PHB!

PhoenixLLM for Coding & Engineering is a specialized vertical LLM running on SkyNet custom-tailored for automating the development of applications in 80+ programming languages, with output quality on-par with GPT4 and GPT-4o. One of its use cases include rapid development of smart contracts and dApps on various public blockchains by using a simple prompt. Try out and test the PhoenixLLM Coder to generate smart contract code and utilize with Ethereum Remix tool ( to compile and deploy. A few examples of smart contracts include DeFi collateralized lending, DeFi liquidity pool, and staking contract. Stay tuned for a smart contract development hackathon using PhoenixLLM Coder + Remix to win some $PHB!

Phoenix AI Labs

37,324 views • 1 year ago

Love this example! Congrats to Google DeepMind! I successfully replicated the Boggle game on SWE Arena. TLDR: Gemini 2.0 Pro won! In the case of Claude 3.5 Sonnet vs Gemini 2.0 Pro, both are given the image and prompt to generate the algo to get the results. The image and prompt are exactly the same. Try it out on SWE Arena: BTW can we get a bigger streaming rate limit for our API, so that ppl can better compare the latest Gemini models with other frontier models? Jeff Dean Logan Kilpatrick 👀

Love this example! Congrats to Google DeepMind! I successfully replicated the Boggle game on SWE Arena. TLDR: Gemini 2.0 Pro won! In the case of Claude 3.5 Sonnet vs Gemini 2.0 Pro, both are given the image and prompt to generate the algo to get the results. The image and prompt are exactly the same. Try it out on SWE Arena: BTW can we get a bigger streaming rate limit for our API, so that ppl can better compare the latest Gemini models with other frontier models? Jeff Dean Logan Kilpatrick 👀

Terry Yue Zhuo

48,220 views • 1 year ago

We’re excited to partner with OpenAI to launch their new open source models natively on Databricks! gpt-oss sets a new standard of quality for open language models, supporting advanced reasoning with the transparency, flexibility and control enterprises need. Running on Databricks, the gpt-oss models connect securely to your data and scale with built-in governance, and expand what you can build and do with GenAI. Try both the 20B and 120B today in the Mosaic AI Playground.

We’re excited to partner with OpenAI to launch their new open source models natively on Databricks! gpt-oss sets a new standard of quality for open language models, supporting advanced reasoning with the transparency, flexibility and control enterprises need. Running on Databricks, the gpt-oss models connect securely to your data and scale with built-in governance, and expand what you can build and do with GenAI. Try both the 20B and 120B today in the Mosaic AI Playground.

Databricks

10,132 views • 11 months ago

Today, we launched browser use inside Codex to further close the build & verify loop for local development! Now, you can ask Codex to build your front end, and test it like a user would by clicking through the app. Codex sees everything a user sees through vision & checks the network/console logs to help debug & fix any issues that it finds. This change brings us closer to fully autonomous coding agents that delivers high quality and tested changes. Watch Codex test my app in the browser, catch & fix a real bug, and doing that loop again with a brand new feature.

James Sun

226,529 views • 3 months ago

Introducing Prompt-to-leaderboard (P2L): a real-time LLM leaderboard tailored exactly to your use case! P2L trains an LLM to generate "prompt-specific" leaderboards, so you can input a prompt and get a leaderboard specifically for that prompt. The model is trained on the 2M human preference votes from Chatbot Arena. P2L Highlights: 🔹Instant leaderboard for any prompt 🗿 🔹Optimal model routing (hit #1 on Chatbot Arena in Jan 2025 with 1395 score 🧏) 🔹Fine-grained model strength & weakness analysis 🤓 Check out our demo and thread below for more details!

Introducing Prompt-to-leaderboard (P2L): a real-time LLM leaderboard tailored exactly to your use case! P2L trains an LLM to generate "prompt-specific" leaderboards, so you can input a prompt and get a leaderboard specifically for that prompt. The model is trained on the 2M human preference votes from Chatbot Arena. P2L Highlights: 🔹Instant leaderboard for any prompt 🗿 🔹Optimal model routing (hit #1 on Chatbot Arena in Jan 2025 with 1395 score 🧏) 🔹Fine-grained model strength & weakness analysis 🤓 Check out our demo and thread below for more details!

Arena.ai

122,525 views • 1 year ago

We’ve been building this in secret for a while. We love Claude Code, and we’ve reimplemented “Claude Code” in Genspark. Introducing Genspark AI Developer - “Claude Code” in the browser - “Claude Code” on mobile - Everyone can build, no coding experience required - Supports multiple frontier models: Claude Sonnet 4, Claude Opus 4.1, GPT‑5, Kimi + Groq 1/4

We’ve been building this in secret for a while. We love Claude Code, and we’ve reimplemented “Claude Code” in Genspark. Introducing Genspark AI Developer - “Claude Code” in the browser - “Claude Code” on mobile - Everyone can build, no coding experience required - Supports multiple frontier models: Claude Sonnet 4, Claude Opus 4.1, GPT‑5, Kimi + Groq 1/4

Eric Jing

23,245,863 views • 11 months ago

Today we announced the Kaggle Game Arena, a new benchmarking platform where AI models and agents can compete head-to-head in strategic games, starting with chess ♟️. Why games, you ask? 🤔 Games are perfect for AI evaluation because they help us understand how models tackle complex reasoning tasks. Many games are a proxy for real-world skills and can test a model's ability in areas like strategic planning, adaptation, and memory. To inaugurate the Game Arena, we're launching an AI chess exhibition tournament. The world’s leading AI models will battle in head-to-head games in a multi-day event running from August 5-7 with streamed games happening daily. Tune-in to root for your favorites at

Today we announced the Kaggle Game Arena, a new benchmarking platform where AI models and agents can compete head-to-head in strategic games, starting with chess ♟️. Why games, you ask? 🤔 Games are perfect for AI evaluation because they help us understand how models tackle complex reasoning tasks. Many games are a proxy for real-world skills and can test a model's ability in areas like strategic planning, adaptation, and memory. To inaugurate the Game Arena, we're launching an AI chess exhibition tournament. The world’s leading AI models will battle in head-to-head games in a multi-day event running from August 5-7 with streamed games happening daily. Tune-in to root for your favorites at

Google AI

132,070 views • 11 months ago

AutoGPT might be the next big step in AI. Here's why Karpathy recently said "AutoGPT is the next frontier of prompt engineering" AutoGPT is the equivalent of giving GPT-based models a memory and a body. You can now give a task to an AI agent and have it autonomously come up with a plan, execute on it, browse the web, and use new data to revise the strategy until the task is completed. It can analyze the market and come up with a trading strategy, customer service, marketing, finance, or other tasks that requires continuous updates. There are three components to it: 1. Architecture: It leverages GPT-4 and GPT-3.5 via API. 2. Autonomous Iterations: AutoGPT can refine its outputs by self-critical review, building on its previous work and integrating prompt history for more accurate results. 3. Memory Management: Integration with Pinecone allows for long-term memory storage, enabling context preservation and improved decision-making. 4. Multi-functionality: Capabilities include file manipulation, web browsing, and data retrieval, distinguishing AutoGPT from previous AI advancements by broadening its application scope.

AutoGPT might be the next big step in AI. Here's why Karpathy recently said "AutoGPT is the next frontier of prompt engineering" AutoGPT is the equivalent of giving GPT-based models a memory and a body. You can now give a task to an AI agent and have it autonomously come up with a plan, execute on it, browse the web, and use new data to revise the strategy until the task is completed. It can analyze the market and come up with a trading strategy, customer service, marketing, finance, or other tasks that requires continuous updates. There are three components to it: 1. Architecture: It leverages GPT-4 and GPT-3.5 via API. 2. Autonomous Iterations: AutoGPT can refine its outputs by self-critical review, building on its previous work and integrating prompt history for more accurate results. 3. Memory Management: Integration with Pinecone allows for long-term memory storage, enabling context preservation and improved decision-making. 4. Multi-functionality: Capabilities include file manipulation, web browsing, and data retrieval, distinguishing AutoGPT from previous AI advancements by broadening its application scope.

Lior Alexander

808,102 views • 3 years ago

Blake Griffin on the Clippers: "This organization's been through a lot. You can build a new arena, you can be at games, you can cheer on the team, you're gonna have all this high energy but at the end of the day an organization is built on a foundation of respect and it's built on how you treat people and I don't think that in this situation Chris was treated right. I'm sorta at a loss for words a little bit"

Blake Griffin on the Clippers: "This organization's been through a lot. You can build a new arena, you can be at games, you can cheer on the team, you're gonna have all this high energy but at the end of the day an organization is built on a foundation of respect and it's built on how you treat people and I don't think that in this situation Chris was treated right. I'm sorta at a loss for words a little bit"

Oh No He Didn't

602,136 views • 7 months ago

If you use LLM-as-judge, this one is for you. (bookmark it) Most teams validate their agent's outputs by calling a frontier model as the judge. It works, until it doesn't. Three problems stack up fast: → Cost: you're hitting a frontier API on every turn, every tool call, every response. In production that burns millions. → Latency: bigger models, remote calls, slow reasoning on every check. → Blind spots: frontier models don't actually know your domain. In finance, insurance, or healthcare, they miss the keywords and principles your work depends on. So I walk through a different approach: train your own small LLM judge. Instead of a giant model, you start with a small one and let the system generate the training data for you. It decomposes your domain, samples synthetic examples, runs them through a debate arena where judges reach consensus, then trains on the refined set. The result is a judge that's cheaper, faster, and more accurate on your data than Gemini, Claude, or GPT, with an OpenAI-compatible endpoint you can even deploy on-prem. I show the whole thing end to end, using a Claude Code plugin and a web interface, with a real insurance RAG grounding evaluator as the example. You can get the plugin here: Here's the full breakdown: 00:00 - Intro 00:12 - Three problems with using frontier LLMs as judges 01:05 - A different approach: train your own small judge 01:31 - How it works (synthetic data and a debate arena) 02:50 - Installing the Claude Code plugin 04:03 - Defining your task with /eval 04:34 - Example: an insurance RAG grounding evaluator 05:51 - Kicking it off and giving early feedback 06:26 - Choosing labels, domain, and strictness 08:30 - The web interface and dashboard 09:52 - Bringing your own example data (optional) 10:26 - The finished model: endpoint, accuracy, and speed 11:16 - Control, on-prem deployment, and interpretability 11:57 - Benchmarks vs frontier models and the GitHub repo 12:30 - Outro I worked with the Plurai team on this. Thanks for sponsoring the video.

If you use LLM-as-judge, this one is for you. (bookmark it) Most teams validate their agent's outputs by calling a frontier model as the judge. It works, until it doesn't. Three problems stack up fast: → Cost: you're hitting a frontier API on every turn, every tool call, every response. In production that burns millions. → Latency: bigger models, remote calls, slow reasoning on every check. → Blind spots: frontier models don't actually know your domain. In finance, insurance, or healthcare, they miss the keywords and principles your work depends on. So I walk through a different approach: train your own small LLM judge. Instead of a giant model, you start with a small one and let the system generate the training data for you. It decomposes your domain, samples synthetic examples, runs them through a debate arena where judges reach consensus, then trains on the refined set. The result is a judge that's cheaper, faster, and more accurate on your data than Gemini, Claude, or GPT, with an OpenAI-compatible endpoint you can even deploy on-prem. I show the whole thing end to end, using a Claude Code plugin and a web interface, with a real insurance RAG grounding evaluator as the example. You can get the plugin here: Here's the full breakdown: 00:00 - Intro 00:12 - Three problems with using frontier LLMs as judges 01:05 - A different approach: train your own small judge 01:31 - How it works (synthetic data and a debate arena) 02:50 - Installing the Claude Code plugin 04:03 - Defining your task with /eval 04:34 - Example: an insurance RAG grounding evaluator 05:51 - Kicking it off and giving early feedback 06:26 - Choosing labels, domain, and strictness 08:30 - The web interface and dashboard 09:52 - Bringing your own example data (optional) 10:26 - The finished model: endpoint, accuracy, and speed 11:16 - Control, on-prem deployment, and interpretability 11:57 - Benchmarks vs frontier models and the GitHub repo 12:30 - Outro I worked with the Plurai team on this. Thanks for sponsoring the video.

Akshay 🚀

29,586 views • 24 days ago

This "AI Software Engineer" is doing something different to everyone else: Databutton specializes in building full-stack applications using React in the front end and Python in the back end. That's it. I think this gives them a huge advantage: They can fine-tune their models on a very focused tech stack and make them better than anyone else. Instead of having to worry about every programming language and tech stack out there, they specialize and make a much better experience. I think this is smart. In the attached video, I started building a landing page to sell a course. You can see in real time how everything works.

This "AI Software Engineer" is doing something different to everyone else: Databutton specializes in building full-stack applications using React in the front end and Python in the back end. That's it. I think this gives them a huge advantage: They can fine-tune their models on a very focused tech stack and make them better than anyone else. Instead of having to worry about every programming language and tech stack out there, they specialize and make a much better experience. I think this is smart. In the attached video, I started building a landing page to sell a course. You can see in real time how everything works.

Santiago

49,379 views • 1 year ago