正在加载视频...

视频加载失败

返回首页

Are open source models catching up to proprietary models? We’ve looked back at 3 years of Arena’s data to show how the race has evolved. For comparison, we’ve taken the top 20% of the models and uncovered the following: - Before mid 2024: The gap was between 100-150 points...

arena's profile picture

Arena.ai

153,108 subscribers

22,447 次观看 • 3 个月前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

5 patterns in Text Arena's price–performance Pareto frontier since 2023: 1. GPT-4-level quality is now ~500x lower cost. - From a ~$50 blended price per million tokens in 2023 to ~$0.10 today. 2. The higher-price end is both better and lower-priced since 2023. - The leading Arena score has climbed ~170 points (1,330 → 1,500). While the price of the higher-end frontier models dropped from ~$50 to ~$20 per million tokens. 3. The low-cost end gained the most. - Under $0.20 per million tokens, the best available model went from ~1,000 Arena score in 2023 to ~1,440 today. 4. The low-cost/top performance gap has nearly closed. - In 2023, sub-$0.20 models trailed the leader by ~350 Arena points. Today, ~60. 5. The cast has rotated quite a bit. - - OpenAI set the 2023–24 benchmark. - AI at Meta strengthened the low-cost end in 2024. - Google DeepMind drove the 2025 jump. - Anthropic holds the peak in 2026. - xAI and Chinese labs like DeepSeek AI, Z.ai, Kimi.ai, Xiaomi MiMo, and Qwen are continuing to push the mid-price frontier.
0:26

5 patterns in Text Arena's price–performance Pareto frontier since 2023: 1. GPT-4-level quality is now ~500x lower cost. - From a ~$50 blended price per million tokens in 2023 to ~$0.10 today. 2. The higher-price end is both better and lower-priced since 2023. - The leading Arena score has climbed ~170 points (1,330 → 1,500). While the price of the higher-end frontier models dropped from ~$50 to ~$20 per million tokens. 3. The low-cost end gained the most. - Under $0.20 per million tokens, the best available model went from ~1,000 Arena score in 2023 to ~1,440 today. 4. The low-cost/top performance gap has nearly closed. - In 2023, sub-$0.20 models trailed the leader by ~350 Arena points. Today, ~60. 5. The cast has rotated quite a bit. - - OpenAI set the 2023–24 benchmark. - AI at Meta strengthened the low-cost end in 2024. - Google DeepMind drove the 2025 jump. - Anthropic holds the peak in 2026. - xAI and Chinese labs like DeepSeek AI, Z.ai, Kimi.ai, Xiaomi MiMo, and Qwen are continuing to push the mid-price frontier.

Arena.ai

58,096 次观看 • 1 个月前

LMARENA EVOLUTION OF TOP 10 MODELS BY ELO OVER TIME Since November 20th, only Google and OpenAI models have been in the Top 10 !!! Anthropic hasn't made the Top 10 since September 27th.
0:33

LMARENA EVOLUTION OF TOP 10 MODELS BY ELO OVER TIME Since November 20th, only Google and OpenAI models have been in the Top 10 !!! Anthropic hasn't made the Top 10 since September 27th.

Lisan al Gaib

44,424 次观看 • 1 年前

✨Spotlight: Top 4 Code Arena open models Open models are moving fast, and this community is directly shaping how that progress shows up in real use. Your builds, votes, and experiments surface what frontier models can actually do in practice. To celebrate the momentum, we’re spotlighting the top open models in Code Arena (WebDev) 🧵👇 🔥 MiniMax (official) — MiniMax-M2.1 🤖 Z.ai — GLM-4.7 ✨ DeepSeek — Deepseek-v3.2-thinking 🚀 Xiaomi MiMo — Mimo-v2-flash (non-thinking) Let’s keep building in the open and raising the bar together 🚀
1:07

✨Spotlight: Top 4 Code Arena open models Open models are moving fast, and this community is directly shaping how that progress shows up in real use. Your builds, votes, and experiments surface what frontier models can actually do in practice. To celebrate the momentum, we’re spotlighting the top open models in Code Arena (WebDev) 🧵👇 🔥 MiniMax (official) — MiniMax-M2.1 🤖 Z.ai — GLM-4.7 ✨ DeepSeek — Deepseek-v3.2-thinking 🚀 Xiaomi MiMo — Mimo-v2-flash (non-thinking) Let’s keep building in the open and raising the bar together 🚀

Arena.ai

16,742 次观看 • 5 个月前

4 years ago we were on the brink of AI becoming proprietary and centralized, when OpenAI kept GPT3 closed and VCs started dumping money on researchers. From fully open science, to fully closed, in a matter of months. It was scary, and 1,000+ leading researchers and scientists banded together to show the world that it was possible to do the same work in the open, and build an ecosystem that benefits everyone. That was the BigScience Research Workshop BLOOM project, and it put us back on track to open science, starting with forward-thinking organizations like Meta releasing OPT. Look at us now. Open models have not only caught up, they're state of the art now. Not just LLMs, but models for document AI, speech to text, text to speech, generating images and more. We're closing in on 2 million open weight models on Hugging Face. Thanks for the reminder Thomas Wolf .
2:20

4 years ago we were on the brink of AI becoming proprietary and centralized, when OpenAI kept GPT3 closed and VCs started dumping money on researchers. From fully open science, to fully closed, in a matter of months. It was scary, and 1,000+ leading researchers and scientists banded together to show the world that it was possible to do the same work in the open, and build an ecosystem that benefits everyone. That was the BigScience Research Workshop BLOOM project, and it put us back on track to open science, starting with forward-thinking organizations like Meta releasing OPT. Look at us now. Open models have not only caught up, they're state of the art now. Not just LLMs, but models for document AI, speech to text, text to speech, generating images and more. We're closing in on 2 million open weight models on Hugging Face. Thanks for the reminder Thomas Wolf .

Jeff Boudier 🤗

21,904 次观看 • 1 年前

“We have a crisis of open source models in the Western world. Outside of China, there are no good open source models. We don't even have any in the US now. The talent, the capital and the focus to be best-in-class at pre-training, mid-training, post-training is an extremely scarce skillset. The answer for a lot of countries may just be to take an open weights model from China, post-train or fine-tune their own version, and have that be what they start from." Everett Randle Why does the West have such poor open source models and is it a national security threat to rely on open-source Chinese models Demis Hassabis Yann LeCun Aidan Gomez Lilian Weng Jan Leike
1:20

“We have a crisis of open source models in the Western world. Outside of China, there are no good open source models. We don't even have any in the US now. The talent, the capital and the focus to be best-in-class at pre-training, mid-training, post-training is an extremely scarce skillset. The answer for a lot of countries may just be to take an open weights model from China, post-train or fine-tune their own version, and have that be what they start from." Everett Randle Why does the West have such poor open source models and is it a national security threat to rely on open-source Chinese models Demis Hassabis Yann LeCun Aidan Gomez Lilian Weng Jan Leike

Harry Stebbings

62,130 次观看 • 11 天前

ELON: CHINA HAS THE BEST OPEN SOURCE AI MODELS RIGHT NOW “The best open source models are generally from China, which is bizarre. I think the second best one, or maybe it’s better than second best, is Grok 2.5. The open source model is actually very good, and we will continue to open source our models.” Source: The All-In Podcast
0:47

ELON: CHINA HAS THE BEST OPEN SOURCE AI MODELS RIGHT NOW “The best open source models are generally from China, which is bizarre. I think the second best one, or maybe it’s better than second best, is Grok 2.5. The open source model is actually very good, and we will continue to open source our models.” Source: The All-In Podcast

Mario Nawfal

1,163,582 次观看 • 8 个月前

Today, we’re introducing Forge, a system for enterprises to build frontier-grade AI models grounded in their proprietary knowledge. 🌎 Forge bridges the gap between generic AI and enterprise-specific needs. Instead of relying on broad, public data, organizations can train models that understand their internal context embedded within systems, workflows, and policies, aligning AI with their unique operations. We have already partnered with world-leading organizations, like ASML, DSO National Laboratories Singapore, Ericsson, European Space Agency, Home Team Science and Technology Agency (HTX) Singapore and Reply to train models on the proprietary data that powers their most complex systems and future-defining technologies.
1:28

Today, we’re introducing Forge, a system for enterprises to build frontier-grade AI models grounded in their proprietary knowledge. 🌎 Forge bridges the gap between generic AI and enterprise-specific needs. Instead of relying on broad, public data, organizations can train models that understand their internal context embedded within systems, workflows, and policies, aligning AI with their unique operations. We have already partnered with world-leading organizations, like ASML, DSO National Laboratories Singapore, Ericsson, European Space Agency, Home Team Science and Technology Agency (HTX) Singapore and Reply to train models on the proprietary data that powers their most complex systems and future-defining technologies.

Mistral AI

438,070 次观看 • 3 个月前

CHINA IS CATCHING UP IN AI 🇨🇳 China’s new open-source AI model, GLM-5.2 by Zhipu AI, reportedly matches Anthropic’s Claude Mythos in security bug detection. The model outperformed some Claude versions in independent cybersecurity benchmarks and can achieve Mythos-level performance. Unlike Claude Mythos, GLM-5.2 is open-source and significantly cheaper to run. The gap between US and Chinese AI models is narrowing rapidly.
0:29

CHINA IS CATCHING UP IN AI 🇨🇳 China’s new open-source AI model, GLM-5.2 by Zhipu AI, reportedly matches Anthropic’s Claude Mythos in security bug detection. The model outperformed some Claude versions in independent cybersecurity benchmarks and can achieve Mythos-level performance. Unlike Claude Mythos, GLM-5.2 is open-source and significantly cheaper to run. The gap between US and Chinese AI models is narrowing rapidly.

CryptoGoos

69,848 次观看 • 2 天前

The First 100 begins now. Verify Quality is live on PrismaX. For the first time, anyone can score the robot training data that models learn from, earn points, and compete to become one of The First 100. Better data. Better models. The standard starts with you.
0:34

The First 100 begins now. Verify Quality is live on PrismaX. For the first time, anyone can score the robot training data that models learn from, earn points, and compete to become one of The First 100. Better data. Better models. The standard starts with you.

PrismaX

108,698 次观看 • 7 天前

"One of the biggest misconceptions" Cerebras CFO Bob Komin pushes back on the small-models narrative. "We serve all models, and there is no limit to the size of the models that we can serve. Today, we're serving trillion parameter models. We're serving trillion parameter models that are internal for OpenAI today. We are currently running OpenAI 5.4 and 5.5 with them."
0:50

"One of the biggest misconceptions" Cerebras CFO Bob Komin pushes back on the small-models narrative. "We serve all models, and there is no limit to the size of the models that we can serve. Today, we're serving trillion parameter models. We're serving trillion parameter models that are internal for OpenAI today. We are currently running OpenAI 5.4 and 5.5 with them."

Deirdre Bosa

83,925 次观看 • 1 个月前

JUST IN: 🇨🇳 China’s new open-source AI model, GLM-5.2 by Zhipu AI, reportedly matches Anthropic’s Claude Mythos in security bug detection. The model outperformed some Claude versions in independent cybersecurity benchmarks and can achieve Mythos-level performance. Unlike Claude Mythos, GLM-5.2 is open-source and significantly cheaper to run. The gap between US and Chinese AI models is narrowing rapidly.
0:32

JUST IN: 🇨🇳 China’s new open-source AI model, GLM-5.2 by Zhipu AI, reportedly matches Anthropic’s Claude Mythos in security bug detection. The model outperformed some Claude versions in independent cybersecurity benchmarks and can achieve Mythos-level performance. Unlike Claude Mythos, GLM-5.2 is open-source and significantly cheaper to run. The gap between US and Chinese AI models is narrowing rapidly.

CryptoTweets

61,327 次观看 • 2 天前

AkashChat provides access to 8 of the leading open-source AI models. Switch between models mid-conversation to leverage the strengths of each, while seamlessly maintaining the context. The chat below switches from Llama 3.3 70B to DeepSeek-R1.
1:14

AkashChat provides access to 8 of the leading open-source AI models. Switch between models mid-conversation to leverage the strengths of each, while seamlessly maintaining the context. The chat below switches from Llama 3.3 70B to DeepSeek-R1.

Akash Network

11,964 次观看 • 1 年前

China is once again unsettling its American competitors in the AI race. Chinese labs have returned to the spotlight with the unveiling of GLM 5.2, described as the most powerful AI model ever trained in China — and one of the leading open‑source models worldwide. 🔍 Key highlights of “GLM 5.2”: ♦️ It is considered the strongest AI model developed in China to date, and ranks among the smartest open‑source models available. ♦️ It operates at a cost less than one‑tenth that of Anthropic’s latest models, making it a far more economical option. ♦️ It is likely to attract developers and companies seeking powerful, low‑cost models — especially amid growing technical and geopolitical restrictions. ♦️ Some evaluations place the model among the top globally, trailing OpenAI’s models while outperforming several competitors in specialized benchmarks and performance tests.
0:42

China is once again unsettling its American competitors in the AI race. Chinese labs have returned to the spotlight with the unveiling of GLM 5.2, described as the most powerful AI model ever trained in China — and one of the leading open‑source models worldwide. 🔍 Key highlights of “GLM 5.2”: ♦️ It is considered the strongest AI model developed in China to date, and ranks among the smartest open‑source models available. ♦️ It operates at a cost less than one‑tenth that of Anthropic’s latest models, making it a far more economical option. ♦️ It is likely to attract developers and companies seeking powerful, low‑cost models — especially amid growing technical and geopolitical restrictions. ♦️ Some evaluations place the model among the top globally, trailing OpenAI’s models while outperforming several competitors in specialized benchmarks and performance tests.

China pulse 🇨🇳

14,018 次观看 • 2 天前

Zach Vorhies on the "Zach-Adams Effect" DTV episode, revealing the truth about the strength of Chinese open source models vs. U.S. frontier AI models. (The gap is shrinking fast...) Full episode at (every Friday)
4:36

Zach Vorhies on the "Zach-Adams Effect" DTV episode, revealing the truth about the strength of Chinese open source models vs. U.S. frontier AI models. (The gap is shrinking fast...) Full episode at (every Friday)

HealthRanger

11,053 次观看 • 2 天前

Announcing Artificial Analysis Video Arena - the first crowdsourced comparison for Text to Video models Text to Video models are accelerating rapidly and crossing quality thresholds every month. We created Video Arena to compare them using the only source of truth for visual media - human preference! Video Arena includes hundreds of videos from the leading video models, including: - Runway's Runway Gen 3 Alpha - Pika's Pika 1.5 - Luma's Dream Machine - MiniMax / Hailuo AI-MiniMax Hub - Kling AI's Kling 1.0 - Zhipu AI's CogVideoX-5B Voting is open now and we’ll be announcing the first leaderboard results within 24 hours. Any predictions? In the meantime, you can see your own ‘personal leaderboard’ of how you’ve ranked the video models after 30 votes. Link to the Artificial Analysis Video Arena in the below tweet! 👇
0:20

Announcing Artificial Analysis Video Arena - the first crowdsourced comparison for Text to Video models Text to Video models are accelerating rapidly and crossing quality thresholds every month. We created Video Arena to compare them using the only source of truth for visual media - human preference! Video Arena includes hundreds of videos from the leading video models, including: - Runway's Runway Gen 3 Alpha - Pika's Pika 1.5 - Luma's Dream Machine - MiniMax / Hailuo AI-MiniMax Hub - Kling AI's Kling 1.0 - Zhipu AI's CogVideoX-5B Voting is open now and we’ll be announcing the first leaderboard results within 24 hours. Any predictions? In the meantime, you can see your own ‘personal leaderboard’ of how you’ve ranked the video models after 30 votes. Link to the Artificial Analysis Video Arena in the below tweet! 👇

Artificial Analysis

27,096 次观看 • 1 年前

Earlier this week at GTC, we announced our partnership with Nvidia. We will work with Nvidia to build strong, American open-source models that are at the frontier of scientific reasoning. These models will be essential for the US to compete with China on science in the coming decades. Jensen is committing to spend tens of billions of dollars developing open-source models, and we are excited to be a partner with them in figuring out how to benchmark, train and use those agents to accelerate scientific research. We have already open-sourced some of the work we have done with them, and are looking forward to open-sourcing more. There are few things today that are more important. See our blog post below, and watch the video to learn more, narrated by the man himself.
1:44

Earlier this week at GTC, we announced our partnership with Nvidia. We will work with Nvidia to build strong, American open-source models that are at the frontier of scientific reasoning. These models will be essential for the US to compete with China on science in the coming decades. Jensen is committing to spend tens of billions of dollars developing open-source models, and we are excited to be a partner with them in figuring out how to benchmark, train and use those agents to accelerate scientific research. We have already open-sourced some of the work we have done with them, and are looking forward to open-sourcing more. There are few things today that are more important. See our blog post below, and watch the video to learn more, narrated by the man himself.

Sam Rodriques

23,200 次观看 • 3 个月前

.Bill Gurley “I don’t think politicians understand how powerful open source is as a competitive weapon. If you have ten companies all doing open source, the minute an idea is shared, it's available to the other ten and they absorb it immediately. I think the odds that the U.S., with two proprietary models, can stay ahead of China, with ten open source ones, is very low.”
1:48

.Bill Gurley “I don’t think politicians understand how powerful open source is as a competitive weapon. If you have ten companies all doing open source, the minute an idea is shared, it's available to the other ten and they absorb it immediately. I think the odds that the U.S., with two proprietary models, can stay ahead of China, with ten open source ones, is very low.”

Codie Sanchez

65,669 次观看 • 3 个月前

We’re excited to partner with OpenAI to launch their new open source models natively on Databricks! gpt-oss sets a new standard of quality for open language models, supporting advanced reasoning with the transparency, flexibility and control enterprises need. Running on Databricks, the gpt-oss models connect securely to your data and scale with built-in governance, and expand what you can build and do with GenAI. Try both the 20B and 120B today in the Mosaic AI Playground.
0:39

We’re excited to partner with OpenAI to launch their new open source models natively on Databricks! gpt-oss sets a new standard of quality for open language models, supporting advanced reasoning with the transparency, flexibility and control enterprises need. Running on Databricks, the gpt-oss models connect securely to your data and scale with built-in governance, and expand what you can build and do with GenAI. Try both the 20B and 120B today in the Mosaic AI Playground.

Databricks

10,086 次观看 • 10 个月前

Something cracked in 2016. Since the late 1970s, the share of teenagers unsure about having children has consistently stayed below 20%. For decades, the gap between boys and girls remained very small, typically just 2 percentage points or less. Boys’ uncertainty has stayed largely stable over time. But girls’ uncertainty skyrocketed after 2016, widening the gender gap to 5 percentage points and continuing to grow, reaching a 9-point difference by 2024. The bottom line is boys haven’t changed much. Girls have.
0:26

Something cracked in 2016. Since the late 1970s, the share of teenagers unsure about having children has consistently stayed below 20%. For decades, the gap between boys and girls remained very small, typically just 2 percentage points or less. Boys’ uncertainty has stayed largely stable over time. But girls’ uncertainty skyrocketed after 2016, widening the gender gap to 5 percentage points and continuing to grow, reaching a 9-point difference by 2024. The bottom line is boys haven’t changed much. Girls have.

Rothmus 🏴

16,915 次观看 • 1 个月前