正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

How good are frontier models at writing Polytopia bots? Introducing Polytopia Bench. 12 models each get 40 minutes to write a Rust bot from scratch.

Kevin Z

9,382 subscribers

33,271 次观看 • 9 天前 •via X (Twitter)

科学技术游戏

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

How it feels using the latest AI tools to summarize earnings transcripts and create excel models from scratch in minutes (your fund is down 26% YTD tho)

How it feels using the latest AI tools to summarize earnings transcripts and create excel models from scratch in minutes (your fund is down 26% YTD tho)

litquidity

102,237 次观看 • 1 年前

8 coding agents (written in rust) who are each writing rust for a simulator while I scroll Grokipedia.

8 coding agents (written in rust) who are each writing rust for a simulator while I scroll Grokipedia.

Elliot Arledge

26,976 次观看 • 1 年前

Agent Arena measures complex, real-world tasks. Data shows that these long-horizon sessions amplify differences in frontier models. Long-horizon tasks in Agent Arena span hundreds of turns, from porting a codebase to Rust and auditing an investment strategy document to writing a math textbook, and more. Comparing the top model from each of the 5 leading labs on Agent Arena reveals a 55-minute gap in how long they spend on long-horizon tasks. The models are: - Claude Opus 5.5 by Anthropic - Gemini 4 Carbon by Google DeepMind - GPT-6 Astra by OpenAI - Muse Wonder Full by AI at Meta - Kimi K3.1 Pro by Kimi.ai In this group, Claude Opus 5.5, #1 in Agent Arena, takes the longest at every task length, peaking at 1 hour 51 minutes. In contrast, Kimi K3.1 Pro spends the least time at every length, with 56 minutes on long-horizon tasks.

Agent Arena measures complex, real-world tasks. Data shows that these long-horizon sessions amplify differences in frontier models. Long-horizon tasks in Agent Arena span hundreds of turns, from porting a codebase to Rust and auditing an investment strategy document to writing a math textbook, and more. Comparing the top model from each of the 5 leading labs on Agent Arena reveals a 55-minute gap in how long they spend on long-horizon tasks. The models are: - Claude Opus 5.5 by Anthropic - Gemini 4 Carbon by Google DeepMind - GPT-6 Astra by OpenAI - Muse Wonder Full by AI at Meta - Kimi K3.1 Pro by Kimi.ai In this group, Claude Opus 5.5, #1 in Agent Arena, takes the longest at every task length, peaking at 1 hour 51 minutes. In contrast, Kimi K3.1 Pro spends the least time at every length, with 56 minutes on long-horizon tasks.

Arena.ai

29,334 次观看 • 7 天前

Grok 4.1 Fast outperforms all major frontier models on τ²-Bench Telecom for agentic tool use Beating the newest models from Google (Gemini 3 Pro), Anthropic (Claude Opus 4.5), and OpenAI (GPT 5.2 xhigh)

Grok 4.1 Fast outperforms all major frontier models on τ²-Bench Telecom for agentic tool use Beating the newest models from Google (Gemini 3 Pro), Anthropic (Claude Opus 4.5), and OpenAI (GPT 5.2 xhigh)

X Freeze

41,019 次观看 • 10 个月前

From frogs to the frontier, Scott Wu covered a lot at Runtime: → Agents are moving from tab completion to “virtual employees” that own outcomes → Frontier models may only need ~2% of workloads; routing handles the rest → 91% of Cognition’s internal Devin sessions start without a human

From frogs to the frontier, Scott Wu covered a lot at Runtime: → Agents are moving from tab completion to “virtual employees” that own outcomes → Frontier models may only need ~2% of workloads; routing handles the rest → 91% of Cognition’s internal Devin sessions start without a human

Modal

47,471 次观看 • 1 个月前

We're open sourcing a 9B model that extracts structured data from documents at near-frontier performance. - 90.2% on our bench, vs Gemini 3.5 Flash at 91.3% - Leads extraction models like NuExtract3 (81.5%) - 9.5s p50 timings - Pass JSON schema

We're open sourcing a 9B model that extracts structured data from documents at near-frontier performance. - 90.2% on our bench, vs Gemini 3.5 Flash at 91.3% - Leads extraction models like NuExtract3 (81.5%) - 9.5s p50 timings - Pass JSON schema

Vik Paruchuri

229,345 次观看 • 4 个月前

Intelligence should not have a gatekeeper. Today, 40+ frontier and open models stand behind a single endpoint. Gata Inference is live.

Intelligence should not have a gatekeeper. Today, 40+ frontier and open models stand behind a single endpoint. Gata Inference is live.

Gata

18,190 次观看 • 3 个月前

Introducing ReactBench A benchmark for coding agents on real React work Models write bad React code - useEffect, slow performance, memory leaks

Introducing ReactBench A benchmark for coding agents on real React work Models write bad React code - useEffect, slow performance, memory leaks

Aiden Bai

345,103 次观看 • 3 个月前

introducing Krea for Grok bot. generate images and video with the best models in the world, right from Grok. try it now 👇

introducing Krea for Grok bot. generate images and video with the best models in the world, right from Grok. try it now 👇

Krea

242,668 次观看 • 19 天前

Today we're introducing Verda Inference, in preview, with day-0 access to Kimi K3.1 Pro, in partnership with Kimi.ai 🧵 Frontier open-weight models through one API, engineered for every request, from a single prompt to long-horizon agents.

Today we're introducing Verda Inference, in preview, with day-0 access to Kimi K3.1 Pro, in partnership with Kimi.ai 🧵 Frontier open-weight models through one API, engineered for every request, from a single prompt to long-horizon agents.

Verda

444,375 次观看 • 7 天前

Introducing SVG Arena! SVGs are XML-based text files that encode geometry, and they're surprisingly tough for state-of-the-art LLMs to get right Here's how bad models (still) are at drawing a pelican on a bicycle - creds to Simon Willison Leaderboard coming soon! In the meantime, here are some recent generations

Introducing SVG Arena! SVGs are XML-based text files that encode geometry, and they're surprisingly tough for state-of-the-art LLMs to get right Here's how bad models (still) are at drawing a pelican on a bicycle - creds to Simon Willison Leaderboard coming soon! In the meantime, here are some recent generations

Design Arena

29,136 次观看 • 10 个月前

Language models understand natural proteins. But can they generalize beyond, to design completely new proteins from scratch? New preprint: A 🧵

Language models understand natural proteins. But can they generalize beyond, to design completely new proteins from scratch? New preprint: A 🧵

Tom Sercu

275,285 次观看 • 3 年前

Even when you get a D rank he looks cool. Sonic team master how to make this crusty models look good good for them lmao

Even when you get a D rank he looks cool. Sonic team master how to make this crusty models look good good for them lmao

Ace super💥

283,632 次观看 • 2 年前

I did it. I built "Cursor for Writing". Introducing Lucid ✨ - Write at the speed of thought - Pull in context from anywhere - Chat + Deep Research

I did it. I built "Cursor for Writing". Introducing Lucid ✨ - Write at the speed of thought - Pull in context from anywhere - Chat + Deep Research

Ray

443,911 次观看 • 1 年前

Today we're excited to announce a major collaboration with Takeda to deploy our frontier models across their entire organization. Takeda scientists now get access to BoltzMol-1, BoltzProt-1, and fine-tuned models for biomolecular modeling and design.

Today we're excited to announce a major collaboration with Takeda to deploy our frontier models across their entire organization. Takeda scientists now get access to BoltzMol-1, BoltzProt-1, and fine-tuned models for biomolecular modeling and design.

Boltz

33,078 次观看 • 4 个月前

A major question in multimodal modeling is how to leverage strong pre-trained models, such as industry-scale LLMs and VLMs, during training to avoid starting from scratch. This is important as the demand for rich and domain-specific multimodal models continues to increase, while training them from scratch is obviously impractical due to limited data and compute. The so-called “any-to-any” multimodal models can model a large diverse dictionary of modalities. That’s good. Their downside is that their architectures are often not decoder-only, which has limited their performance in practice and prevents them from leveraging strong pre-trained decoder-only models as priors. We are releasing MODUS (ICML Conference '26) a decoder-only any-to-any multimodal model to address some of these questions. A single transformer decoder predicts any modality from any others with no modality-specific heads, losses, or task pipelines. We show efficient adaptation of established models, e.g., BAGEL, to rich any-to-any multimodal modeling. We are releasing 14B to 77B parameter models. All materials are open-source. The download links and demos here 🧵

A major question in multimodal modeling is how to leverage strong pre-trained models, such as industry-scale LLMs and VLMs, during training to avoid starting from scratch. This is important as the demand for rich and domain-specific multimodal models continues to increase, while training them from scratch is obviously impractical due to limited data and compute. The so-called “any-to-any” multimodal models can model a large diverse dictionary of modalities. That’s good. Their downside is that their architectures are often not decoder-only, which has limited their performance in practice and prevents them from leveraging strong pre-trained decoder-only models as priors. We are releasing MODUS (ICML Conference '26) a decoder-only any-to-any multimodal model to address some of these questions. A single transformer decoder predicts any modality from any others with no modality-specific heads, losses, or task pipelines. We show efficient adaptation of established models, e.g., BAGEL, to rich any-to-any multimodal modeling. We are releasing 14B to 77B parameter models. All materials are open-source. The download links and demos here 🧵

Amir Zamir

24,521 次观看 • 4 个月前

Qwen 3.7-max beats Opus 4.7 and GPT-5.5 We tested three frontier models on a real agentic task: write a Tetris bot that plays the game and trains itself. Each model could read its own code, run benchmarks, and rewrite itself across 10 iterations. Then we compared the final bots head to head. Qwen 3.7-Max: training cost $1.32, bot improvement +56% Claude Opus 4.7: training cost $12.15, bot improvement +28% GPT-5.5: training cost $2.85, bot improvement +7% Qwen won on every dimension - biggest jump, 9× cheaper than Claude, 2× cheaper than GPT. Long agentic loops is where Qwen Max actually delivers.

Qwen 3.7-max beats Opus 4.7 and GPT-5.5 We tested three frontier models on a real agentic task: write a Tetris bot that plays the game and trains itself. Each model could read its own code, run benchmarks, and rewrite itself across 10 iterations. Then we compared the final bots head to head. Qwen 3.7-Max: training cost $1.32, bot improvement +56% Claude Opus 4.7: training cost $12.15, bot improvement +28% GPT-5.5: training cost $2.85, bot improvement +7% Qwen won on every dimension - biggest jump, 9× cheaper than Claude, 2× cheaper than GPT. Long agentic loops is where Qwen Max actually delivers.

atomic.chat

871,419 次观看 • 5 个月前