Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Today we’re releasing APK Arena: a frontier benchmark for evaluating AI agents on phone-use. We ran GPT 5.6 for a side by side comparison where it outperformed other models across the board and is currently leading on APK Arena. 50+ levels: taps, vision, memory, games, real tasks. Here's how... show more

Jan Schnyder

2,312 subscribers

10,107 görüntüleme • 2 ay önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Today we're shipping Nemotron 3 Ultra. A 550B MoE frontier-intelligence open model built for long-running agents. It delivers 5x faster inference and lowers the cost of complex agentic tasks by up to 30% versus other open frontier models.

Today we're shipping Nemotron 3 Ultra. A 550B MoE frontier-intelligence open model built for long-running agents. It delivers 5x faster inference and lowers the cost of complex agentic tasks by up to 30% versus other open frontier models.

NVIDIA AI

1,329,682 görüntüleme • 3 ay önce

Dropping a new AI benchmark: TetrisBench and a human vs AI Tetris arena 🕹️ See how you rank against the frontier models. Only 1 human has beaten a model so far, so try it out be the 2nd! Stats so far: Opus 4.5 > GPT5.2 > Sonnet > Grok > most humans 🧵

Dropping a new AI benchmark: TetrisBench and a human vs AI Tetris arena 🕹️ See how you rank against the frontier models. Only 1 human has beaten a model so far, so try it out be the 2nd! Stats so far: Opus 4.5 > GPT5.2 > Sonnet > Grok > most humans 🧵

Yoko

35,709 görüntüleme • 8 ay önce

Databricks is excited to partner with OpenAI on GPT-5.5, their latest frontier model. GPT-5.5 will be available in Unity AI Gateway on launch. You can use it with coding tools such as Codex, or to power your enterprise agents. GPT-5.5 is state-of-the-art on many benchmarks including OfficeQA Pro, our benchmark for evaluating grounded reasoning on enterprise tasks. We are partnering with OpenAI to co-launch on Databricks. Hear more from our co-founder Patrick Wendell and OpenAI CRO Denise Holland Dresser on GPT-5.5 in Databricks.

Databricks is excited to partner with OpenAI on GPT-5.5, their latest frontier model. GPT-5.5 will be available in Unity AI Gateway on launch. You can use it with coding tools such as Codex, or to power your enterprise agents. GPT-5.5 is state-of-the-art on many benchmarks including OfficeQA Pro, our benchmark for evaluating grounded reasoning on enterprise tasks. We are partnering with OpenAI to co-launch on Databricks. Hear more from our co-founder Patrick Wendell and OpenAI CRO Denise Holland Dresser on GPT-5.5 in Databricks.

Databricks

12,917 görüntüleme • 4 ay önce

Today, we’re introducing Decide Agent, a frontier-level AI agent for Excel analysis. Decide Agent scored 82.50% on SpreadsheetBench, the public standard benchmark for Excel AI agents, ranking #4 globally. This is the same leaderboard used by Claude, OpenAI, and Microsoft to evaluate their spreadsheet agents. We’re here to push the frontier in Excel and data analytics. Decide Agent is now generally available at trydecide . ai

Today, we’re introducing Decide Agent, a frontier-level AI agent for Excel analysis. Decide Agent scored 82.50% on SpreadsheetBench, the public standard benchmark for Excel AI agents, ranking #4 globally. This is the same leaderboard used by Claude, OpenAI, and Microsoft to evaluate their spreadsheet agents. We’re here to push the frontier in Excel and data analytics. Decide Agent is now generally available at trydecide . ai

Ab.

40,512 görüntüleme • 7 ay önce

Muse Spark claims to outperform leading AI models like Opus, GPT, and Gemini. We put it through a real-world finance stress test. Beep agents can now use Muse Spark for agentic trading on Sui. Deploy now: 🧵

Muse Spark claims to outperform leading AI models like Opus, GPT, and Gemini. We put it through a real-world finance stress test. Beep agents can now use Muse Spark for agentic trading on Sui. Deploy now: 🧵

Beep

96,829 görüntüleme • 2 ay önce

Arena reached a $100M annual revenue run rate just 8 months after launching our evaluation product. We started as a research project at UC Berkeley with a simple mission: measure AI progress through real-world use. As AI shifts from chatbots to agents taking on longer, higher-stakes work, the problem matters more than ever. Today, Arena measures real-world AI utility with a community of tens of millions. With Agent Arena, we’re evaluating long-running agents on complex, real-world tasks - how they use tools, adapt to feedback, recover from errors, and accomplish goals set by humans. We are excited to keep deepening our work in agentic evaluations. Here’s Anastasios Nikolas Angelopoulos on what this milestone means and where we go from here:

Arena reached a $100M annual revenue run rate just 8 months after launching our evaluation product. We started as a research project at UC Berkeley with a simple mission: measure AI progress through real-world use. As AI shifts from chatbots to agents taking on longer, higher-stakes work, the problem matters more than ever. Today, Arena measures real-world AI utility with a community of tens of millions. With Agent Arena, we’re evaluating long-running agents on complex, real-world tasks - how they use tools, adapt to feedback, recover from errors, and accomplish goals set by humans. We are excited to keep deepening our work in agentic evaluations. Here’s Anastasios Nikolas Angelopoulos on what this milestone means and where we go from here:

Arena.ai

212,256 görüntüleme • 2 ay önce

GLM 5.3, GPT 5.6 Luna, Claude Opus 5, Grok 4.6, DeepSeek V4, Kimi K3, and a cloud browser for agents - all FREE. > DuckDuckGo: GPT 5.6 Luna and older models, no signup: > LM Arena: free side-by-side, Opus 5, GPT-5.6 Sol, Grok 4.6, Qwen 3.8 Max: > Nvidia NIM: Kimi K3 free, 40 requests/minute: > OpenCode: 7 free models including DeepSeek V4 Flash, no API key: > Bright Data: 5K credits/month, cloud browser, MCP for agents: Anthropic just dropped Fable 5.1 today - new benchmark leader. Meanwhile the free tier keeps growing. The gap between paid and open keeps shrinking.

GLM 5.3, GPT 5.6 Luna, Claude Opus 5, Grok 4.6, DeepSeek V4, Kimi K3, and a cloud browser for agents - all FREE. > DuckDuckGo: GPT 5.6 Luna and older models, no signup: > LM Arena: free side-by-side, Opus 5, GPT-5.6 Sol, Grok 4.6, Qwen 3.8 Max: > Nvidia NIM: Kimi K3 free, 40 requests/minute: > OpenCode: 7 free models including DeepSeek V4 Flash, no API key: > Bright Data: 5K credits/month, cloud browser, MCP for agents: Anthropic just dropped Fable 5.1 today - new benchmark leader. Meanwhile the free tier keeps growing. The gap between paid and open keeps shrinking.

ＺＥＦＩ

29,454 görüntüleme • 18 gün önce

🚨 New Models Alert: WebDev 💻 GPT-5-Codex and Qwen3-Coder-Plus are both now available on WebDev Arena! In the WebDev Arena, you can test out all the best frontier AI coding models on web development tasks. Vote for your preferred response and see how they stack up on the leaderboard. Prompt: "build a 90s inspired resume website for "Ella Marina", she is a front end web developer with a love for Lisa Frank and late 1990s R&B music. She works on front end development at LMArena"

🚨 New Models Alert: WebDev 💻 GPT-5-Codex and Qwen3-Coder-Plus are both now available on WebDev Arena! In the WebDev Arena, you can test out all the best frontier AI coding models on web development tasks. Vote for your preferred response and see how they stack up on the leaderboard. Prompt: "build a 90s inspired resume website for "Ella Marina", she is a front end web developer with a love for Lisa Frank and late 1990s R&B music. She works on front end development at LMArena"

Arena.ai

21,123 görüntüleme • 1 yıl önce

Robotics: coding agents’ next frontier. So how good are they? We introduce CaP-X: an open-source framework and benchmark for coding agents, where they write code for robot perception and control, execute it on sim and real robots, observe the outcomes, and iteratively improve code reliability. From NVIDIA Berkeley AI Research CMU Robotics Institute Stanford AI Lab 🧵

Robotics: coding agents’ next frontier. So how good are they? We introduce CaP-X: an open-source framework and benchmark for coding agents, where they write code for robot perception and control, execute it on sim and real robots, observe the outcomes, and iteratively improve code reliability. From NVIDIA Berkeley AI Research CMU Robotics Institute Stanford AI Lab 🧵

Max Fu

179,196 görüntüleme • 5 ay önce

Introducing Agent Mode: Agentic AI is now measured in the Arena. Agent Mode can do deep research, create reports, generate images, build websites, debug code, and more. It completes more complex tasks by using tools like web search, bash in a sandbox environment, image generation, file writing, and asking follow-up questions. Frontier models are waiting for you in Agent Mode to take on real-world tasks. GPT-5.5, Claude Opus 4.7, Gemini 3.1 Pro, and top open models. Test them yourself.

Introducing Agent Mode: Agentic AI is now measured in the Arena. Agent Mode can do deep research, create reports, generate images, build websites, debug code, and more. It completes more complex tasks by using tools like web search, bash in a sandbox environment, image generation, file writing, and asking follow-up questions. Frontier models are waiting for you in Agent Mode to take on real-world tasks. GPT-5.5, Claude Opus 4.7, Gemini 3.1 Pro, and top open models. Test them yourself.

Arena.ai

505,797 görüntüleme • 3 ay önce

🚨BIG NEWS: 🎬 Video Arena is now live on the web! Test out Veo 3.1, Sora 2, Seedance v1.5 Pro, Kling 2.6 Pro, Wan 2.5 & more. What started last summer as a small Discord bot experiment has grown into a rigorous way to measure and understand how frontier video models perform with real-world use. Thank you to our wonderful community for all the feedback! Today, we’re opening up access by making it available on the web. 🎥 Generate videos with 15 different frontier AI models and compare them head-to-head. 📊 Vote for the best output to power the leaderboards.

🚨BIG NEWS: 🎬 Video Arena is now live on the web! Test out Veo 3.1, Sora 2, Seedance v1.5 Pro, Kling 2.6 Pro, Wan 2.5 & more. What started last summer as a small Discord bot experiment has grown into a rigorous way to measure and understand how frontier video models perform with real-world use. Thank you to our wonderful community for all the feedback! Today, we’re opening up access by making it available on the web. 🎥 Generate videos with 15 different frontier AI models and compare them head-to-head. 📊 Vote for the best output to power the leaderboards.

Arena.ai

63,006 görüntüleme • 8 ay önce

Today we announced the Kaggle Game Arena, a new benchmarking platform where AI models and agents can compete head-to-head in strategic games, starting with chess ♟️. Why games, you ask? 🤔 Games are perfect for AI evaluation because they help us understand how models tackle complex reasoning tasks. Many games are a proxy for real-world skills and can test a model's ability in areas like strategic planning, adaptation, and memory. To inaugurate the Game Arena, we're launching an AI chess exhibition tournament. The world’s leading AI models will battle in head-to-head games in a multi-day event running from August 5-7 with streamed games happening daily. Tune-in to root for your favorites at

Today we announced the Kaggle Game Arena, a new benchmarking platform where AI models and agents can compete head-to-head in strategic games, starting with chess ♟️. Why games, you ask? 🤔 Games are perfect for AI evaluation because they help us understand how models tackle complex reasoning tasks. Many games are a proxy for real-world skills and can test a model's ability in areas like strategic planning, adaptation, and memory. To inaugurate the Game Arena, we're launching an AI chess exhibition tournament. The world’s leading AI models will battle in head-to-head games in a multi-day event running from August 5-7 with streamed games happening daily. Tune-in to root for your favorites at

Google AI

132,302 görüntüleme • 1 yıl önce

The teams shipping AI agents right now are bleeding money on the dumbest possible expense: teaching a 400B-parameter model to read a file name. Every time an AI agent needs to "see" something today, it routes an image through a frontier model. OCR, object detection, checking if a button exists on screen. You're paying GPT-4o or Claude pricing for tasks that require perception, not reasoning. One agent workflow processing a few thousand screenshots per day can burn through more on vision calls than on the actual thinking. Perceptron's Isaac is 2B parameters. Built by the team that created Meta's Chameleon multimodal models. On perceptive benchmarks, it matches or beats models 50x its size. The VQA, OCR, and object detection scores are competitive with models running on infrastructure that costs orders of magnitude more. The MCP wrapper is the distribution play. One install command and every Claude Code agent can offload vision tasks to a model that runs on a single consumer GPU. The agent keeps its reasoning in the frontier model and routes perception to a specialist. That split is how you get vision-heavy agent workflows from "technically possible but expensive" to "cheap enough to run on everything." This is the same pattern that won in every other compute-intensive stack. General-purpose handles orchestration. Specialists handle the heavy lifting. Graphics went through it. Audio went through it. Video encoding went through it. Vision in AI agents is next. The teams building agents that see 10,000 images a day will care about this before anyone else does.

The teams shipping AI agents right now are bleeding money on the dumbest possible expense: teaching a 400B-parameter model to read a file name. Every time an AI agent needs to "see" something today, it routes an image through a frontier model. OCR, object detection, checking if a button exists on screen. You're paying GPT-4o or Claude pricing for tasks that require perception, not reasoning. One agent workflow processing a few thousand screenshots per day can burn through more on vision calls than on the actual thinking. Perceptron's Isaac is 2B parameters. Built by the team that created Meta's Chameleon multimodal models. On perceptive benchmarks, it matches or beats models 50x its size. The VQA, OCR, and object detection scores are competitive with models running on infrastructure that costs orders of magnitude more. The MCP wrapper is the distribution play. One install command and every Claude Code agent can offload vision tasks to a model that runs on a single consumer GPU. The agent keeps its reasoning in the frontier model and routes perception to a specialist. That split is how you get vision-heavy agent workflows from "technically possible but expensive" to "cheap enough to run on everything." This is the same pattern that won in every other compute-intensive stack. General-purpose handles orchestration. Specialists handle the heavy lifting. Graphics went through it. Audio went through it. Video encoding went through it. Vision in AI agents is next. The teams building agents that see 10,000 images a day will care about this before anyone else does.

Aakash Gupta

55,978 görüntüleme • 5 ay önce

GPT-6 Astra is now the best-performing frontier model for antibody developability prediction in our benchmark. 🧬 Finding an antibody that binds is only the beginning. Will it aggregate? Will it stay stable? Can it actually behave like a drug? GPT-6 Astra predicts these developability properties better than the other frontier models we tested. 🤯 And yes, the interactive visualization showing how an antibody actually works was also built with Astra in about 1 hour. This model looks seriously powerful. Congrats gbt Sam Altman #insilicoSOTAFM

GPT-6 Astra is now the best-performing frontier model for antibody developability prediction in our benchmark. 🧬 Finding an antibody that binds is only the beginning. Will it aggregate? Will it stay stable? Can it actually behave like a drug? GPT-6 Astra predicts these developability properties better than the other frontier models we tested. 🤯 And yes, the interactive visualization showing how an antibody actually works was also built with Astra in about 1 hour. This model looks seriously powerful. Congrats gbt Sam Altman #insilicoSOTAFM

Andrew Aiginin

309,071 görüntüleme • 9 gün önce

Use Case 3: One-Shot Blindfold Chess Can an AI hold an entire game state in memory without drifting? To test Fugu Ultra’s persona stability and sustained memory, we had it play 4 back-to-back games of blindfold chess. Every model played the same way: no board shown, requiring them to hold the full game state entirely in memory. We matched Fugu Ultra against 3 leading frontier models and a 2100-Elo Stockfish engine. The Results: Fugu Ultra outplayed all 4 opponents. Where the other models eventually drifted or lost track of the board state, Fugu remained accurate, ending every single game in checkmate. Watch the full sequence below to see Fugu capitalize the moment the other models slip.

Use Case 3: One-Shot Blindfold Chess Can an AI hold an entire game state in memory without drifting? To test Fugu Ultra’s persona stability and sustained memory, we had it play 4 back-to-back games of blindfold chess. Every model played the same way: no board shown, requiring them to hold the full game state entirely in memory. We matched Fugu Ultra against 3 leading frontier models and a 2100-Elo Stockfish engine. The Results: Fugu Ultra outplayed all 4 opponents. Where the other models eventually drifted or lost track of the board state, Fugu remained accurate, ending every single game in checkmate. Watch the full sequence below to see Fugu capitalize the moment the other models slip.

Sakana AI

125,318 görüntüleme • 2 ay önce

Stop paying for three AI subscriptions just to use the best models. One private platform gives you all of them, and it never records your prompts. Use chat, images, video, even a video editor. Every frontier model, live on Part 6: The AI Talk

Stop paying for three AI subscriptions just to use the best models. One private platform gives you all of them, and it never records your prompts. Use chat, images, video, even a video editor. Every frontier model, live on Part 6: The AI Talk

OpenGradient (∇, ∇)

14,636 görüntüleme • 2 ay önce

I found an AI browser that is actually awesome FlowithOS is a web browser COMPLETELY powered by AI You give it tasks, the agents work for you in the background It's like an army of AI agents working for you 24/7 Here's a guide on how it works and how you can get started today

I found an AI browser that is actually awesome FlowithOS is a web browser COMPLETELY powered by AI You give it tasks, the agents work for you in the background It's like an army of AI agents working for you 24/7 Here's a guide on how it works and how you can get started today

Alex Finn

25,490 görüntüleme • 9 ay önce

Introducing AI Lab Monetization Play. AI Lab Monetization Play tracks the revenue run-rate of major AI laboratories in real time using public and alternative data signals. The Play provides a continuously updated view of commercial scale across the frontier AI landscape. ARR Dashboard The dashboard estimates annual recurring revenue for Anthropic, OpenAI, and other frontier labs. Each estimate is broken down by revenue stream, growth trajectory, and implied valuation multiple. It surfaces which labs are scaling fastest and where the commercial momentum is concentrated. Model Routing and Market Share The Play tracks real-time model usage share across inference providers. It measures routing volume through aggregators to identify which models are capturing production workloads. Shifts in routing patterns signal adoption inflections before they appear in reported financials. Model Pricing and Capabilities The Play maintains a comprehensive pricing table across all major foundation models. Input and output token costs, context windows, and capability tiers are mapped side by side. It serves as the reference layer for competitive positioning and margin structure across the model ecosystem. Available to all paid clients on our full-stack research platform. Start here:

Introducing AI Lab Monetization Play. AI Lab Monetization Play tracks the revenue run-rate of major AI laboratories in real time using public and alternative data signals. The Play provides a continuously updated view of commercial scale across the frontier AI landscape. ARR Dashboard The dashboard estimates annual recurring revenue for Anthropic, OpenAI, and other frontier labs. Each estimate is broken down by revenue stream, growth trajectory, and implied valuation multiple. It surfaces which labs are scaling fastest and where the commercial momentum is concentrated. Model Routing and Market Share The Play tracks real-time model usage share across inference providers. It measures routing volume through aggregators to identify which models are capturing production workloads. Shifts in routing patterns signal adoption inflections before they appear in reported financials. Model Pricing and Capabilities The Play maintains a comprehensive pricing table across all major foundation models. Input and output token costs, context windows, and capability tiers are mapped side by side. It serves as the reference layer for competitive positioning and margin structure across the model ecosystem. Available to all paid clients on our full-stack research platform. Start here:

FUNDA

37,874 görüntüleme • 2 ay önce

Introducing AttuneBench! We built this benchmark on a simple premise: for self-improving AI to reach its full usefulness to humanity, it needs high EQ. We decomposed EQ into distinct skills and evaluated 11 frontier models across 50 real-life topics, from relationships and marriage to school and job stress, using 50,000+ first-person annotations.

Introducing AttuneBench! We built this benchmark on a simple premise: for self-improving AI to reach its full usefulness to humanity, it needs high EQ. We decomposed EQ into distinct skills and evaluated 11 frontier models across 50 real-life topics, from relationships and marriage to school and job stress, using 50,000+ first-person annotations.

Thoughtful

19,457 görüntüleme • 3 ay önce

So how good is gpt-oss at function calling for real use cases and how to implement it correctly on Groq? Meet Autosheet: a GroqLabs project to showcase gpt-oss models running on GroqInc! It's a browser spreadsheet driven by AI with built-in MCP server support. A video & 🧵

So how good is gpt-oss at function calling for real use cases and how to implement it correctly on Groq? Meet Autosheet: a GroqLabs project to showcase gpt-oss models running on GroqInc! It's a browser spreadsheet driven by AI with built-in MCP server support. A video & 🧵

Rick Lamers

26,740 görüntüleme • 1 yıl önce