正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

When evaluated against the WebVoyager benchmark, which tests agent performance on end-to-end real world web tasks, Project Mariner achieved a state-of-the-art result of 83.5% working as a single agent setup.

Google DeepMind

1,125,017 subscribers

23,958 次观看 • 1 年前 •via X (Twitter)

新闻政治科学技术

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Microsoft presents Windows Agent Arena Evaluating Multi-Modal OS Agents at Scale discuss: Large language models (LLMs) show remarkable potential to act as computer agents, enhancing human productivity and software accessibility in multi-modal tasks that require planning and reasoning. However, measuring agent performance in realistic environments remains a challenge since: (i) most benchmarks are limited to specific modalities or domains (e.g. text-only, web navigation, Q&A, coding) and (ii) full benchmark evaluations are slow (on order of magnitude of days) given the multi-step sequential nature of tasks. To address these challenges, we introduce the Windows Agent Arena: a reproducible, general environment focusing exclusively on the Windows operating system (OS) where agents can operate freely within a real Windows OS and use the same wide range of applications, tools, and web browsers available to human users when solving tasks. We adapt the OSWorld framework (Xie et al., 2024) to create 150+ diverse Windows tasks across representative domains that require agent abilities in planning, screen understanding, and tool usage. Our benchmark is scalable and can be seamlessly parallelized in Azure for a full benchmark evaluation in as little as 20 minutes. To demonstrate Windows Agent Arena's capabilities, we also introduce a new multi-modal agent, Navi. Our agent achieves a success rate of 19.5% in the Windows domain, compared to 74.5% performance of an unassisted human. Navi also demonstrates strong performance on another popular web-based benchmark, Mind2Web. We offer extensive quantitative and qualitative analysis of Navi's performance, and provide insights into the opportunities for future research in agent development and data generation using Windows Agent Arena.

Microsoft presents Windows Agent Arena Evaluating Multi-Modal OS Agents at Scale discuss: Large language models (LLMs) show remarkable potential to act as computer agents, enhancing human productivity and software accessibility in multi-modal tasks that require planning and reasoning. However, measuring agent performance in realistic environments remains a challenge since: (i) most benchmarks are limited to specific modalities or domains (e.g. text-only, web navigation, Q&A, coding) and (ii) full benchmark evaluations are slow (on order of magnitude of days) given the multi-step sequential nature of tasks. To address these challenges, we introduce the Windows Agent Arena: a reproducible, general environment focusing exclusively on the Windows operating system (OS) where agents can operate freely within a real Windows OS and use the same wide range of applications, tools, and web browsers available to human users when solving tasks. We adapt the OSWorld framework (Xie et al., 2024) to create 150+ diverse Windows tasks across representative domains that require agent abilities in planning, screen understanding, and tool usage. Our benchmark is scalable and can be seamlessly parallelized in Azure for a full benchmark evaluation in as little as 20 minutes. To demonstrate Windows Agent Arena's capabilities, we also introduce a new multi-modal agent, Navi. Our agent achieves a success rate of 19.5% in the Windows domain, compared to 74.5% performance of an unassisted human. Navi also demonstrates strong performance on another popular web-based benchmark, Mind2Web. We offer extensive quantitative and qualitative analysis of Navi's performance, and provide insights into the opportunities for future research in agent development and data generation using Windows Agent Arena.

AK

19,684 次观看 • 1 年前

Finally, we're so excited to bring Project Mariner capabilities to AI Mode in Search, Agent Mode in Google Gemini, and as a standalone web app. This will allow the products you know and love to intelligently manage complex multi-step tasks on your behalf.

Finally, we're so excited to bring Project Mariner capabilities to AI Mode in Search, Agent Mode in Google Gemini, and as a standalone web app. This will allow the products you know and love to intelligently manage complex multi-step tasks on your behalf.

Google AI

29,189 次观看 • 1 年前

I set a Ralph loop using agent-browser + Parallel Web Systems to do a ton of market research. Perfect way to have an agent working while I do other things. Decided to setup my iMac for a lot of the looping tasks because it makes it easier to keep the repo clones and work product separated.

I set a Ralph loop using agent-browser + Parallel Web Systems to do a ton of market research. Perfect way to have an agent working while I do other things. Decided to setup my iMac for a lot of the looping tasks because it makes it easier to keep the repo clones and work product separated.

Ryan Carson

34,837 次观看 • 4 个月前

my new vibe code setup: 1 orchestrator agent which controls 85 sub-agents working in parallel each sub-agent spawns from my stream of consciousness and tests from the main orchestrator Here's how it works:

my new vibe code setup: 1 orchestrator agent which controls 85 sub-agents working in parallel each sub-agent spawns from my stream of consciousness and tests from the main orchestrator Here's how it works:

Greg Kamradt

316,172 次观看 • 1 年前

The end result of Project 2025

The end result of Project 2025

The Daily Sneed™

32,802,044 次观看 • 1 年前

We're excited to power LemonSlice's new video agent product. Turn your real-time voice agent into a video agent using a single photo. The underlying model—which autoscales on Modal 😉—streams infinite-length talking video at 20fps using a single GPU.

We're excited to power LemonSlice's new video agent product. Turn your real-time voice agent into a video agent using a single photo. The underlying model—which autoscales on Modal 😉—streams infinite-length talking video at 20fps using a single GPU.

Modal

17,225 次观看 • 6 个月前

It's true, most AI coding tools make you do the grunt testing and debugging work. That's why we invested in researching and building a state-of-the-art app testing tool. Replit Agent 3 will spin up a browser and test your app end-to-end.

It's true, most AI coding tools make you do the grunt testing and debugging work. That's why we invested in researching and building a state-of-the-art app testing tool. Replit Agent 3 will spin up a browser and test your app end-to-end.

Amjad Masad

109,857 次观看 • 6 个月前

A very short look at our original animation project, The End Of The World. We're still working on it, and it's a ton of fun. Hopefully I'll be able to provide a release date soon.

A very short look at our original animation project, The End Of The World. We're still working on it, and it's a ton of fun. Hopefully I'll be able to provide a release date soon.

Kannas Ricebowl

108,216 次观看 • 5 个月前

JISOO reacting to the performance of the group Allday Project at Cartier's year-end party!

JISOO reacting to the performance of the group Allday Project at Cartier's year-end party!

َ

10,680 次观看 • 7 个月前

Sonnet-3.7 is available in Cursor! We've been very impressed by its coding ability, especially on real-world agentic tasks. It appears to be the new state of the art.

Sonnet-3.7 is available in Cursor! We've been very impressed by its coding ability, especially on real-world agentic tasks. It appears to be the new state of the art.

Cursor

1,115,178 次观看 • 1 年前

SOMEONE OPEN SOURCED A SMALL MODEL TRAINED SPECIFICALLY AS A PERSONAL AGENT ROUTER DECIDES WHAT RUNS LOCAL VS CLOUD AUTOMATICALLY ROUTES TASKS TO THE RIGHT MODEL BASED ON COMPLEXITY SMARTER AGENT ORCHESTRATION WITHOUT THE OVERHEAD

SOMEONE OPEN SOURCED A SMALL MODEL TRAINED SPECIFICALLY AS A PERSONAL AGENT ROUTER DECIDES WHAT RUNS LOCAL VS CLOUD AUTOMATICALLY ROUTES TASKS TO THE RIGHT MODEL BASED ON COMPLEXITY SMARTER AGENT ORCHESTRATION WITHOUT THE OVERHEAD

0xMarioNawfal

157,414 次观看 • 3 个月前

As a naive citizen of Pakistan, I may have grievances against you, but as a student of realpolitik and State craftsmanship, I call you a successful Statesman!!! Your State was your ultimate end, and you successfully achieved that end: Politics is normalised; economy, stabilised, defence, reinforced and diplomacy, bolstered. Ladies and Gentlemen! Although clad in uniform, he is the Statesman from within.

As a naive citizen of Pakistan, I may have grievances against you, but as a student of realpolitik and State craftsmanship, I call you a successful Statesman!!! Your State was your ultimate end, and you successfully achieved that end: Politics is normalised; economy, stabilised, defence, reinforced and diplomacy, bolstered. Ladies and Gentlemen! Although clad in uniform, he is the Statesman from within.

Suleman Shah Rashdi

169,381 次观看 • 7 个月前

MAMAMOO Hwasa forgot which finger to bite at the end of a Gogobebe performance. She is so adorable^^

MAMAMOO Hwasa forgot which finger to bite at the end of a Gogobebe performance. She is so adorable^^

DragonKazoo77

17,722 次观看 • 8 个月前

The U.K. is burning. America has a black crime epidemic. The Middle East is about to explode again. The Economy is on the edge. This is the end result of allowing jews to accumulate wealth and power for 80 years. This is the end result of the Liberal world order. No more.

The U.K. is burning. America has a black crime epidemic. The Middle East is about to explode again. The Economy is on the edge. This is the end result of allowing jews to accumulate wealth and power for 80 years. This is the end result of the Liberal world order. No more.

𝖍𝖎𝖙𝖑𝖊𝖗 𝖑𝖔𝖛𝖊𝖉 𝖈𝖍𝖗𝖎𝖘𝖙

42,750 次观看 • 14 天前

2 Cursor Composer agents working together on the same project. one builds the project, the other reviews once the first agent is done and writes a report. cursor rules file is in comment to achieve this. first agent is instructed to build the project second agent is instructed to write a monitoring script to determine when the project is done then to review the project for errors and write a report This prompt can be improved. This is meant as a baseline If you are interested in learning how to use Cursor more in depth, I have a 28 chapter 1000x Cursor Course building full apps from scratch with 19 hours of content. link is in my bio if you are interested

2 Cursor Composer agents working together on the same project. one builds the project, the other reviews once the first agent is done and writes a report. cursor rules file is in comment to achieve this. first agent is instructed to build the project second agent is instructed to write a monitoring script to determine when the project is done then to review the project for errors and write a report This prompt can be improved. This is meant as a baseline If you are interested in learning how to use Cursor more in depth, I have a 28 chapter 1000x Cursor Course building full apps from scratch with 19 hours of content. link is in my bio if you are interested

echo.hive

149,034 次观看 • 1 年前

this might be the end of Sketch & Agent after what just happened 💔

this might be the end of Sketch & Agent after what just happened 💔

yoxic

14,791,461 次观看 • 1 年前

GPU Web Acceleration Test (This is not a benchmark, it reflects real-life web GPU performance)

GPU Web Acceleration Test (This is not a benchmark, it reflects real-life web GPU performance)

Pixel Gamer 4k

66,214 次观看 • 1 年前

when she single-handley brings state-of-the-art hair physics to 2K

when she single-handley brings state-of-the-art hair physics to 2K

latrell

530,562 次观看 • 9 个月前

MILEY CYRUS NEW SINGLE TONIGHT END OF THE WORLD

MILEY CYRUS NEW SINGLE TONIGHT END OF THE WORLD

Miley Topic

21,845 次观看 • 1 年前