Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

How capable are web agents at solving knowledge work tasks? 🤔 Are LLMs up to the challenge? 🤖 Introducing WorkArena: a benchmark where agents meet the world 𝘸𝘪𝘭𝘥 web of enterprise software 🌐🖥️ Paper: Website: 🧵 1/7

Alexandre Lacoste

1,139 subscribers

24,530 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Meet Embodied Web Agents that bridge physical-digital realms. Imagine embodied agents that can search for online recipes, shop for ingredients and cook for you. Embodied web agents search internet information for implementing real-world embodied tasks. All data, codes and web environments are available at Paper link:

Meet Embodied Web Agents that bridge physical-digital realms. Imagine embodied agents that can search for online recipes, shop for ingredients and cook for you. Embodied web agents search internet information for implementing real-world embodied tasks. All data, codes and web environments are available at Paper link:

Yining Hong

54,264 views • 1 year ago

Meet Cabinet: Paper Clip + KB. for quite some time I've been thinking how LLMs are missing the knowledge base - where I can dump CSVs, PDFs, and most important - inline web app. running on Claude Code with agents with heartbeats and jobs

Meet Cabinet: Paper Clip + KB. for quite some time I've been thinking how LLMs are missing the knowledge base - where I can dump CSVs, PDFs, and most important - inline web app. running on Claude Code with agents with heartbeats and jobs

Hila Shmuel

495,920 views • 3 months ago

AI web agents like Operator and Anthropic’s Computer Use can operate a browser, but the LLMs inside are brittle, and you can’t trust what’s on the web. In this 🧵, I’ll show how adversaries can fool Anthropic’s web agent into sending phishing emails or revealing credit card info.

AI web agents like Operator and Anthropic’s Computer Use can operate a browser, but the LLMs inside are brittle, and you can’t trust what’s on the web. In this 🧵, I’ll show how adversaries can fool Anthropic’s web agent into sending phishing emails or revealing credit card info.

Micah Goldblum

42,969 views • 1 year ago

AI agents are getting real jobs now?! Introducing a managed marketplace where companies post roles, and agents get to work. Powered by in collaboration with Amazon Web Services, Anthropic, Wix, and Mesh. There's an agent out there who is the perfect fit for your team! Come and find it 🤖

AI agents are getting real jobs now?! Introducing a managed marketplace where companies post roles, and agents get to work. Powered by in collaboration with Amazon Web Services, Anthropic, Wix, and Mesh. There's an agent out there who is the perfect fit for your team! Come and find it 🤖

monday.com

35,696 views • 3 months ago

we are building at the Y Combinator web agents hackathon this weekend. introducing clawtrace: on-call security for openclaw web agents. openclaw is powerful. but it’s hard to fully trust what your agents are actually doing. that’s why we built clawtrace. clawtrace lets you: - know when your agents go rogue - detect anomalies in behavior - track sessions, tool calls, thinking states here’s how it works:

we are building at the Y Combinator web agents hackathon this weekend. introducing clawtrace: on-call security for openclaw web agents. openclaw is powerful. but it’s hard to fully trust what your agents are actually doing. that’s why we built clawtrace. clawtrace lets you: - know when your agents go rogue - detect anomalies in behavior - track sessions, tool calls, thinking states here’s how it works:

Sri Laasya Nutheti

18,524 views • 4 months ago

we are building at the Y Combinator web agents hackathon this weekend. introducing clawtrace: on-call security for openclaw web agents. openclaw is powerful. but it’s hard to fully trust what your agents are actually doing. that’s why we built clawtrace. clawtrace lets you: - know when your agents go rogue - detect anomalies in behavior - track sessions, tool calls, thinking states here’s how it works:

we are building at the Y Combinator web agents hackathon this weekend. introducing clawtrace: on-call security for openclaw web agents. openclaw is powerful. but it’s hard to fully trust what your agents are actually doing. that’s why we built clawtrace. clawtrace lets you: - know when your agents go rogue - detect anomalies in behavior - track sessions, tool calls, thinking states here’s how it works:

Divya Saini

22,323 views • 4 months ago

NEW: DevRev raises a $100M Series A to build enterprise AI apps & agents - Valued at $1.1B 🦄 The core of their platform is an enterprise knowledge graph that powers AI apps & agents for customer service, product, software engineering, etc. Here's how DevRev works:

NEW: DevRev raises a $100M Series A to build enterprise AI apps & agents - Valued at $1.1B 🦄 The core of their platform is an enterprise knowledge graph that powers AI apps & agents for customer service, product, software engineering, etc. Here's how DevRev works:

Chief AI Officer

11,222 views • 1 year ago

Introducing 🦀 CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents 🦀 CRAB provides an end-to-end and easy-to-use framework to build multimodal agents, operate environments, and create benchmarks to evaluate them, featuring three key components: - 🔀 Cross-environment support - agents can operate tasks in 📱 Android and 💻 Ubuntu. - 🕸️ Graph evaluator - provides a fine-grain evaluation metric for agents. - 🤖 Task generation - composes subtasks to automatically generate tasks. By connecting all devices to agents, 🦀CRAB unlocks greater capabilities for human-like tasks than ever before. Use 🦀 CRAB to benchmark your multimodal agents! - 👨‍💻 Check out the repository: - 📝 Read the paper: - 🌐 Find out more via the project page: - 🐫 Join our community:

Introducing 🦀 CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents 🦀 CRAB provides an end-to-end and easy-to-use framework to build multimodal agents, operate environments, and create benchmarks to evaluate them, featuring three key components: - 🔀 Cross-environment support - agents can operate tasks in 📱 Android and 💻 Ubuntu. - 🕸️ Graph evaluator - provides a fine-grain evaluation metric for agents. - 🤖 Task generation - composes subtasks to automatically generate tasks. By connecting all devices to agents, 🦀CRAB unlocks greater capabilities for human-like tasks than ever before. Use 🦀 CRAB to benchmark your multimodal agents! - 👨‍💻 Check out the repository: - 📝 Read the paper: - 🌐 Find out more via the project page: - 🐫 Join our community:

CAMEL-AI.org

72,661 views • 1 year ago

AI agents are creating a new enterprise challenge: economics. At Dell Technologies Technologies World 2026, Ihab Tarazi of Dell and NVIDIA's Jason Hardy explained why the next phase of enterprise AI isn't about access to models, it's about controlling the cost of compute, storage, and the explosion of data agents create. The next wave of AI success will be defined by how organizations manage the economics behind agents at scale. $NVDA $DELL

AI agents are creating a new enterprise challenge: economics. At Dell Technologies Technologies World 2026, Ihab Tarazi of Dell and NVIDIA's Jason Hardy explained why the next phase of enterprise AI isn't about access to models, it's about controlling the cost of compute, storage, and the explosion of data agents create. The next wave of AI success will be defined by how organizations manage the economics behind agents at scale. $NVDA $DELL

Six Five Media

26,975 views • 1 month ago

🚀 Introducing Boson V3 - Decentralized AI Commerce Post Web commerce for humans and AI TL;DR Crypto x AI Agents -> Parabolic adoption = The end of the Web and Platforms Crypto needs AI to bridge complex UX AI agents need Crypto for trusted execution of transactions Discover how Boson V3 is the base layer for the Trillion-Dollar Post Web Commerce Era! 👇🧵

🚀 Introducing Boson V3 - Decentralized AI Commerce Post Web commerce for humans and AI TL;DR Crypto x AI Agents -> Parabolic adoption = The end of the Web and Platforms Crypto needs AI to bridge complex UX AI agents need Crypto for trusted execution of transactions Discover how Boson V3 is the base layer for the Trillion-Dollar Post Web Commerce Era! 👇🧵

Boson

2,294,110 views • 1 year ago

The real challenge isn’t building agents. It’s discovering them. Trusting them. Coordinating them. Just like the early web had millions of websites, agentic AI will have millions of autonomous agents. Now, agents need infrastructure to find, verify, and work with other agents. Listen to Humayun discuss how to make the process secure, autonomous and scalable.

The real challenge isn’t building agents. It’s discovering them. Trusting them. Coordinating them. Just like the early web had millions of websites, agentic AI will have millions of autonomous agents. Now, agents need infrastructure to find, verify, and work with other agents. Listen to Humayun discuss how to make the process secure, autonomous and scalable.

Fetch.ai

12,035 views • 5 months ago

🚀We are excited to introduce the Tool Decathlon (Toolathlon), a benchmark for language agents on diverse, complex, and realistic tool use. ⭐️32 applications and 600+ tools based on real-world software environments ⭐️Execution-based, reliable evaluation ⭐️Realistic, covering daily and professional scenarios Toolathlon reveals significant shortcomings of SOTA LLMs in realistic tool-use tasks, where Claude Sonnet 4.5 achieves 38.6% success rate. It also indicates a clear gap between open-source and leading proprietary models. Check our blog: Github: Paper: 🧵⬇️

🚀We are excited to introduce the Tool Decathlon (Toolathlon), a benchmark for language agents on diverse, complex, and realistic tool use. ⭐️32 applications and 600+ tools based on real-world software environments ⭐️Execution-based, reliable evaluation ⭐️Realistic, covering daily and professional scenarios Toolathlon reveals significant shortcomings of SOTA LLMs in realistic tool-use tasks, where Claude Sonnet 4.5 achieves 38.6% success rate. It also indicates a clear gap between open-source and leading proprietary models. Check our blog: Github: Paper: 🧵⬇️

Junxian He

43,743 views • 8 months ago

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

Deyao Zhu

351,506 views • 17 days ago

How well do today’s frontier models handle long-horizon, multi-step web agent tasks, such as identifying the top 25 U.S. CS PhD programs with ML/AI faculty likely accepting students and compiling the results into a structured sheet? Check out our new work on Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks Paper: Leaderboard: We introduce Odysseys, a benchmark of 200 long-horizon tasks derived from real browsing sessions and evaluated on the live Internet. We show that binary pass/fail is inadequate in this setting and propose rubric-based evaluation, which better aligns with human judgment and provides more informative signals. Across leading models, the best achieves only 44.5% success, leaving substantial headroom. We further introduce a Trajectory Efficiency metric (rubric score per step) and find efficiency remains extremely low (1.15%), highlighting a key bottleneck. Odysseys provides a realistic benchmark for measuring progress toward web agents capable of sustained, efficient, real-world operation. See a more detailed thread by Jing Yu Koh.

How well do today’s frontier models handle long-horizon, multi-step web agent tasks, such as identifying the top 25 U.S. CS PhD programs with ML/AI faculty likely accepting students and compiling the results into a structured sheet? Check out our new work on Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks Paper: Leaderboard: We introduce Odysseys, a benchmark of 200 long-horizon tasks derived from real browsing sessions and evaluated on the live Internet. We show that binary pass/fail is inadequate in this setting and propose rubric-based evaluation, which better aligns with human judgment and provides more informative signals. Across leading models, the best achieves only 44.5% success, leaving substantial headroom. We further introduce a Trajectory Efficiency metric (rubric score per step) and find efficiency remains extremely low (1.15%), highlighting a key bottleneck. Odysseys provides a realistic benchmark for measuring progress toward web agents capable of sustained, efficient, real-world operation. See a more detailed thread by Jing Yu Koh.

Russ Salakhutdinov

22,518 views • 2 months ago

.Aaron Levie says AI agents may take years to diffuse from engineering into the rest of knowledge work. "The models are really good at code, and the work is verifiable." "You have 5 or 10 things that make agents work in an enterprise context for engineering." "The users are less technical, the data is much more fragmented, the systems are much more legacy." "That's why it's going to be a number of years for this diffusion to roll from what we're seeing in Silicon Valley, into the rest of knowledge work."

.Aaron Levie says AI agents may take years to diffuse from engineering into the rest of knowledge work. "The models are really good at code, and the work is verifiable." "You have 5 or 10 things that make agents work in an enterprise context for engineering." "The users are less technical, the data is much more fragmented, the systems are much more legacy." "That's why it's going to be a number of years for this diffusion to roll from what we're seeing in Silicon Valley, into the rest of knowledge work."

MTS

13,046 views • 2 months ago

Sat down with Aaron Levie, CEO of Box, to discuss where AI agents are going, how agentic workflows are transforming software, why agents will require much more memory, and how token budgets for enterprises are evolving. Thank you Aaron Levie for taking the time! 00:00 - Intro 01:30 - Where are we in AI 06:25 - The Role of Agents in Software 11:40 - The Role of Memory for Agents 14:45 - TokenMaxxing 24:54 - Why Are There AI Doomers? 31:50 - Aaron's Take on Nvidia

Sat down with Aaron Levie, CEO of Box, to discuss where AI agents are going, how agentic workflows are transforming software, why agents will require much more memory, and how token budgets for enterprises are evolving. Thank you Aaron Levie for taking the time! 00:00 - Intro 01:30 - Where are we in AI 06:25 - The Role of Agents in Software 11:40 - The Role of Memory for Agents 14:45 - TokenMaxxing 24:54 - Why Are There AI Doomers? 31:50 - Aaron's Take on Nvidia

amit

95,235 views • 19 days ago

“AI agents will outperform humans at almost all jobs by 2026–2027.” - The forecast is everywhere. So we built the exam to test that claim, on real labor-market aligned work. On the hardest tier, top agents pass 2.6%. Meet Agents' Last Exam (ALE), a rolling benchmark measuring whether agents can actually do real jobs. 🧵👇

“AI agents will outperform humans at almost all jobs by 2026–2027.” - The forecast is everywhere. So we built the exam to test that claim, on real labor-market aligned work. On the hardest tier, top agents pass 2.6%. Meet Agents' Last Exam (ALE), a rolling benchmark measuring whether agents can actually do real jobs. 🧵👇

Yiyou Sun

94,943 views • 1 month ago

INTELLIGENT TASKS ARE A STEPPING STONE TO AGI Today, we are launching ChatLLM Tasks. We have hooked up tools like web search, email, and web scrappers to mini-agents that can be triggered on a schedule. These tasks combine intelligent tool use with crons! We think this is the first step towards AGI. The next step is to connect our AI engineer to create more complex tasks. AGI STEP ONE - DONE!

INTELLIGENT TASKS ARE A STEPPING STONE TO AGI Today, we are launching ChatLLM Tasks. We have hooked up tools like web search, email, and web scrappers to mini-agents that can be triggered on a schedule. These tasks combine intelligent tool use with crons! We think this is the first step towards AGI. The next step is to connect our AI engineer to create more complex tasks. AGI STEP ONE - DONE!

Bindu Reddy

18,486 views • 1 year ago

Meet Database Agents. A team of li’l librarians in your database — keeping it up to date automatically using context from the page, your workspace, and the web. No more “we’ll update the database later.” Starting to roll out on Biz + Enterprise.

Meet Database Agents. A team of li’l librarians in your database — keeping it up to date automatically using context from the page, your workspace, and the web. No more “we’ll update the database later.” Starting to roll out on Biz + Enterprise.

Notion

299,269 views • 3 months ago

Bloomberg's Mark Gurman says iOS and the App Store are already legacy features, and that AI agents are the future of Apple's software: "The AI agent-ification of iOS is happening." "Apps are the past. AI agents are already here, and that's the move forward." "AI agents are where the world is going, and I totally expect Apple to move in this direction."

Bloomberg's Mark Gurman says iOS and the App Store are already legacy features, and that AI agents are the future of Apple's software: "The AI agent-ification of iOS is happening." "Apps are the past. AI agents are already here, and that's the move forward." "AI agents are where the world is going, and I totally expect Apple to move in this direction."

TBPN

30,699 views • 5 months ago