Anthropic just announced Computer Use It allows Claude to control your computer screen based on a prompt and take actions on your behalf The use cases in agentic coding with automated debugging, customer support, and education are going to be INSANE

It works by taking static screenshots that are constantly sent back to the API in real-time Then Clade can move your cursor, click, and type text Claude 3.5 Sonnet is now the first ever frontier model to offer a feature like 'Computer Use' to the public

It's worth noting a couple of limitations: 1. You can't train computer use on internal data (yet) 2. It can take action for ~15 minutes but is limited with context windows, etc. When I chatted with @AnthropicAI, they told me that they believe this is the GPT-3 days for computer use-type systems. We're still very early.

Here's the full announcement from Anthropic They also announced an upgraded Claude 3.5 Sonnet that crushes in benchmarks, and Claude 3.5 Haiku (new small model)

Can't think of a single security risk.

The line between human intent and machine execution just got thinner. Delegating tasks is one thing—ceding control is another. When AI can take action on your behalf, we move beyond tools into territory where humans no longer need to do anything at all. Convenient? Absolutely. But at what point do we stop calling it human productivity? Nice share @rowancheung

It’s like pair programming on steroids Cool idea

Automated debugging while you're out on a lunch break :)

Very cool. looks like my kids won’t be coding themselves then! 😂

I think not just coding -- but also writing, optimizing design, learning new skills, etc. Future gens will all have agents teaching them, improving everything they do, and just generally morphing what we do for 'work' Amazing times ahead

woah this killed @autotabai just one day after launch