Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Today we're excited to launch Action: a Claude computer use launcher for macOS. Action is a macOS launcher that can take actions (click, type, and more) on your Mac using Claude’s computer use API. The interface is a floating window triggered by a keyboard shortcut, similar to Spotlight. This... show more

Lawrence Chen

1,258 subscribers

21,622 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von Lawrence Chen

Lawrence Chenvor 1 Jahr

Since Action operates at the operating system level, it has access to your authenticated apps. This means that you don't have to pass your credentials through a third party in order to automate apps like Doordash, Amazon, and Uber. Watch Action order Doordash using my account:

Profilbild von Lawrence Chen

Lawrence Chenvor 1 Jahr

Here, Action installs the SwiftAnthropic package by interacting with Xcode:

Profilbild von Lawrence Chen

Lawrence Chenvor 1 Jahr

You can also use Action as an LLM that has immediate context of your screen. Unlike the ChatGPT Mac app, taking a screenshot with Action doesn't require extra steps -- it's automatically included when you send a message. Here, I use it to extract a table from a PDF in Preview:

Profilbild von Lawrence Chen

Lawrence Chenvor 1 Jahr

Action is written purely in Swift, and the DMG size is quite small at roughly 3 MB. You can download the macOS client today at (requires Anthropic API key).

Profilbild von Page to Pixel Publishing

Page to Pixel Publishingvor 2 Jahren

The Art of Flight is a homage to 80s/90s arcade action shmups with a fresh twist on the genre. Pilot multiple ships at the same time to take on oncoming waves of enemies in this fast paced space shooter. Wishlist on Steam today!

Profilbild von Austin Wang

Austin Wangvor 1 Jahr

This is a super cool use of computer agents - really drives momentum of AI agents to the new year!

Profilbild von Wesley Tjangnaka

Wesley Tjangnakavor 1 Jahr

Amazing stuff!

Profilbild von Subhash Ramesh

Subhash Rameshvor 1 Jahr

🔥🔥

Profilbild von Adi

Adivor 1 Jahr

@ycombinator This simply doesn’t work. Been trying for the past hour. Asked it to order my usual from doordash and this is where I am

Profilbild von Lawrence Chen

Lawrence Chenvor 1 Jahr

@ycombinator Hm, do you think it's a fundamental issue with claude computer use or was it an implementation error? Did it try to open chrome first and forget to press 'return'?

Profilbild von Alex Xi

Alex Xivor 1 Jahr

Cool stuff!

Ähnliche Videos

Autopilot for Mac powered by LLM You are witnessing an AI (Claude 3.7) auto-piloting my mac. Impressed with Claude 3.7 as a coder? Wait till you see a "user" Claude. macOS-use is a gradio app from Ofir Ozeri for making AI "use" your mac to do stuff. 1-click install is here.

Autopilot for Mac powered by LLM You are witnessing an AI (Claude 3.7) auto-piloting my mac. Impressed with Claude 3.7 as a coder? Wait till you see a "user" Claude. macOS-use is a gradio app from Ofir Ozeri for making AI "use" your mac to do stuff. 1-click install is here.

cocktail peanut

27,757 Aufrufe • vor 1 Jahr

Excited to launch my first macOS app! DopeDrop is a tiny, native macOS app that adds delightful effects to the files you AirDrop to your Mac.

Excited to launch my first macOS app! DopeDrop is a tiny, native macOS app that adds delightful effects to the files you AirDrop to your Mac.

Noman

26,637 Aufrufe • vor 1 Monat

Anthropic just announced Computer Use It allows Claude to control your computer screen based on a prompt and take actions on your behalf The use cases in agentic coding with automated debugging, customer support, and education are going to be INSANE

Anthropic just announced Computer Use It allows Claude to control your computer screen based on a prompt and take actions on your behalf The use cases in agentic coding with automated debugging, customer support, and education are going to be INSANE

Rowan Cheung

1,024,432 Aufrufe • vor 1 Jahr

Today, we’re releasing a feature that allows Claude to control your computer: Mouse, keyboard, and screen, giving it the ability to use any app. I believe this is especially useful if used with Dispatch, which allows you to remotely control Claude on your computer while you’re away.

Today, we’re releasing a feature that allows Claude to control your computer: Mouse, keyboard, and screen, giving it the ability to use any app. I believe this is especially useful if used with Dispatch, which allows you to remotely control Claude on your computer while you’re away.

Felix Rieseberg

4,787,912 Aufrufe • vor 3 Monaten

Imagine an AI application that can type anywhere you can and use the full context of what's on your screen. This is the application we all deserve (at least if you have macOS.) Check out Omnipilot. It's an app that works with every other macOS application and uses Claude Sonet 3.5 in the background—it also supports Gemini and GPT-4o. Here is the idea: You can use the tool to ask questions about anything on your screen. Or you can use it to autocomplete the text you are typing. You don't need to copy and paste anymore or waste your time providing context to a model. It sees what you see. It works right where you are. That's pretty cool. Here are a couple of cool examples: • Use it to reply to an email • Use it in the terminal to autocomplete a command • Use it to finish a document • Use to send a message on Slack AI at the system level is bonkers. You can read a ton more on their Product Hunt launch page: Thanks to the Omnipilot team for collaborating with me on this post!

Imagine an AI application that can type anywhere you can and use the full context of what's on your screen. This is the application we all deserve (at least if you have macOS.) Check out Omnipilot. It's an app that works with every other macOS application and uses Claude Sonet 3.5 in the background—it also supports Gemini and GPT-4o. Here is the idea: You can use the tool to ask questions about anything on your screen. Or you can use it to autocomplete the text you are typing. You don't need to copy and paste anymore or waste your time providing context to a model. It sees what you see. It works right where you are. That's pretty cool. Here are a couple of cool examples: • Use it to reply to an email • Use it in the terminal to autocomplete a command • Use it to finish a document • Use to send a message on Slack AI at the system level is bonkers. You can read a ton more on their Product Hunt launch page: Thanks to the Omnipilot team for collaborating with me on this post!

Santiago

72,306 Aufrufe • vor 2 Jahren

Today, we’re excited to announce AI Actions! AI Actions turn your chatbot into an AI Agent that can take actions by calling any API. This lets the agent do things like upgrade_subscription, track_shipment, book_appointment, and more. Here’s how it works: 1. Set up your API in Chatbase or select from prebuilt actions like Cal.com get_available_slots and Matt Slack send_message. 2. Provide a prompt to guide the Agent on when and how to use the action. 3. Your AI Agent takes care of the rest, executing actions for your customers, giving them a mind-blowing experience. These actions can be internal APIs for app-specific tasks or it can be APIs for external tools your business uses like a ticketing system, a CRM, e-commerce platform, or really anything else. We're raising the bar for customer experience.

Today, we’re excited to announce AI Actions! AI Actions turn your chatbot into an AI Agent that can take actions by calling any API. This lets the agent do things like upgrade_subscription, track_shipment, book_appointment, and more. Here’s how it works: 1. Set up your API in Chatbase or select from prebuilt actions like Cal.com get_available_slots and Matt Slack send_message. 2. Provide a prompt to guide the Agent on when and how to use the action. 3. Your AI Agent takes care of the rest, executing actions for your customers, giving them a mind-blowing experience. These actions can be internal APIs for app-specific tasks or it can be APIs for external tools your business uses like a ticketing system, a CRM, e-commerce platform, or really anything else. We're raising the bar for customer experience.

Chatbase

410,583 Aufrufe • vor 1 Jahr

Our first short course with Anthropic! Building Towards Computer Use with Anthropic. This teaches you to build an LLM-based agent that uses a computer interface by generating mouse clicks and keystrokes. Computer Use is an important, emerging capability for LLMs that will let AI agents do many more tasks than were possible before, since it lets them interact with interfaces designed for humans to use, rather than only tools that provide explicit API access. I hope you will enjoy learning about it! This course is taught by Anthropic's Head of Curriculum, Colt_Steele. You'll learn to apply image reasoning and tool use to "use" a computer as follows: a model processes an image of the screen, analyzes it to understand what's going on, and navigates the computer via mouse clicks and keystrokes. This course goes through the key building blocks, and culminates in a demo of an AI assistant that uses a web browser to search for a research paper, downloads the PDF, and finally summarizes the paper for you. In detail, you’ll: - Learn about Anthropic's family of models, when to use which one, and make API requests to Claude - Use multi-modal prompts that combine text and image content blocks, and also work with streaming responses - Improve your prompting by using prompt templates, using XML to structure prompts, and providing examples - Implement prompt caching to reduce cost and latency - Apply tool-use to build a chatbot that can call different tools to respond to queries - See all these building blocks come together in Computer Use demo Please sign up here:

Our first short course with Anthropic! Building Towards Computer Use with Anthropic. This teaches you to build an LLM-based agent that uses a computer interface by generating mouse clicks and keystrokes. Computer Use is an important, emerging capability for LLMs that will let AI agents do many more tasks than were possible before, since it lets them interact with interfaces designed for humans to use, rather than only tools that provide explicit API access. I hope you will enjoy learning about it! This course is taught by Anthropic's Head of Curriculum, Colt_Steele. You'll learn to apply image reasoning and tool use to "use" a computer as follows: a model processes an image of the screen, analyzes it to understand what's going on, and navigates the computer via mouse clicks and keystrokes. This course goes through the key building blocks, and culminates in a demo of an AI assistant that uses a web browser to search for a research paper, downloads the PDF, and finally summarizes the paper for you. In detail, you’ll: - Learn about Anthropic's family of models, when to use which one, and make API requests to Claude - Use multi-modal prompts that combine text and image content blocks, and also work with streaming responses - Improve your prompting by using prompt templates, using XML to structure prompts, and providing examples - Implement prompt caching to reduce cost and latency - Apply tool-use to build a chatbot that can call different tools to respond to queries - See all these building blocks come together in Computer Use demo Please sign up here:

Andrew Ng

170,399 Aufrufe • vor 1 Jahr

We’ve built a macOS app for all the power users out there. It launches on Nov 3rd. Pre-order on your Mac computer at the link below to have it installed on launch day.

We’ve built a macOS app for all the power users out there. It launches on Nov 3rd. Pre-order on your Mac computer at the link below to have it installed on launch day.

untitledinbrackets

38,211 Aufrufe • vor 8 Monaten

Run Artist AI on Your Local Computer. Just published a 1 click local launcher for the Artist Gradio app on Huggingface by Sylvain Filoni -- lets you take any existing image and stylize it with high quality results. Run locally on all platforms with one click: Mac, Linux, Windows.

Run Artist AI on Your Local Computer. Just published a 1 click local launcher for the Artist Gradio app on Huggingface by Sylvain Filoni -- lets you take any existing image and stylize it with high quality results. Run locally on all platforms with one click: Mac, Linux, Windows.

cocktail peanut

14,482 Aufrufe • vor 2 Jahren

Here is Codex building TetrisBar, a Tetris for macOS that you can play anytime, right in your menu bar! It's using our new Build macOS Apps plugin! You can learn how to use it here:

Here is Codex building TetrisBar, a Tetris for macOS that you can play anytime, right in your menu bar! It's using our new Build macOS Apps plugin! You can learn how to use it here:

Thomas Ricouard

21,369 Aufrufe • vor 3 Monaten

Today, we're launching computer use on the cloud, operating a remote Apple macbook with Vy, starting with a rollout on Discord! 🥳 It is completely free to try! Just use any of the Ask Vy Channels on Discord. Then, start your prompt with /vy, such as with: /vy this is my task! We're really excited to see what you all do!! Details 👇:

Today, we're launching computer use on the cloud, operating a remote Apple macbook with Vy, starting with a rollout on Discord! 🥳 It is completely free to try! Just use any of the Ask Vy Channels on Discord. Then, start your prompt with /vy, such as with: /vy this is my task! We're really excited to see what you all do!! Details 👇:

Vercept

41,492 Aufrufe • vor 1 Jahr

Anthropic just launched Computer-Use-Agents on Claude Desktop, meaning it can now take control of your entire computer environment. This is huge for businesses and enterprise, except it's limited by the fact that it requires a personal computer. If you deploy it into a virtual machine on Orgo, you can now manage hundreds or thousands of your clients' Claude accounts, setting up their skills, capabilities, connectors, etc, all on their own machines in the cloud on Orgo.

Anthropic just launched Computer-Use-Agents on Claude Desktop, meaning it can now take control of your entire computer environment. This is huge for businesses and enterprise, except it's limited by the fact that it requires a personal computer. If you deploy it into a virtual machine on Orgo, you can now manage hundreds or thousands of your clients' Claude accounts, setting up their skills, capabilities, connectors, etc, all on their own machines in the cloud on Orgo.

nick vasilescu

38,157 Aufrufe • vor 3 Monaten

I’ve created a way to use your iPhone to travel through time and apps on your Mac. It features a liquid glass timeline that lets you jump between recently opened apps on your Mac and updates in real-time. You can even pin apps to the front of the timeline to create the perfect app launcher combination for your current workflow.

I’ve created a way to use your iPhone to travel through time and apps on your Mac. It features a liquid glass timeline that lets you jump between recently opened apps on your Mac and updates in real-time. You can even pin apps to the front of the timeline to create the perfect app launcher combination for your current workflow.

Phil Traut ᯅ

129,502 Aufrufe • vor 2 Monaten

WOW! I'm so excited about this. OpenAI Developers said Codex was good at Computer Use, but I wasn't prepared for this. For the last two weeks I've been working on a Computer Use skill to work with Linux. And while I had some success, it was a pretty frustrating experience. That is...until the breakthrough. Using accessibility tools, Codex can now control my entire computer, not just the browser. There are limits to this, of course, but what a time to be alive. This Computer Use skill will unlock and entirely new set of automations, all powered by Codex. Demonstration below 👇

WOW! I'm so excited about this. OpenAI Developers said Codex was good at Computer Use, but I wasn't prepared for this. For the last two weeks I've been working on a Computer Use skill to work with Linux. And while I had some success, it was a pretty frustrating experience. That is...until the breakthrough. Using accessibility tools, Codex can now control my entire computer, not just the browser. There are limits to this, of course, but what a time to be alive. This Computer Use skill will unlock and entirely new set of automations, all powered by Codex. Demonstration below 👇

am.will

79,647 Aufrufe • vor 3 Monaten

Today we're excited to introduce Vy, our AI that sees and acts on your computer. At Vercept, our mission is to reinvent how humans use computers–enabling you to accomplish orders of magnitude more than what you can do today. Vy is a first glimpse at AI that sees and uses your computer just like you do. It runs natively on your computer, with access to all your applications and accounts that you’re already signed into. It is available for download starting today! As part of the release, we’ve built VyUI, an AI model bridging the gap between language and your screen. It sets a new state-of-the-art on UI grounding benchmarks like ScreenSpot v1, ScreenSpot v2, and GroundUI Web–beating leading models from OpenAI, Google, and Anthropic. Check out what Vy can do in the thread below: 🧵👇

Today we're excited to introduce Vy, our AI that sees and acts on your computer. At Vercept, our mission is to reinvent how humans use computers–enabling you to accomplish orders of magnitude more than what you can do today. Vy is a first glimpse at AI that sees and uses your computer just like you do. It runs natively on your computer, with access to all your applications and accounts that you’re already signed into. It is available for download starting today! As part of the release, we’ve built VyUI, an AI model bridging the gap between language and your screen. It sets a new state-of-the-art on UI grounding benchmarks like ScreenSpot v1, ScreenSpot v2, and GroundUI Web–beating leading models from OpenAI, Google, and Anthropic. Check out what Vy can do in the thread below: 🧵👇

Vercept

123,748 Aufrufe • vor 1 Jahr

✉️@FridayMail (YC F24) is building an email assistant that understands your emails, learns to predict the action you would take, and performs those actions for you so you can focus on the emails that matter. Congrats on the launch, @allennaliath and Advait Paliwal!

✉️@FridayMail (YC F24) is building an email assistant that understands your emails, learns to predict the action you would take, and performs those actions for you so you can focus on the emails that matter. Congrats on the launch, @allennaliath and Advait Paliwal!

Y Combinator

35,935 Aufrufe • vor 1 Jahr

I can't tell you the last time I was so excited to see a new AI capability in action. We plugged in Claude computer use in Replit ⠕ Agent as a human feedback replacement. And… it just works! I feel it won't take long until our agent will become fully autonomous.

I can't tell you the last time I was so excited to see a new AI capability in action. We plugged in Claude computer use in Replit ⠕ Agent as a human feedback replacement. And… it just works! I feel it won't take long until our agent will become fully autonomous.

Michele Catasta

185,920 Aufrufe • vor 1 Jahr

Computer Use is now a built-in tool supported in Gemini 3.5 Flash. 💻 Developers can now use 3.5 Flash to build custom agents that see and take action across browser, mobile and desktop environments.

Computer Use is now a built-in tool supported in Gemini 3.5 Flash. 💻 Developers can now use 3.5 Flash to build custom agents that see and take action across browser, mobile and desktop environments.

Google

241,335 Aufrufe • vor 25 Tagen

People are overlooking Google Gemini Realtime models for computer use It gave me sub 100ms latency with computer use. It has a much larger context window and is much cheaper as well Combine that with local OCR and local screen detection model based on Omniparser by Microsoft it works under 100ms action taking when combined with Cua I also put in a harness for Nous Research Hermes with it. You can access it all at your tip of your cursor. You can draw on your screen to give your agents a context And I am making it Open Source! Link in the Comments Sundar Pichai Min-Liang Tan Mojtaba Seyedhosseini

People are overlooking Google Gemini Realtime models for computer use It gave me sub 100ms latency with computer use. It has a much larger context window and is much cheaper as well Combine that with local OCR and local screen detection model based on Omniparser by Microsoft it works under 100ms action taking when combined with Cua I also put in a harness for Nous Research Hermes with it. You can access it all at your tip of your cursor. You can draw on your screen to give your agents a context And I am making it Open Source! Link in the Comments Sundar Pichai Min-Liang Tan Mojtaba Seyedhosseini

Milind S

38,054 Aufrufe • vor 1 Monat