8/ Computer Use Are A Sadness - 🎖️Engagement Award Exploiting computer use agents using LLM security vulnerabilities for fun and profit Hijack agents that try to access your computer

OpenAI’s biggest competitor just gave AI the ability to control computers We gave 250+ hackers 24 hours to see what it’s capable of Here’s what we saw at the Nexgen Computer Use Agents Hackathon w/ @AnthropicAI + @AgentOpsAI @notablecap at @AGIHouseSF (🧵):

1/ LLM (Love Language Model) Agent that automatically finds matches on Hinge with an iPhone simulator controller Powered by @screen_pipe

2/ Task Bridge Computer use agent that reads through your slack messages, identifies tasks, and submits them to Asana Zero effort task management with computer use

3/ Tool use team Multi-agent “bureaucracy” for instructing computer use agents to perform tasks better than a single standalone agent Multi-agent computer use @_wenxichen

4/ Open Source Computer Use 🥉3rd Place Computer use system built from the ground up with Open Source and tools. Convert screenshots to grids and use pyautogui, element localization, image -> text and an agent loop to perform tasks @JvNixon @SimeonRadev @ChrisIturri

5/ Phaedrus - 🥈2nd Place Quality assurance agent that goes through UX flow task lists, attempts to test they work, and automatically creates Jira tickets for broke features @bayka @yersultansapar

6/ Tests AI - 🥇1st place UI tests written in natural language that execute computer use agents to test frontends @Andy_AJT @benswerd @theshortjacob @anchnk

7/ AutoMac Mac desktop app that uses Claude to automatically control actions on your computer with natural language @trendt

9/ Sharestack 🎖️AgentStack award Computer use agent built with AgentStack, @AgentOpsAI, and @crewAIInc to turn browser tabs and desktop documents into interactive knowledge graphs

10/ Kura Browser automation agent that automatically creates salesforce leads

Thanks again to @lauraham150 @AGIHouseSF @notablecap @AnthropicAI @AWSstartups @neondatabase @cramforce @MultiOn_AI @DivGarg_ @mdshelton_ @JvNixon for judging + sponsoring! Want to try out computer use agents? Give @AgentOpsAI a try. And for more hacks follow @AlexReibman