6/ Tests AI - 🥇1st place UI tests written in natural language that execute computer use agents to test frontends Andy T Benۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗۗ☁️ Jacob Zwang anchnk

OpenAI’s biggest competitor just gave AI the ability to control computers We gave 250+ hackers 24 hours to see what it’s capable of Here’s what we saw at the Nexgen Computer Use Agents Hackathon w/ @AnthropicAI + @AgentOpsAI @notablecap at @AGIHouseSF (🧵):

1/ LLM (Love Language Model) Agent that automatically finds matches on Hinge with an iPhone simulator controller Powered by @screen_pipe

2/ Task Bridge Computer use agent that reads through your slack messages, identifies tasks, and submits them to Asana Zero effort task management with computer use

3/ Tool use team Multi-agent “bureaucracy” for instructing computer use agents to perform tasks better than a single standalone agent Multi-agent computer use @_wenxichen

4/ Open Source Computer Use 🥉3rd Place Computer use system built from the ground up with Open Source and tools. Convert screenshots to grids and use pyautogui, element localization, image -> text and an agent loop to perform tasks @JvNixon @SimeonRadev @ChrisIturri

5/ Phaedrus - 🥈2nd Place Quality assurance agent that goes through UX flow task lists, attempts to test they work, and automatically creates Jira tickets for broke features @bayka @yersultansapar

7/ AutoMac Mac desktop app that uses Claude to automatically control actions on your computer with natural language @trendt

8/ Computer Use Are A Sadness - 🎖️Engagement Award Exploiting computer use agents using LLM security vulnerabilities for fun and profit Hijack agents that try to access your computer

9/ Sharestack 🎖️AgentStack award Computer use agent built with AgentStack, @AgentOpsAI, and @crewAIInc to turn browser tabs and desktop documents into interactive knowledge graphs

10/ Kura Browser automation agent that automatically creates salesforce leads

Thanks again to @lauraham150 @AGIHouseSF @notablecap @AnthropicAI @AWSstartups @neondatabase @cramforce @MultiOn_AI @DivGarg_ @mdshelton_ @JvNixon for judging + sponsoring! Want to try out computer use agents? Give @AgentOpsAI a try. And for more hacks follow @AlexReibman

@Andy_AJT @benswerd @theshortjacob @anchnk Now do the obvious, loop thisback to code writing. Et voila you just built a full stack coder.

@Andy_AJT @theshortjacob @anchnk Link to repo for those who are interested

@benswerd @theshortjacob @anchnk For all those asking heres the Open Source Repo for TestsAI Feel free to reach out if you want advice on setting it up.

@Andy_AJT @benswerd @theshortjacob @anchnk Removing the huge pain of tests, great !!!

@Andy_AJT @benswerd @theshortjacob @anchnk Is the source for “Tests AI” available?