i tested agent-desktop (opus 5.5) vs. cua (opus 5.5) and vs. codex's computer use (terra 6.1) agent-desktop finished 26 of 35 jobs on my mac cua finished 33, codex finished 34 the jobs it missed were dragging, clicking on a canvas and scrolling a huge list, all with its... mouse-free mode let it take over the real mouse and it did all three, one try each i'm sticking with cua and codex's computer use for now, they are more robust and faster (cua is my own tuned build)show more

this is great! surface for improvement is always there. will push more reliability fixes soon!

great project i will keep an eye on it as you make improvements keep it up 👊🏻

🫡

sickkkkk great benchmarking very necessary with all these tools coming out i'm pretty happy with agent-desktop tho it does a lot of browser specific stuff very well

Are you by any chance bothered by foreground control?

What is cua?

best open source computer use available at the moment

bro this looks crazy but do one person needs all this at one time?

not sure what you're asking i'm trying to find the best computer use so i can set it up and forget about it so no, you don't need all of this at once, but if it can't do something from the test, it'll fail you eventually

Same for me, Cua got the edge over agent-desktop in my testing

wait for mine, i got it so deeply integrated with custom linux destop environment that owns the composer

but does codex already support linux computer use?

Is that terra on fast mode or regular mode?

regular on medium

For a first release those are solid numbers

yh

canvas and drag jobs are where I would drop computer use altogether, if the app has any api at all thats the path I would script

Useful comparison, thanks. Interesting that the misses were all canvas, drag and long-list scrolling. Feels like mouse-free modes still struggle with anything that isn't a clean UI element.

This is so fkn useful thx. Do you by any chance log the cost as well?

i'd rerun those same failed jobs with automatic fallback to the real mouse, and count the human interruptions. curious whether the practical gap is mostly detecting when to switch modes, rather than completing the task once switched.

What is this benchmark? I also got a custom one would like to try it

26/35 这个差距本身说明不了优劣：丢的任务集中在拖拽、画布点击和长列表滚动，都是无障碍树覆盖不到的操作。无人值守场景下，失败上报的诚实度决定了这套工具能不能真的放手运行，静默谎报完成会让运行失去可信的失败信号。

when mouse-free mode failed those jobs, did agent-desktop still report them as done?

It does struggle on non accessibility tree apps tbf

Terra is a beast. I also felt it once I handed a computer task and saw it quickly start taking over.

maybe we got to get you to AgentWeb (check pinned :))

were the misses the same jobs for all three? if dragging and canvas fail everywhere i'd blame the screenshot and click loop more than the model

@Legendaryy

the misses being drag and canvas stuff makes total sense anything without a clean accessibility tree is still pixel guessing so mouse free mode will always hit that wall

Thanks for posting this. Was just about to try it out!

Who was cheapest? Agent desktop uses accessibility rather than screenshots to navigate, that is why I was interested in it.

Did you test it in a Mac scoped apps env or VM. Wonder if there is difference in speed?

how did you score the 35 jobs: checking the final state on the mac yourself, or taking the agent's own report of done? curious if the 9 misses hold up under the same check.

Split the 9 misses before you rank it. A miss it reports as a miss is safe: retry it or hand it the real mouse. A miss it reports as done is the risk, and a mouse-free mode that cannot drag cannot tell it failed. For set-and-forget work, that false-done count is the real score.

most tests skip the part where it fails, so this one is useful