Here is my experimental agent QA walking idea. Agents walking and testing Omarchy with little direction other than "You are the theme tester" (its much more extensive than that, but this is X, no room for such nonsense).

four agents poking at omarchy at once, that's a saturday afternoon rig honestly

i technically have 30, but it gets really hard to know what the heck is going on when I do that :)

Microsoft has been shipping this QA strategy for years They just call it Windows Insider 🤣

You bastard! You are taking peoples jobs using AI to test Omarchy.

the agents when they find out omarchy has a light theme

Hi Prime, does this run locally or in the cloud?

yes

The failure mode I've hit with loosely-prompted QA agents is confident false positives - one of mine reported "all done" on a batch of tests that turned out simulated, not actually run. Now I require pasted raw test-runner output, never a self-reported pass.

If you ever try something similar on the side of mobile (emulator/connected physical devices) then do share them as well. I currently feel that QA self healing automations are better on desktop but fail in case of mobile development, since mobile is bottlenecked by adb (considering android) limitations. Plus testing them in stage networks where we need to configure proxy is real pain in the a**. I am myself working on its end to end QA automation

my agents only test my patience 😭

I love how AI pilled you have become :p

"you are the theme tester" and the agent files a bug that just says "too much purple". honestly fair

The role instead of a checklist is the interesting bit. A script only checks what you already expect; a wandering theme tester stumbles into the rest.

Cool! Are you doing dev livestreams while working on omarchy QA? Enjoyed the ones you did in early September on this topic :-)

So the OS has an "MCP"-like API, allowing an agent to do I/O like a user would do with a mouse and a keyboard? And the agents are using that to test various user journeys?

that's awesome. I am building a design tool (like figma) optimized for omarchy (with mcp) that can also design themes, so that would be perfect for QA😍

share the prompt porfavor

This is cool 👀 Does it follow a set of predefined steps, or does the agent come up with its own test scenarios every run?

does the theme tester actually have taste or does it just say every theme looks great

agent as QA tester is the NPC mode we didn't know we needed

the agents are walking before i am

Nice i had my agent write a fuzz testing thing for mine

giving an agent a vague role and letting it wander is a better bug finder than most scripted tests. humans stop looking where they already expect things to work

a low contrast theme and a broken one look a lot alike to the theme tester, so most of the work is teaching it which ugly is on purpose

Day three and it only wants to test Tokyo Night

how does it decide a theme's broken, screenshot diffs or the model eyeballing it

This is X pretty pictures and videos only please...

That's Hendrickson's exploratory testing charter: explore [target] with [resources] to discover [information]. Now the testers never get bored.

We're all building the same thing 😭😂😭😂

you gave an ai agent a desktop and told it to explore. that's not qa anymore, that's putting a raccoon in an office and observing the damage

what does it report back , screenshots of broken themes or actual bug notes ?

theme testing is a good stress case because the bugs are visual. no stack trace to latch onto, the agent has to actually look at the screen

letting it just wander the app finds stuff scripted tests never would

Giving each agent one narrow role like theme tester is the clever part. A generalist QA agent reports everything; a specialist that only cares about themes catches the contrast bug a human stops seeing after day two.