e2e + jev from TypeSafe AI ⚡ I'm building an open-source framework for running e2e tests with agents. supports web, mobile (and more!) available soon:

@typesafeai what the heckkkk

@typesafeai hot

@typesafeai can it reproduce a failing test before fixing it?

@typesafeai Really cool bro🔥

@typesafeai sheesh

@typesafeai Sick. Any timelines?

@typesafeai October 1st 👀

@typesafeai early access possible? could help you with bug fixes + feedback (if any)

@typesafeai jev in the e2e loop is where flake shows. same scenario ×10 high conf flakes from dom timing vs boolean ready gates

@typesafeai another builder picking jev as the agent layer for e2e. the dom snapshot approach keeps beating screenshot loops on reliability. what handles the mobile side, appium-style drivers or something custom?

@typesafeai Signed up - appreciate what you are doing for us.

@typesafeai is Jev classifying pass/fail after the run, or driving the clicks? a classifier I can slot in. a Jev-driven UI loop is a different harness.

@typesafeai Cool! I did this as an experiment - potentially some possibilities for your product. npm install runora npx runora init Let me know what you think!

@typesafeai oh my god, that would be the most usefull thing. i need that.

@typesafeai as ai apps grow more complex how will testerArmy tackle the challenge of flaky tests and false positives ensuring reproducibility at scale without slowing teams down

Congrats on shipping this. Agents fit e2e better than most places people put them, since a test is already described as intent. The part worth designing for early is the second run. A deterministic suite fails loudly when a selector moves. An agent driven one quietly succeeds by finding another path, which is great for maintenance and rough for diagnosis, because a real UI regression looks identical to a self heal. Recording the action trace the agent chose and diffing it against the last run turns that into a signal instead of a mystery.

@typesafeai agent e2e on mobile is where i burn days. selectors dying mid-run hurts more than the app flake.

@typesafeai As LLM are deterministic languages we don't need be worry about run twice e2e tests with Jev if the first attempt fail. Worst case, imagine if we decide to use typescript + jest or maybe common js instead. 👀 it would be a nightmare to see milliseconds output.

nice. the failure mode id watch for on agent-e2e is treating the clicker and the oracle as one model. keep actuation (playwright/agent) separate from judgment: Jev as pass/fail Choice + needs_human Noul on the trace/screenshot/diff state, with a confidence floor so flaky agent noise doesnt become a green build. shadow that gate before you trust it on CI.

@typesafeai I need this for GUI apps

@typesafeai Wait, e2e is some kind of new framework!? Whole life I expected e2e as End to End aka some custom tests 2nd time I see it in my twitter timeline, what its about?

@typesafeai how does it handle state persistence between agent steps during long runs