Loading video...

Video Failed to Load

Go Home

Release Day 🎉 Meet OpenApps — a pure-Python, open-source ecosystem for stress-testing UI agents at scale. Runs on a single CPU. Generates thousands of unique UI variations. And it reveals just how fragile today’s SOTA agents are. (Yes, even GPT-4 and Claude struggle.)

10,143 views • 10 months ago •via X (Twitter)

5 Comments

Dr. Karen Ullrich's profile picture
Dr. Karen Ullrich10 months ago

Why OpenApps? ✅ Unlimited data for evaluation and training ✅ Lightweight — single CPU, no Docker, no OS emulators ✅ Reliable rewards from ground-truth app state ✅ Plug-and-play with any RL Gym or multimodal LLM agent

Dr. Karen Ullrich's profile picture
Dr. Karen Ullrich10 months ago

This isn’t just a benchmark — it’s a generator. 🏭 With simple YAML configs, you can spawn thousands of variants of 6 fully functional apps. Swap themes, scramble layouts, inject “challenging” fonts… If your agent really understands UIs, this will show it.

Dr. Karen Ullrich's profile picture
Dr. Karen Ullrich10 months ago

⭐️ 📖 🛠️ Want to build new apps or tasks? Contributions welcome. Let’s push UI agents forward — together. 🚀

Arsh's profile picture
Arsh10 months ago

fascinating, so you're saying that even the current state-of-the-art models like GPT-4 and Claude are fragile under massive UI stress testing. what's the average time-to-failure (TTF) you observed for these models in OpenApps tests, and how does it compare to a baseline of non-stressed model performance?

Pranav Pawar's profile picture
Pranav Pawar10 months ago

Appreciate the simplicity of this framework!👏

Related Videos

Anthropic's most viral feature is now open-source! Until now, Anthropic's Generative UI capabilities only existed inside its own products. CopilotKit🪁 just shipped Open Generative UI, an open-source implementation of Claude Artifacts that works in any app. The agent generates HTML/SVG at runtime, and CopilotKit streams it token-by-token into a sandboxed iframe inside the app's chat. So the user can watch the UI assemble itself in real time, not after the full response is ready. The sandbox is fully isolated with no access to the parent app, the DOM, or user data. So if the agent hallucinates broken markup or unexpected JavaScript, nothing leaks outside the iframe. Under the hood, the agent does not select from pre-built components. Instead, it generates arbitrary visuals from scratch every time. The output is unconstrained by default, but you can shape it by defining prompt-based skills that teach the agent specific visual formats or guidelines. For instance, a skill prompt can guide the agent toward producing a Chart.js dashboard with proper axis labels and responsive sizing, or an interactive 3D model with rotation controls. The video below shows this in action, and the output quality you see actually comes from the skills layer. Open Generative UI runs on AG-UI, so it works out of the box with LangGraph, CrewAI, Mastra, Google ADK, AWS Strands, and more. It also ships with a standalone MCP server that plugs into Claude Code, Cursor, or any MCP-compatible client. And the entire stack is built on top of CopilotKit, the open-source frontend framework for agents and generative UI. 30k+ GitHub stars, with SDKs for React, Next.js, Angular, and Vue. I have shared the GitHub repo and a live playground in the replies!

Akshay 🚀

87,048 views • 5 months ago