Using WebMCP tools, I can surface all sorts of functionality in my app. I can batch process, change themes and dark mode, check on higher level goals that I'm keeping track of. All of this functionality exists already! You can do it manually if you prefer. But you could... also do this by voice, and that's where some untapped potential for a11y comes in too. I like the flexibility of what this offers, maybe you do too.show more

did the same for two apps that i now use frequently- BingeWatcher (movie and tv-show discovery with lineups and watchlists) and ROUGH//CUT (in browser video editor) :))

Nice!

thanks a lot!

A couple people asked me why I showed chat and not voice- the answer is simple: I was sick and I sound like a frog. 🐸 I'll record one when I am not making ribbit noises

This is cool.

yes, I did some experimentation earlier. What I understood is WebMCP for the frontend, MCP is for the backend but for AI agents.

Yes, you’re correct

👍

WebMCP makes the UI an agent API without rebuilding the backend.

There are some very creative use cases for it too:

WebMCP that still works as a normal UI is the right constraint. Agent-only surfaces rot the moment a human needs to take over.

Not necessarily- I think we’ll see more handoffs between experiences over time. There are certain batch actions I might have an agent do and then want to step in to augment individual values, for instance. However, just surfacing the existing functionality is just fine too!

Is WebMCP maybe a good fresh take on accessibility?

Hey Sarah, I'd love to show you what we're building for WebMCP @Senro_AI.

;🍎💯

Voice control for dev tools sounds great until you're debugging out loud in a coffee shop.

I’m not doing voice control here, I’m just typing. That’s sort of the point- it’s flexible to your situation ☺️

Do you have a link to this? I'd love to use it for benchmarking and metrics on an article I'm writting.

Yeah I’m getting some last features over the line and then I’ll open source it

What's the chat UI on the right?

I made it a part of the app. I embedded Gemini and surfaced a UI that can be voice or chat

the a11y angle is the sleeper. app actions as callable tools is basically a machine-readable menu, which is exactly what screen readers always wanted and rarely got reliably.

1000%

I made my app webmcp enable as well, Quiak question what are using apart from codex to test and it and does it require API key ?🤔

I really like the view, and the image on top. Great job

we run enough browser agents to feel this: a semantic tool surface beats teaching the model to click pixels every time. the accessibility path becomes an interface you can test, not a demo you hope works.

does one undo roll back the whole batch, or would it step through each task change?

This is good to do client side tool calling. Makes sense. Really like it so far. But is this Copilotkit for the sideChat? PS. tc.

Thanks! Just Gemini

Are you running an LLM on the browser? Or still calling a frontier in this case?

I've embedded Gemini and made it part of the app experience. You don't have to do it that way, though

What does "embedded Gemini" mean? Is the model actually running in the page and driving the existing features through WebMCP? What's the difference in practice between embedding Gemini in the app vs just calling the API?

Yeah it's running in the page and driving existing WebMCP tooling. I exposed some special tools to do things my app can't do as well like batch processing or checking against larger goals. No big difference here, and you can still bring your own/drive it another way. The app is for me, so I wanted it as part of my experience.

Woah! Imagine if the whole OS was like this?! Wait a second, wasn't there a ChromeOS?

The voice part is really interesting. Being able to just say what you want instead of clicking through menus would help a lot of people. Have you tested it with voice yet, or is that still the plan?

Yes! I almost did a voice one, and will probably still post it. I've been sick this week and my voice sounds like a frog 🐸 ribbit

The accessibility angle is what caught my attention. Being able to say what you want done could make a complicated app feel much more approachable.