The open ai realtime api is sick! I hooked it up to control my browser so I could browse the web with my voice 🤯

btw if you are interested in using this sign up at we are working on getting it out to ya’ll

How in the heck are you doing page understanding, straight up from serializing the html, vision based, what magic is this? Surprised it works this well, I know it's a controlled demo, but still.

Yeah it is a custom dom format that is merging stuff from html and things we can get from the chrome debug transport. The raw html is too much context and doesn't work reliably on a lot of websites.

This is an awesome demo! It’s what I had in mind to evolve this demo into from a year ago, but I would have never gotten to where you are with this demo 😄

Demonstrating tech with impractical use cases undermines the value proposition. Voice AI for sandwich ordering: slower, costlier, no clear benefit. Choose demos that highlight genuine advantages and efficiency gains. Otherwise, it's just novelty without utility.

wow that’s insane 🤯 an amazing usecase would be E2E testing i would pay for it to test all the features of my SaaSs when i push some changes

pretty cool. I hope you release the extension.

we are working on it! spending a lot of my sanding down edge-cases so you can reliably throw it at stuff and trust it to fail gracefully

I do remember a websim websockets demo from ya that was pretty cool, don't I? :3

@bishal_ghimire Can we hook it up to ImageRenderer?