Skip navigation
You can build gui tools that will send messages back to codex (debugging visual issues in my math…
0:00 / 0:29

You can build gui tools that will send messages back to codex (debugging visual issues in my math…

Steve Ruiz

@steveruizok61K followers

Visit on X
210 likesDownloadOriginal post

17,262 views

You can build gui tools that will send messages back to codex (debugging visual issues in my math game)

19 Comments

Comments from the original post on X (Twitter)

  • Artemi Vaarakallio's profile picture

    @aaapinen

    finally the screenshot can explain itself. gui state back into codex is such an obvious missing loop.

    1 like

  • Silicon Jungle's profile picture

    @JungleSilicon

    yes! i love making custom ui's that talk to codex server!

    1 like

  • Vinny's profile picture

    @hot_town

    How does sending messages to codex work? I tried this with Claude code but had to use a loop to essentially poll the app and pick up waiting messages

    1 like

  • Steve Ruiz's profile picture

    @steveruizok

    I'm told it uses the queue command?

    2 likes

  • Jonas Templestein's profile picture

    @jonas

    It’s so good. I did this with a 3d rendering task. Just fly through the model of our property and shoot a gun to annotate where the model isn’t right and then codex fixes it

    1 like

  • Rik Arends's profile picture

    @rikarends

    Yea thats how my visual debugger in makepad works as well

    1 like

  • LH's profile picture

    @lh________hl

    Lovely. Have you got a tutorial on how to do this?

  • Nilu Kulasingham's profile picture

    @nilukulasingham

    is this publically availiable?

  • Aaron Soules's profile picture

    @a10s_code

    Love this. How does it work?

  • Ben Mo's profile picture

    @benmodev

    pinning the comment to one animation frame is huge. otherwise 'fix the arm' can somehow turn into a redesign of the entire character lol

  • Blixt's profile picture

    @blixt

    this is great! doesn't it bother you to use a terminal for an increasingly gui world tho? it really grinds my gears that we're actively trying to reimplement modern gui in a 60-year-old environment lol

  • Steve Ruiz's profile picture

    @steveruizok

    As long as I need to have the code open (90% for CSS and copywriting) i might as well use the terminal. I like the apps too but they're hogs

  • Blixt's profile picture

    @blixt

    yeah i rebuilt a harness for myself to get rid of said hogs to get out of the terminal, so i just use zed for looking at code these days... i just find it wild that claude code has hover effects and dividers and other complex ui, while giving you janky text selection etc

  • Francisco Gomez's profile picture

    @fgomezserna

    The underrated detail: the tool talks back in Codex's format - element, frame, coordinates - not prose. Screenshots explain to humans; structured state makes the fix reproducible for the agent. Closing that loop both ways is what agent GUI tooling should standardize on.

  • Harsh Manwani's profile picture

    @harshmanwanii

    Circling the broken pixel instead of typing a paragraph describing it is such an obvious win I am annoyed I did not think of it.

  • Neel's profile picture

    @neel_sh_

    gui tools that talk back to the agent. debugging just got fun

  • Ben Dus's profile picture

    @beeendus

    Describing a UI bug to an agent can take longer than spotting it. I'd rather click the broken element and say "this should line up with that." Steve Ruiz's Codex demo makes me want interfaces that help you give the instruction.

  • Tenderbase's profile picture

    @Tenderbase_kw

    Excuse my ignorance. Is this Tldraw offline or something separately built on top of the Tldraw SDK?

  • Steve Ruiz's profile picture

    @steveruizok

    Something separate, just an ad hoc part of a different project

    1 like