inspired by Paper: claude code driving the tldraw desktop app

I haven't set up an MCP server for this BTW. Instead, I'm exposing a local server with some endpoints to get data from or post code that runs in a scope with the editor + helpers. Good chance to try out the code mode theory @threepointone @KentonVarda

@threepointone @KentonVarda we might end up with an MCP anyway but the model seems to be pretty good with just using the editor API. There's a LOT that the editor can do and llms are pretty good at tldraw code

skipping MCP and just exposing a local server with endpoints is honestly the smarter move for this kind of integration. MCP adds a layer of abstraction that makes sense for generic tool use but when you already have a rich editor API... just let the model talk to it directly. less overhead, faster iteration, and the model can use the full API surface instead of whatever subset you decided to wrap. curious how the screenshot + shape data combo works for spatial reasoning

@paper Yeah exactly!

@paper wait this is so cool. and the local server approach instead of MCP makes total sense for visual tools. tighter feedback loop, less abstraction in the way 👀

@paper Claude Code + tldraw is the ultimate prototyping combo. Wild how fast you can go from idea to visual.

@paper which branch is this on?

@paper so satisfying

tldraw's agent kit gives Claude Code two inputs per turn: a screenshot of the canvas + structured shape data (positions, types, IDs). from that it can create/update/delete shapes, draw strokes, batch-align elements, and move the viewport. essentially: the canvas becomes a writable tool environment, not just a view.

@paper @excalidraw

@paper You can try my free flow chart drawing service free 5 api and you can use your own api url and sk, (including and excalidraw) , i filled a lot tools for ai ,

I've seen developers underestimate the power of code generators like Claude Code, focusing solely on custom-built solutions. But think about it - what if you could unlock a world of possibilities by creating a platform that lets designers and developers work together seamlessly? That's where the real innovation lies, in bridging the gap between creativity and code.

@paper tldraw has desktop app?

@paper works with other harnesses too? Like pi, codex, open code?

@paper Anything yeah, just a local endpoint

@paper very nice! :)

@paper

@paper How can one setup this?

@paper We have a desktop app with releases here, though the source is (currently) private. I'm just developing on a branch atm but it seems like an obvious feature to ship!