I asked Claude Fable 5.1 to paint the Mona Lisa with JavaScript.

How this was made: 1) Start with a photo of the actual Mona Lisa, so Claude and the JavaScript program have something to look at while painting. 2) Claude wrote a JavaScript program to paint. The program continuously compares the painting with the photo to decide where the next strokes are needed. 3) Each paint stroke is drawn by a custom paintbrush that Claude coded. 4) Claude used its vision capabilities as a critic. After every change, it rendered the painting into an image and looked at it. It did things like zooming in on the face and comparing it with the same part of the actual photo, then revised the code.

No fing way!! That's such a fascinating idea and result! 👏👏👏 This might just be the first "stroke by stroke" explanation of how the greatest art pieces where created (more or less, I guess. 😉) Could definitely be a cool course idea for art students.

Actually, I originally asked Fable to paint it stroke by stroke, the way Leonardo da Vinci would have painted it. Here's that version.

@JonnyJelinek You're new profile image is ready!

@kevin_t_ngo Took the liberty to upscale and change it a bit - now THAT could really be my new profile pic. 😄

@kevin_t_ngo very cuty

@kevin_t_ngo Ja, näh?! 😜

Damn actually ? that technique would be nice for a scrollytelling site

Yeah if I don’t run out of usage before the weekend 🥲

5.1 is as good as hungry in usage

Next level 🔥

Ty 🙏

If Leonardo da Vinci and Vincent van Gogh had a child

ooooh maybe parts of my code are in the training data for this :)

And people are still like “when agi?”

Amazing! Is it using a reference image, or doing it from memory?

Yeah, it looks at a reference image but every one of the ~15,000 strokes is planned and painted by code

Very cool!

How about svg?

I’m trying to conserve Fable usage, maybe next week 🥲

Has anyone tried hooking Claude up to one of those one-armed robots with some brushes and colors?

That goes to say nobody could paint monalisa better than vinci

woh!

woah Fable 5.1 is a pretty good javascript painter

Would love to see your setup here! What are you using for the brushes?

It’s plain JS on 2D canvas. No libraries. The JS code builds the brush strokes in many layers to make it look like a real brush stroke

So you're letting the agent draw using a tool that takes coordinates, a color & stroke width as input? And then you render it as a specific "texture stroke"? But you must have given the agents specific tools & strategies encoded in prompts/tools on top of that I assume?

Claude wrote a JS program that paints. It kept looking at output renders and revised its code over a few hours, until the painting looked right

I'd be very curious to know how it evaluated the results between each “look.” Did it write some kind of evaluation function; for example, one that compares histogram values? I’m asking because, in my setups, I’ve simply relied on the model’s vision capabilities, but I haven’t achieved results anywhere near as good as what you’re seeing here! :)

Both. It looked at output renders (vision capabilities) and compared them to an image of the Mona Lisa. It also wrote a scoring algorithm/function. Hope this helps with your side quest a bit :)

showing this to leo now

I'm assuming it just did some kind of genetic algorithm thing. It didn't, like, manually generate the brush strokes.

Receba

Why? The Mona Lisa was already painted centuries ago.

leaving this comment here so I get notified by the community notice in case

These image-via-code demos read as sketches because the model writes shapes blind, never seeing the canvas it's drawing on.

What an incredible idea!

Is that how diffusion works?

Now do Astra

Reminds me of the time I asked @grok about the nature of consciousness:

let's ask to complete the painting, i heard da Vinci didn't complete it

Nice, but not quite there yet.

What the FUCK!??

傻逼玩意，还我血汗钱，曹尼玛的

it's like watching the mona lisa getting painted

Sorry, but actually it created an image of the painting. Just so people know…it didn’t actually paint the real Mona Lisa since it was already painted many years ago. AI isn’t capable of doing things like that…yet?

javascript mona lisa is unreal

Wow

it’s like watching mona lisa being painted

Now post another with GPT Astra and you've winning content for 2 posts.