Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

co-seeing art with machine: an interactive demo where i browse the visual responses of a multimodal llm to prompts about the painting, from “who’s in here?” and to “what’s unusual about this painting?” the model replies not just with words, but with visual annotations as well.

Kat ⊷ the Poet Engineer

84,710 subscribers

37,359 views • 1 year ago •via X (Twitter)

Arts Science & Technology Education

Anya Rossi• Live Now

Private livecam show

10 Comments

Ryan Sudhakaran1 year ago

Feels like how a Sherlock Holmes/House/Monk type character would view a scene

☼░▒▅1 year ago

your stuff is so sick!

λlex1 year ago

everytime I see your projects I want to play with them myself!

Jackywine1 year ago

future

good1 year ago

incredible 🤍

Silicon Jungle1 year ago

nice

Kris Crawford1 year ago

So sick

Ian Evers1 year ago

Hi! is there a github for this?

Nigel Higgs1 year ago

Your hand gesture demos are very smooth, I would love to have a local version to control my computer with.

Colton Batts1 year ago

We’re doing the coolest things on the Internet and it’s kinda not even close.

Related Videos

Gemini 3 brings new capabilities to Google Gemini including visual layout and dynamic view for more visual and interactive responses. Check it out in action with these 3 prompts:

Gemini 3 brings new capabilities to Google Gemini including visual layout and dynamic view for more visual and interactive responses. Check it out in action with these 3 prompts:

Google

75,218 views • 7 months ago

Not just painting, this is art with an insane level of detail.

Not just painting, this is art with an insane level of detail.

sc🧚🏻‍♀️

4,489,752 views • 3 months ago

mapping out the visual language of film using a multimodal llm: i fed frames of a short film to a vision-language model and mapped out its ratings of surrealism and presence of human figure in each moment along the timeline. the result is an interactive playback interface based on these 2 dimensions:

mapping out the visual language of film using a multimodal llm: i fed frames of a short film to a vision-language model and mapped out its ratings of surrealism and presence of human figure in each moment along the timeline. the result is an interactive playback interface based on these 2 dimensions:

Kat ⊷ the Poet Engineer

94,701 views • 1 year ago

More tests of painting with a realtime ai transformation. This time, the model settings ensure that the ai stays closer to the painting, which gives me more control over the final look of things. Eventually, we should be able to get as fine-grained control as we want with our painting.. to be able to put most of our energy into painting details in areas that matter the most in the composition. I'll post below how I got this to work. #painting #ai

More tests of painting with a realtime ai transformation. This time, the model settings ensure that the ai stays closer to the painting, which gives me more control over the final look of things. Eventually, we should be able to get as fine-grained control as we want with our painting.. to be able to put most of our energy into painting details in areas that matter the most in the composition. I'll post below how I got this to work. #painting #ai

Martin Nebelong

594,441 views • 2 years ago

Guys, this is the future of Digital Painting...!!! I combined LCM (a real-time generative model) with Photoshop and you can create amazing art with just a few brush strokes!

Guys, this is the future of Digital Painting...!!! I combined LCM (a real-time generative model) with Photoshop and you can create amazing art with just a few brush strokes!

Cristian Peñas ░░░░░░░░

72,702 views • 2 years ago

Got the vibe but not the words? Starting today, you can simply show or tell AI Mode about what you're looking for and get rich, visual results to explore. ✨ This experience works by bringing together Search’s world-class visual understanding from Lens and Image search with Gemini 2.5’s advanced multimodal capabilities. When you show AI Mode what you’re searching for it uses our new “visual search fan-out” technique to get a deeper understanding of what’s in an image (including subtle details and secondary objects), and runs multiple queries in the background. This helps it understand the visual context and nuance of your request to deliver relevant visual responses. You can also tell AI Mode what vibe you’re going for (no image needed). For example, ask AI Mode for bedroom design inspo and go back and forth to refine your vision. We’re rolling out this experience in English in the U.S. starting this week. Try it out and let us know what you think!

Got the vibe but not the words? Starting today, you can simply show or tell AI Mode about what you're looking for and get rich, visual results to explore. ✨ This experience works by bringing together Search’s world-class visual understanding from Lens and Image search with Gemini 2.5’s advanced multimodal capabilities. When you show AI Mode what you’re searching for it uses our new “visual search fan-out” technique to get a deeper understanding of what’s in an image (including subtle details and secondary objects), and runs multiple queries in the background. This helps it understand the visual context and nuance of your request to deliver relevant visual responses. You can also tell AI Mode what vibe you’re going for (no image needed). For example, ask AI Mode for bedroom design inspo and go back and forth to refine your vision. We’re rolling out this experience in English in the U.S. starting this week. Try it out and let us know what you think!

Google AI

68,179 views • 9 months ago

Meet MoshiVis🎙️🖼️, the first open-source real-time speech model that can talk about images! It sees, understands, and talks about images — naturally, and out loud. Voice interaction with a compact model endowed with visual understanding opens up new applications, from audio description for the visual impaired to visual access to information. Try it out 👉 Blog post 👉

Meet MoshiVis🎙️🖼️, the first open-source real-time speech model that can talk about images! It sees, understands, and talks about images — naturally, and out loud. Voice interaction with a compact model endowed with visual understanding opens up new applications, from audio description for the visual impaired to visual access to information. Try it out 👉 Blog post 👉

kyutai

47,924 views • 1 year ago

im not gonna lie i never thought about castle model kits but that is actually so impressive, seeing the weathering, lining and painting just for the stone walls alone just makes it look so cool as a diorama

im not gonna lie i never thought about castle model kits but that is actually so impressive, seeing the weathering, lining and painting just for the stone walls alone just makes it look so cool as a diorama

delty

14,224 views • 1 year ago

Sometimes my process is just starting with chaotically laying colours down and seeing where the painting decides to take me

Sometimes my process is just starting with chaotically laying colours down and seeing where the painting decides to take me

yue

226,559 views • 5 months ago

Episode 6: The Visual Artist With every flick of #justin's brushstrokes, there's a story that unfolds. And in every painting he creates, there's an inspiration to dream & to believe just like he did. Dipaphy Hartby justin 💚 #InspiredbyJustin #KaarawanNiSB19Justin

Episode 6: The Visual Artist With every flick of #justin's brushstrokes, there's a story that unfolds. And in every painting he creates, there's an inspiration to dream & to believe just like he did. Dipaphy Hartby justin 💚 #InspiredbyJustin #KaarawanNiSB19Justin

KenTin Kindle 😸🍎

20,583 views • 11 months ago

Koko the Clown singing and dancing "St James Infirmary" with the voice and rotoscoped body of Cab Calloway is about as good as visual art gets.

Koko the Clown singing and dancing "St James Infirmary" with the voice and rotoscoped body of Cab Calloway is about as good as visual art gets.

old toons

75,687 views • 1 year ago

Just as Google made the internet searchable, we're making the real world searchable with Visual Language Model🌍 Meet NATIX WorldSeek — an interactive platform where you can search our multi-camera database to find what matters to you. Full announcement👇

Just as Google made the internet searchable, we're making the real world searchable with Visual Language Model🌍 Meet NATIX WorldSeek — an interactive platform where you can search our multi-camera database to find what matters to you. Full announcement👇

NATIX Network

62,043 views • 6 months ago

Just played this mod called "Pitch Black" which made by the same creator of "Solomons Seared Symphony" It's a demo atm, but similar to SSS, I'm in love with the art style and overall visual style. Music here is catchy as hell too! Give it a try! "Phony" by Adamb & NexusVGM

Just played this mod called "Pitch Black" which made by the same creator of "Solomons Seared Symphony" It's a demo atm, but similar to SSS, I'm in love with the art style and overall visual style. Music here is catchy as hell too! Give it a try! "Phony" by Adamb & NexusVGM

Daily FNF Reviewer (Synecdoche Funkin ???%)

20,743 views • 1 year ago

DeepMind just dropped Lyria, a state-of-the-art music generative model. Last year, I said 2022 was the year of pixels and 2023 would be the year of soundwaves. We are making great progress here! The most impressive demo is converting humming to a full instrument suite. I think Lyria will unlock all the operators we are used to in image models: text-based editing, style transfer, in-painting (fill out tracks), out-painting (continue a track), super-resolution, etc. Lyria is deployed as an intuitive software tool to musicians, in partnership with YouTube. This is the right move: ship the model! With enough artists on board, Lyria could spin a data flywheel that learns from the artists' feedback and editing signals. Congrats to DeepMind for shipping! Launch blog:

DeepMind just dropped Lyria, a state-of-the-art music generative model. Last year, I said 2022 was the year of pixels and 2023 would be the year of soundwaves. We are making great progress here! The most impressive demo is converting humming to a full instrument suite. I think Lyria will unlock all the operators we are used to in image models: text-based editing, style transfer, in-painting (fill out tracks), out-painting (continue a track), super-resolution, etc. Lyria is deployed as an intuitive software tool to musicians, in partnership with YouTube. This is the right move: ship the model! With enough artists on board, Lyria could spin a data flywheel that learns from the artists' feedback and editing signals. Congrats to DeepMind for shipping! Launch blog:

Jim Fan

384,184 views • 2 years ago

One-shot translation from GLSL to React, followed by a few refinements to create this visual with v0 (Opus 4.8). See how the model interpreted and validated my prompts through auto-captured screenshots. Full conversation here:

One-shot translation from GLSL to React, followed by a few refinements to create this visual with v0 (Opus 4.8). See how the model interpreted and validated my prompts through auto-captured screenshots. Full conversation here:

Yohei Nishitsuji

16,870 views • 1 month ago

Machine Mental Imagery Empower Multimodal Reasoning with Latent Visual Tokens

Machine Mental Imagery Empower Multimodal Reasoning with Latent Visual Tokens

AK

43,498 views • 1 year ago

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here:

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here:

Andrew Ng

181,457 views • 1 year ago

“I can’t talk about it without crying.” Sondheim speaking about the painting at the heart of Sunday in the Park with George:

“I can’t talk about it without crying.” Sondheim speaking about the painting at the heart of Sunday in the Park with George:

The Sondheim Hub

38,387 views • 5 months ago