Martin Nebelong
@MartinNebelong • 33,751 subscribers
🎨 Exploring the latent space between creativity and Ai. Principal creative technologist at Google (through Yunojuno). Prev: Lumalabs, Media Molecule, LEGO etc.
Shorts
Videos
As you know, my explorations of the Gen AI space is ultimately all about creative control. You should be able to shape the generative matter using all your artistic sensibilities and your aesthetic sense. OpenAI's Sora is a huge technological leap, but what excites me the most about it is the modalities where it depends on input other than text alone. Such as video to video. Here's an example of how Sora can change an input video. Base video🧵
Martin Nebelong7,519,799 görüntüleme • 2 yıl önce
I had to do this of course - test out the new Google Omni Flash model with my favorite 3d art sandbox - Dreams. Just a quick test for now.. I'll be sharing more examples and techniques soon but wanted to show you this one. There's something very fitting about combining "handmade" 3d models and animations sculpted from scratch in software that runs on 2013 hardware, with some of the most advanced AI technology in the world. As part of my work for Google, I've used Google V2V models on a few different projects now - Ancestra, Dear Upstairs Neighbors and most recently one of the IO films (I'll share more about that project soon) 🥰 - and video continues to be my favorite form of input for Generative Ai workflows. If an image says more than a thousand words, a video says more than a million. And knowing that these capabilities are now in the hands of all of you, makes me very excited to see the new workflows and projects that will come out of it. What do you think of Omni so far?
Martin Nebelong37,667 görüntüleme • 14 gün önce
Here's a quick test of a new tool called Krea. It's like the Real-time latent consistency model tests I've done with my photoshop and Dreams creations, but instead in a single application with faster refresh rates and higher quality output. I find that it strays too far from the drawing, and the editor is still quite limited but it's interesting to see nonetheless. It's not too difficult to imagine tech like this built right into Photoshop. And yes, I do know how to draw even if it doesn't show here 😅, I've been sculpting, painting and drawing for most of my life and I think it's exciting that tools like this will eventually help us break down some of the technical walls between our imagination and the visuals we create. And I'm not necessarily talking about drawing alone here because I don't see that going away as an art form, and I wouldn't want it to. But a hybrid of different art forms merging into a new, immersive art medium where anything is possible. Want to change the camera angle of your painting? Sure, go ahead. Want to make an animation of moving the camera around and seeing the light change in the scene as you do? In the example here, being able to move things around, change lighting and see it all reflected in the resulting image, is pretty amazing. Like a combination of the benefits of working in 3D, but with 2d tools. The moving version at the end of the video was made using Runways Gen-2. #art #ai
Martin Nebelong1,668,143 görüntüleme • 2 yıl önce
For generative AI to become an interesting art tool, we need much more control over the output. The slot-machine-like nature of pure text-to-image leaves too much to chance. Using the "Real-time Latent Consistency Model" that I'm using in the example here, is the first time I truly got a glimpse of a future where we'll be able to use our artistic skills and sensibility, to get control over AI image gen. Systems like these will never be able to match the quality or originality of a skilled artist, it won't surprise us in the same way an artist can. Things are a mess in terms of the training data these models are based on, and the questions about copyright concerns and about a time when everything will look the same are very valid. At some point capabilities like these will be embedded in photoshop, and anyone will be able to generate a pretty picture. But to create interesting designs, to tell original stories and to surprise us, we need creatives and artists with something on their mind. We'll be able to create immersive worlds, by making brush-strokes and sculpt marks, without needing to worry about all the dials, plugins, wires of our 3d and 2d tools today. I love to sculpt, I love to draw, and I love to explore new mediums and new ways to create. The Gen AI tools we have today are far from perfect, and things need to be steered in a better direction. For that we need artists to help point the way. Gen AI isn't going away.. it's too powerful and has the potential to allow us to tell stories like never before. Like all other big technological shifts, tech like this will come at a cost, but it will also open up new opportunities and empower a new generation of storytellers. I might be naive, but I believe that human ingenuity and creativity will persevere in this new world ♥️ #art #ai
Martin Nebelong1,659,721 görüntüleme • 2 yıl önce
With Veo 3 and Flow out in the world, here's a few examples of videos I've created with Veo 3. The first video is an example of the incredible voice/audio capabilities. The second one is a test of doing a longer form video (edited in Premiere). Generated with Veo.
Martin Nebelong295,965 görüntüleme • 1 yıl önce
More tests of painting with a realtime ai transformation. This time, the model settings ensure that the ai stays closer to the painting, which gives me more control over the final look of things. Eventually, we should be able to get as fine-grained control as we want with our painting.. to be able to put most of our energy into painting details in areas that matter the most in the composition. I'll post below how I got this to work. #painting #ai
Martin Nebelong594,387 görüntüleme • 2 yıl önce
Adobe.. has released a tool that combines Generative Gaussian Splats with a Diffusion layer and it's not all over the internet? WHAT IS GOING ON :D I had to test this out ofc! Substance 3D viewer, the new 3d viewer just released by Adobe, not only supports viewing of a large, large list of 3d formats, but it also supports generating of 3d models as Gaussian Splats. Combine that with the built-in 3d to image functionality and you basically have at least a few of the steps I've been doing for the last year or so using multiple tools. Exciting stuff! The new Photoshop Beta also supports the 3D viewer and you can import 3d files directly in photoshop as smart objects linked to the viewer. The elements you prompt in based on your models, can be easily exported to your clipboard without backgrounds making it very easy to use this in your normal image editing workflows. I have a suspicion that we'll see much more of this in other Adobe tools and I'm very curious to see this used for things like this in more 3d tools than "just" Adobe Neo (which is fantastic btw). #adobe #art #gaussiansplatting
Martin Nebelong279,919 görüntüleme • 1 yıl önce
Generating 360 footage in Veo3 and watching it on my VR headset feels like scifi 😍 Just add (360° to your prompt)
Martin Nebelong175,572 görüntüleme • 1 yıl önce
This is real-time.. The debate about Nvidia's latest neural rendering tech is ongoing. One side sees the potential of the technology to bridge the gap across the uncanny valley, to give us a much less rigid creative process than a to b to c.. the other side shouts "slop" and tries to shame any creative who dares to explore this new technology, any chance they get. Meanwhile, I'm over here saying "I told you so".. Gen Ai is here to stay, its becoming real-time and Nvidia now has the "RTX on" for AI which I've been talking about for years. We need creatives to help shape meaningful tools that combine this tech with much more granular creative control.. with tools where you as an artist see your creative thumbprint from beginning to end in the parts of the work that matters the most to you. If you're shaming artists or storytellers who embrace these tools, you're doing them a disservice in my opinion. If they don't learn to utilize these tools and workflows, to make them their own and to shape them, others will. There's no way this tech won't be a huge part of the creative industry and of the way we tell stories. It's happening around us as we speak. That said, at the core of it, do what makes you happy and create in any way you like. Don't let people tell you that "you can't do this" or "you have to do it like this".
Martin Nebelong42,747 görüntüleme • 2 ay önce
![Veo has an incredible "hidden" ability. It's one of my absolute favorite aspects of Veo 3 i2v. It's ability to transport elements through the latent space, with this simple but very powerful prompt structure: “Instantly jump/cut on frame 1. [Describe the new context]" With this structure, I was able to use my quick drawing as the main subject in all the scenes you saw here. Seeing how the model is able to retain my personal sketch in all of these different contexts.. Feels magical honestly 😍 Music from My personal opinion about the question asked here, should be clear if you've followed along with my posts 🔥🎨](https://image.24vids.com/tw-1945128949438197784/ext_tw_video_thumb/1945128597372432384/pu/img/b8oBAiuvTzgTXO6T.jpg)
Veo has an incredible "hidden" ability. It's one of my absolute favorite aspects of Veo 3 i2v. It's ability to transport elements through the latent space, with this simple but very powerful prompt structure: “Instantly jump/cut on frame 1. [Describe the new context]" With this structure, I was able to use my quick drawing as the main subject in all the scenes you saw here. Seeing how the model is able to retain my personal sketch in all of these different contexts.. Feels magical honestly 😍 Music from My personal opinion about the question asked here, should be clear if you've followed along with my posts 🔥🎨
Martin Nebelong126,373 görüntüleme • 10 ay önce
Midjourney Office hours: They're working on a 3d system that will allow you to move into Midjourney images. Not polygons, not NeRF, not Gaussians. But a new NeRF-like format, with a team lead by someone who played an important part in the rollout of NeRF technology. Sorry polygons, but most Midjourney images already look better than most games today.. Agreed on the polygons.. they'll be a thing of the past in a few years I hope. If the person leading the work on Midjourney's new 3d format isn't Alex Evans I'll be very disappointed ;-) Take a look at this small video and consider that a very big part of the Dreams Graphics Engine was due to the work of Alex Evans. #art #ai
Martin Nebelong218,142 görüntüleme • 1 yıl önce
Look at the realism here! Hot take: This blows most 'old-school' CG humans out of the water. 😮 "But you have no control".. it's getting there! Using 'Facepoke' as input for Runway's video-to-video, it's clear we're entering a new era of creation. #AIArt #GenerativeAI #FutureOfCG
Martin Nebelong210,264 görüntüleme • 1 yıl önce
So the title might be a bit bombastic, but if this isn't one of the most incredible glimpses of a future way of creating and rendering 3D.. I don't know what is. This was sculpted on my PS5 in mere minutes and transformed as I was sculpting, through a "Real-time Latent Consistency Model". Sony, you should reconsidder your plans for Dreams and make a Dreams V2 with generative AI or sell it to someone that will.. just saying ♥️ No other 3D tool would allow me to sculpt this freely which turns out to be a super-power in the land of AI. #madeindreams #ai
Martin Nebelong240,608 görüntüleme • 2 yıl önce
