Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Seeing the amazing new SMERF technology immediately made me imagine a time when we can walk around in environments like this, styled to be anything we can imagine. All happening in realtime, shaped with voice prompts or virtual paint or sculpt marks. Here's a test of using some of... show more

Martin Nebelong

33,632 subscribers

36,338 views • 2 years ago •via X (Twitter)

Arts Science & Technology Music

Anya Rossi• Live Now

Private livecam show

11 Comments

Martin Nebelong2 years ago

The paper and the real-time demos can be found here: It's incredible, really! Man.. wish I could capture my Dreams scenes like this. The AI transformed views of the demo spaces were upscaled and enhanced with Magnific ai. Here's a few examples.

Martin Nebelong2 years ago

In the video, I use scree capture to feed my browser window into @krea_ai for that realtime AI transformation based on a text prompt.

John Nack2 years ago

“@BilawalSidhu Mode, engage!” 🤩

Daniel Duckworth2 years ago

This is absolutely wild! The future is now.

kfant2 years ago

wen splat 2 splat?

Stone Vs The Metaverse2 years ago

I would have assumed that SMERF technology would only make everything small, blue and annoying.

Martin Nebelong2 years ago

Yeah that is a bit of a disappointment actually 😬

👾James (Game Dev) Anderson👾2 years ago

"It's like seeing the world through a new camera lens of imagination." I like that.

Brioche.ai2 years ago

Absolutely! The potential of SMERF technology is mind-blowing. The possibilities are truly endless!

Sylwester Mielniczuk 🔮💎2 years ago

I've never been into AI; it makes me sick after a day or two. I usually use a web UI pipeline (canvas drawing apps/webcam) to image2image (SD) on my M2, but the massive output feels disconnected, almost mocking. I am not a creator, I am AI operator, actor. It makes me really sad.

Martin Nebelong2 years ago

Ah sorry to hear you feel that way.. I think it's all about finding a way for us to use these tools without feeling like our creative soul is lost in the process. Haven't found the perfect workflow yet but I'll keep looking! Could you imagine a way to use these tools without feeling like you do now?

Related Videos

I've been a proponent of the idea of combining 3d and generative ai for a long time - as a way to get the best of both worlds: The precision and control of 3d - the detail, realism, lighting and speed of generative models. Here's another example of that idea, realized through a procedural system - entirely vibe-coded with Fable. Inspired by a photograph I took of an old tower in a town in southern France. In the first part of the video I'm using Krea AI's realtime model - later examples show how Nano Banana 2 can be used to take 3d stills and transform them to pretty much anything. What do you think of workflows like this compared to purely generating video or images from text, image or video input?

I've been a proponent of the idea of combining 3d and generative ai for a long time - as a way to get the best of both worlds: The precision and control of 3d - the detail, realism, lighting and speed of generative models. Here's another example of that idea, realized through a procedural system - entirely vibe-coded with Fable. Inspired by a photograph I took of an old tower in a town in southern France. In the first part of the video I'm using Krea AI's realtime model - later examples show how Nano Banana 2 can be used to take 3d stills and transform them to pretty much anything. What do you think of workflows like this compared to purely generating video or images from text, image or video input?

Martin Nebelong

20,265 views • 1 month ago

Ironheart portraying how dangerous the gross misuse of AI can be, how using the likeness of a person (dead or alive) to create a life like image can result in unhealthy attachments, inability to process emotions/grieve properly or disrespecting the memory of a deceased person.

Ironheart portraying how dangerous the gross misuse of AI can be, how using the likeness of a person (dead or alive) to create a life like image can result in unhealthy attachments, inability to process emotions/grieve properly or disrespecting the memory of a deceased person.

HOT GIRL CAP ⍟

168,087 views • 1 year ago

For generative AI to become an interesting art tool, we need much more control over the output. The slot-machine-like nature of pure text-to-image leaves too much to chance. Using the "Real-time Latent Consistency Model" that I'm using in the example here, is the first time I truly got a glimpse of a future where we'll be able to use our artistic skills and sensibility, to get control over AI image gen. Systems like these will never be able to match the quality or originality of a skilled artist, it won't surprise us in the same way an artist can. Things are a mess in terms of the training data these models are based on, and the questions about copyright concerns and about a time when everything will look the same are very valid. At some point capabilities like these will be embedded in photoshop, and anyone will be able to generate a pretty picture. But to create interesting designs, to tell original stories and to surprise us, we need creatives and artists with something on their mind. We'll be able to create immersive worlds, by making brush-strokes and sculpt marks, without needing to worry about all the dials, plugins, wires of our 3d and 2d tools today. I love to sculpt, I love to draw, and I love to explore new mediums and new ways to create. The Gen AI tools we have today are far from perfect, and things need to be steered in a better direction. For that we need artists to help point the way. Gen AI isn't going away.. it's too powerful and has the potential to allow us to tell stories like never before. Like all other big technological shifts, tech like this will come at a cost, but it will also open up new opportunities and empower a new generation of storytellers. I might be naive, but I believe that human ingenuity and creativity will persevere in this new world ♥️ #art #ai

For generative AI to become an interesting art tool, we need much more control over the output. The slot-machine-like nature of pure text-to-image leaves too much to chance. Using the "Real-time Latent Consistency Model" that I'm using in the example here, is the first time I truly got a glimpse of a future where we'll be able to use our artistic skills and sensibility, to get control over AI image gen. Systems like these will never be able to match the quality or originality of a skilled artist, it won't surprise us in the same way an artist can. Things are a mess in terms of the training data these models are based on, and the questions about copyright concerns and about a time when everything will look the same are very valid. At some point capabilities like these will be embedded in photoshop, and anyone will be able to generate a pretty picture. But to create interesting designs, to tell original stories and to surprise us, we need creatives and artists with something on their mind. We'll be able to create immersive worlds, by making brush-strokes and sculpt marks, without needing to worry about all the dials, plugins, wires of our 3d and 2d tools today. I love to sculpt, I love to draw, and I love to explore new mediums and new ways to create. The Gen AI tools we have today are far from perfect, and things need to be steered in a better direction. For that we need artists to help point the way. Gen AI isn't going away.. it's too powerful and has the potential to allow us to tell stories like never before. Like all other big technological shifts, tech like this will come at a cost, but it will also open up new opportunities and empower a new generation of storytellers. I might be naive, but I believe that human ingenuity and creativity will persevere in this new world ♥️ #art #ai

Martin Nebelong

1,660,565 views • 2 years ago

"Hah - generative ai can't even make an image of a hand with the right number of fingers.." "Stop pushing this slop" It's way past the point now where it must be clear to everyone, that generative ai is here to stay AND that the quality will continue to increase. I've been talking about this trajectory for years now, and I've been working towards finding ways to combine the strength of these models, with the best of what I love about "old-school" creation. Building with my hands, moving a pencil across the paper and seeing shapes emerge, moving a building slightly to the right to get just that composition I had in mind. Being fully immersed in a scene I'm building in VR, being inspired by the immersion to take the story in a new direction. For years I've been talking about how powerful the combination of 3d and generative ai is, be it traditional 3d, SDF volumes in Dreams or Gaussian splats - with experiments around using V2V as a "render pass" or with experiments around realtime ai. Enough talk you might think, where's the proof? It's all around us these days honestly and here's a small test I did during some OOO. Blender MPC + Fable - a pretty powerful combination! With a bit of Google Omni Fast on top as a "render" pass. What do you think of where this is heading? Hopeful, disheartened, inspired or the opposite? Can you imagine working with tools like this in a way where we still retain the human "spark" and the creative nerve that makes each persons creation unique?

"Hah - generative ai can't even make an image of a hand with the right number of fingers.." "Stop pushing this slop" It's way past the point now where it must be clear to everyone, that generative ai is here to stay AND that the quality will continue to increase. I've been talking about this trajectory for years now, and I've been working towards finding ways to combine the strength of these models, with the best of what I love about "old-school" creation. Building with my hands, moving a pencil across the paper and seeing shapes emerge, moving a building slightly to the right to get just that composition I had in mind. Being fully immersed in a scene I'm building in VR, being inspired by the immersion to take the story in a new direction. For years I've been talking about how powerful the combination of 3d and generative ai is, be it traditional 3d, SDF volumes in Dreams or Gaussian splats - with experiments around using V2V as a "render pass" or with experiments around realtime ai. Enough talk you might think, where's the proof? It's all around us these days honestly and here's a small test I did during some OOO. Blender MPC + Fable - a pretty powerful combination! With a bit of Google Omni Fast on top as a "render" pass. What do you think of where this is heading? Hopeful, disheartened, inspired or the opposite? Can you imagine working with tools like this in a way where we still retain the human "spark" and the creative nerve that makes each persons creation unique?

Martin Nebelong

49,689 views • 1 month ago

My photoshop is more fun than yours 😜 With a bit of help from Krea ai. It's a crazy feeling to see brushstrokes transformed like this in realtime.. And the feeling of control is magnitudes better than with text prompts. #ai #art

My photoshop is more fun than yours 😜 With a bit of help from Krea ai. It's a crazy feeling to see brushstrokes transformed like this in realtime.. And the feeling of control is magnitudes better than with text prompts. #ai #art

Martin Nebelong

213,920 views • 2 years ago

"Maybe the right UI for AI is a generative UI." Here's my new interview with Stephen Chau featuring a live demo of Cove's ground-breaking visual AI interface that breaks free from chatbots. We had a great chat about: - Real world use cases for visual AI interfaces - Generative UI and just-in-time AI apps - How Stephen scaled Uber Eats to $25B ARR Some quotes from Stephen Chau: "In computing, we moved from the command line to graphic user interfaces. Chatbots are like the command line era of AI. What's next is this..." "Imagine the entire layout of the workspace and all its contents is something made just in time, based on what you're trying to do..." "Our goal with Uber Eats was to build a business that's the same size as the Rides business. Here's what mattered..." 📌 Watch now:

"Maybe the right UI for AI is a generative UI." Here's my new interview with Stephen Chau featuring a live demo of Cove's ground-breaking visual AI interface that breaks free from chatbots. We had a great chat about: - Real world use cases for visual AI interfaces - Generative UI and just-in-time AI apps - How Stephen scaled Uber Eats to $25B ARR Some quotes from Stephen Chau: "In computing, we moved from the command line to graphic user interfaces. Chatbots are like the command line era of AI. What's next is this..." "Imagine the entire layout of the workspace and all its contents is something made just in time, based on what you're trying to do..." "Our goal with Uber Eats was to build a business that's the same size as the Rides business. Here's what mattered..." 📌 Watch now:

Peter Yang

74,721 views • 1 year ago

wrote a new song for the queen AI 👑🤖 𝖦𝗋𝗂𝗆𝖾𝗌 ⏳ "livin in a world full of robots what a time to be alive, all my friends like AI" sincerely, angelbaby3045.eth 🌐 WesGhost

wrote a new song for the queen AI 👑🤖 𝖦𝗋𝗂𝗆𝖾𝗌 ⏳ "livin in a world full of robots what a time to be alive, all my friends like AI" sincerely, angelbaby3045.eth 🌐 WesGhost

angelbaby3045.eth 🌐

91,123 views • 3 years ago

A lot of the work we did on Fairly OddParents: A New Wish was to capture the spirit of a 2D show in CG. Some of our secrets to getting that feel can now be yours! Like this: It all starts with the camera...

A lot of the work we did on Fairly OddParents: A New Wish was to capture the spirit of a 2D show in CG. Some of our secrets to getting that feel can now be yours! Like this: It all starts with the camera...

Trevor Young

1,333,805 views • 2 years ago

🕹️ The way of creating anything in real time with LCM-LoRA technology is so natural that it's here to stay. That's for sure! 💡 This is just a quick test I did using Krea to build an RPG dungeon. Imagine creating a video game in REAL TIME!

🕹️ The way of creating anything in real time with LCM-LoRA technology is so natural that it's here to stay. That's for sure! 💡 This is just a quick test I did using Krea to build an RPG dungeon. Imagine creating a video game in REAL TIME!

Javi Lopez ⛩️

164,254 views • 2 years ago

Marc Andreessen says AI is hyper democratizing. “Is the future of the world going to be one or a small number of companies, or for that matter, governments or super AIs that kind of own and control everything?” “Or is this a technology that's going to diffuse out and be in everybody's hands?” “This is the time in which AI is actually hyper democratizing. It has spread to more individuals, both in the country and around the world in the shortest period of time of any new technology I think in history.” “The best AIs in the world are in the consumer products.” “This technology is going to be in everybody's hands. And everybody is going to be able to use it to optimize the things that they do, have it be a thought partner, assistant for building companies.” Marc Andreessen 🇺🇸

Marc Andreessen says AI is hyper democratizing. “Is the future of the world going to be one or a small number of companies, or for that matter, governments or super AIs that kind of own and control everything?” “Or is this a technology that's going to diffuse out and be in everybody's hands?” “This is the time in which AI is actually hyper democratizing. It has spread to more individuals, both in the country and around the world in the shortest period of time of any new technology I think in history.” “The best AIs in the world are in the consumer products.” “This technology is going to be in everybody's hands. And everybody is going to be able to use it to optimize the things that they do, have it be a thought partner, assistant for building companies.” Marc Andreessen 🇺🇸

a16z

79,935 views • 10 months ago

The OFFICIAL preview with Prof. Brian Keating “I call these aliens of the gaps. It’s a form of almost religious worship. Same is happening with AI. The worship in AI is creating a god in our image. It’s one of the oldest stories of all time. Humans developed technology, and they said, ‘We don’t need to go on top of a mountain God made. We can build the tower ourselves.’”

The OFFICIAL preview with Prof. Brian Keating “I call these aliens of the gaps. It’s a form of almost religious worship. Same is happening with AI. The worship in AI is creating a god in our image. It’s one of the oldest stories of all time. Humans developed technology, and they said, ‘We don’t need to go on top of a mountain God made. We can build the tower ourselves.’”

Shawn Ryan

82,788 views • 2 months ago

Such a strange feeling to see brush-strokes, compositions and color transform like this with AI as I paint.. almost like seeing worlds and images flow from the tip of the Wacom pencil. Done using Krea ai as an overlay over my painting software as I paint. The final zoom is a combination of Krea and Magnific enhancements over the original sequence. #art #ai

Such a strange feeling to see brush-strokes, compositions and color transform like this with AI as I paint.. almost like seeing worlds and images flow from the tip of the Wacom pencil. Done using Krea ai as an overlay over my painting software as I paint. The final zoom is a combination of Krea and Magnific enhancements over the original sequence. #art #ai

Martin Nebelong

67,951 views • 2 years ago

Excited to show some surprising inventions on generative multiplayer games we made at Google with Stanford. We call the work MultiGen. I've always been inspired by early studios like id Software with Doom or Blizzard with Warcraft bringing networked video games to the next level. We are at the point in history where we can make strides like them, but for generative games. It's a strange feeling to be in the age of generative video games while still discovering how exactly to train the models and design the tools that make them useful. All of the tools that have been invented for classic game engines need to be redesigned for generative games. For example level and world design is not entirely possible with existing technology. We introduce editable memory to diffusion game engines that allow for design of new levels via a minimap. But we can easily imagine how this can be expanded with different creation tools. The end goal of this research direction is to allow game designers to be able to guide the generation process of their world, at the granularity that they prefer. Editable memory also allows us to add multiplayer to Generative Doom. We were amazed when we saw GameNGen some years ago, and now you can play it live with friends in real-time, on your couch or even online. Shared representations like our editable memory seem like the future for this type of experience. Models are, in some cases, expensive and approximate encoders but great interpolators and extrapolators. Leveraging their strengths lets you have completely new experiences that can be realized now and not in the distant future. This work was started at my previous team and continued in collaboration with Stanford. Congratulations to all for the discoveries.

Excited to show some surprising inventions on generative multiplayer games we made at Google with Stanford. We call the work MultiGen. I've always been inspired by early studios like id Software with Doom or Blizzard with Warcraft bringing networked video games to the next level. We are at the point in history where we can make strides like them, but for generative games. It's a strange feeling to be in the age of generative video games while still discovering how exactly to train the models and design the tools that make them useful. All of the tools that have been invented for classic game engines need to be redesigned for generative games. For example level and world design is not entirely possible with existing technology. We introduce editable memory to diffusion game engines that allow for design of new levels via a minimap. But we can easily imagine how this can be expanded with different creation tools. The end goal of this research direction is to allow game designers to be able to guide the generation process of their world, at the granularity that they prefer. Editable memory also allows us to add multiplayer to Generative Doom. We were amazed when we saw GameNGen some years ago, and now you can play it live with friends in real-time, on your couch or even online. Shared representations like our editable memory seem like the future for this type of experience. Models are, in some cases, expensive and approximate encoders but great interpolators and extrapolators. Leveraging their strengths lets you have completely new experiences that can be realized now and not in the distant future. This work was started at my previous team and continued in collaboration with Stanford. Congratulations to all for the discoveries.

Nataniel Ruiz

104,932 views • 6 months ago

$🪐 WORLD CREATORS Generative AI opened the door for everyone to tell their stories without the need for a crew or a big budget. We started slowly with images and saw an enormous interest in tools like Midjourney, DALL-E, or Leonardo. Now, a bit more than a year later, we find ourselves in front of new technologies that are transforming our way of life, how we create, and how we consume content. ChatGPT is rapidly replacing Googling, and all the generative AI tools are replacing all the stock photos and videos, with tools like Runway enabling storytelling for everyone. And we won't take over Hollywood. We will surpass it in ways we can't even imagine right now. We'll be able to create worlds and stories beyond our wildest imagination. Generative AI is enabling creative people to go 1000x on what was possible before. At the core of everything we'll still need human creativity, and we have to understand that AI tools without it will be nothing. Until then, enjoy a short video I've put together to illustrate a small fraction of the worlds we will create soon. What worlds and stories are you excited to create?$

🪐 WORLD CREATORS Generative AI opened the door for everyone to tell their stories without the need for a crew or a big budget. We started slowly with images and saw an enormous interest in tools like Midjourney, DALL-E, or Leonardo. Now, a bit more than a year later, we find ourselves in front of new technologies that are transforming our way of life, how we create, and how we consume content. ChatGPT is rapidly replacing Googling, and all the generative AI tools are replacing all the stock photos and videos, with tools like Runway enabling storytelling for everyone. And we won't take over Hollywood. We will surpass it in ways we can't even imagine right now. We'll be able to create worlds and stories beyond our wildest imagination. Generative AI is enabling creative people to go 1000x on what was possible before. At the core of everything we'll still need human creativity, and we have to understand that AI tools without it will be nothing. Until then, enjoy a short video I've put together to illustrate a small fraction of the worlds we will create soon. What worlds and stories are you excited to create?

Alex Patrascu

59,270 views • 2 years ago

I built a tool to imagine how AI can help us create better, more transparent research. Instead of only seeing the tiny number of analyses the author chooses to ossify in a static pdf, imagine being able to talk to the papers you read and ask for different analyses. That's what I built. This early prototype uses our vote-by-mail extension. You get to see the initial analyses we thought made sense, but you can ask the LLM to show you way more! e.g., "what would the results look like if we excluded Utah" or "what would the results look like if we excluded Washington and used linear trends" etc. With research agents like Claude code, researchers are going to be able to automatically search across millions of potential analyses. Our current paradigm is not equipped to transmit this complexity effectively or make sure we don't end up with all p-hacked results. My lab and I are thinking about how we can transform the way we do and disseminate research to adjust for this new world. This visualizer is the first of what will hopefully be a wide range of ideas. Big props to Janet Malzahn for suggesting this one. Check out the tool at the link below.

I built a tool to imagine how AI can help us create better, more transparent research. Instead of only seeing the tiny number of analyses the author chooses to ossify in a static pdf, imagine being able to talk to the papers you read and ask for different analyses. That's what I built. This early prototype uses our vote-by-mail extension. You get to see the initial analyses we thought made sense, but you can ask the LLM to show you way more! e.g., "what would the results look like if we excluded Utah" or "what would the results look like if we excluded Washington and used linear trends" etc. With research agents like Claude code, researchers are going to be able to automatically search across millions of potential analyses. Our current paradigm is not equipped to transmit this complexity effectively or make sure we don't end up with all p-hacked results. My lab and I are thinking about how we can transform the way we do and disseminate research to adjust for this new world. This visualizer is the first of what will hopefully be a wide range of ideas. Big props to Janet Malzahn for suggesting this one. Check out the tool at the link below.

Andy Hall

70,844 views • 7 months ago

Here's a quick test of a new tool called Krea. It's like the Real-time latent consistency model tests I've done with my photoshop and Dreams creations, but instead in a single application with faster refresh rates and higher quality output. I find that it strays too far from the drawing, and the editor is still quite limited but it's interesting to see nonetheless. It's not too difficult to imagine tech like this built right into Photoshop. And yes, I do know how to draw even if it doesn't show here 😅, I've been sculpting, painting and drawing for most of my life and I think it's exciting that tools like this will eventually help us break down some of the technical walls between our imagination and the visuals we create. And I'm not necessarily talking about drawing alone here because I don't see that going away as an art form, and I wouldn't want it to. But a hybrid of different art forms merging into a new, immersive art medium where anything is possible. Want to change the camera angle of your painting? Sure, go ahead. Want to make an animation of moving the camera around and seeing the light change in the scene as you do? In the example here, being able to move things around, change lighting and see it all reflected in the resulting image, is pretty amazing. Like a combination of the benefits of working in 3D, but with 2d tools. The moving version at the end of the video was made using Runways Gen-2. #art #ai

Here's a quick test of a new tool called Krea. It's like the Real-time latent consistency model tests I've done with my photoshop and Dreams creations, but instead in a single application with faster refresh rates and higher quality output. I find that it strays too far from the drawing, and the editor is still quite limited but it's interesting to see nonetheless. It's not too difficult to imagine tech like this built right into Photoshop. And yes, I do know how to draw even if it doesn't show here 😅, I've been sculpting, painting and drawing for most of my life and I think it's exciting that tools like this will eventually help us break down some of the technical walls between our imagination and the visuals we create. And I'm not necessarily talking about drawing alone here because I don't see that going away as an art form, and I wouldn't want it to. But a hybrid of different art forms merging into a new, immersive art medium where anything is possible. Want to change the camera angle of your painting? Sure, go ahead. Want to make an animation of moving the camera around and seeing the light change in the scene as you do? In the example here, being able to move things around, change lighting and see it all reflected in the resulting image, is pretty amazing. Like a combination of the benefits of working in 3D, but with 2d tools. The moving version at the end of the video was made using Runways Gen-2. #art #ai

Martin Nebelong

1,668,679 views • 2 years ago

How can AI enhance the creative process of a world-renowned industrial designer? 🎨 We partnered with the visionary Ross Lovegrove and MODEM to build a tool using Gemini and our image generation technology to translate his signature aesthetic into a new concept. 🪑

How can AI enhance the creative process of a world-renowned industrial designer? 🎨 We partnered with the visionary Ross Lovegrove and MODEM to build a tool using Gemini and our image generation technology to translate his signature aesthetic into a new concept. 🪑

Google DeepMind

232,108 views • 11 months ago

Systems like OpenAI's new Sora video model, just shows us that anyone can create a pretty video or a beautiful picture using nothing but a text prompt. But if we are to use the technology to create interesting art, thought-provoking cinema, and great games.. we need control of the output down to a granular level if that's what our project calls for. We need better tools and better interfaces to interact with the generative matter. Here's another example of what that could look like. Imagine this combined with a system like Sora. With complete temporal coherence and an ability to lock down the look of elements once you decide they look as they should. #ThePowerofDreams #Sora

Systems like OpenAI's new Sora video model, just shows us that anyone can create a pretty video or a beautiful picture using nothing but a text prompt. But if we are to use the technology to create interesting art, thought-provoking cinema, and great games.. we need control of the output down to a granular level if that's what our project calls for. We need better tools and better interfaces to interact with the generative matter. Here's another example of what that could look like. Imagine this combined with a system like Sora. With complete temporal coherence and an ability to lock down the look of elements once you decide they look as they should. #ThePowerofDreams #Sora

Martin Nebelong

52,888 views • 2 years ago

AI is destroying the imagination and creativity of the human mind. Works of inspired art like this cannot be created in a world dominated by ChatGPT and OpenAI. People will lose the ability to write, imagine, and create. All art will become regurgitated slop, created and manipulated by technocrats and their Indian programmers. AI is training the mind not to think.

AI is destroying the imagination and creativity of the human mind. Works of inspired art like this cannot be created in a world dominated by ChatGPT and OpenAI. People will lose the ability to write, imagine, and create. All art will become regurgitated slop, created and manipulated by technocrats and their Indian programmers. AI is training the mind not to think.

Lilly

27,324 views • 4 months ago