正在加载视频...

视频加载失败

返回首页

Googles Omni Flash model is such a great combination with Dreams 3d artwork. For most of the generations here, simple prompts like these were all it took: "Wrap the video in stark realism" or "Wrap the video in the realism of the provided image". Reposting with the correct edit...

MartinNebelong's profile picture

Martin Nebelong

33,619 subscribers

12,371 次观看 • 2 个月前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

A level of realism you could never achieve with traditional 3d. A level of control you could never get with text to video. This is the type of workflows I'm most excited about.. the type that allows us to paint, sculpt, act and direct worlds into existence.
2:03

A level of realism you could never achieve with traditional 3d. A level of control you could never get with text to video. This is the type of workflows I'm most excited about.. the type that allows us to paint, sculpt, act and direct worlds into existence.

Martin Nebelong

49,301 次观看 • 1 年前

I had to do this of course - test out the new Google Omni Flash model with my favorite 3d art sandbox - Dreams. Just a quick test for now.. I'll be sharing more examples and techniques soon but wanted to show you this one. There's something very fitting about combining "handmade" 3d models and animations sculpted from scratch in software that runs on 2013 hardware, with some of the most advanced AI technology in the world. As part of my work for Google, I've used Google V2V models on a few different projects now - Ancestra, Dear Upstairs Neighbors and most recently one of the IO films (I'll share more about that project soon) 🥰 - and video continues to be my favorite form of input for Generative Ai workflows. If an image says more than a thousand words, a video says more than a million. And knowing that these capabilities are now in the hands of all of you, makes me very excited to see the new workflows and projects that will come out of it. What do you think of Omni so far?
0:45

I had to do this of course - test out the new Google Omni Flash model with my favorite 3d art sandbox - Dreams. Just a quick test for now.. I'll be sharing more examples and techniques soon but wanted to show you this one. There's something very fitting about combining "handmade" 3d models and animations sculpted from scratch in software that runs on 2013 hardware, with some of the most advanced AI technology in the world. As part of my work for Google, I've used Google V2V models on a few different projects now - Ancestra, Dear Upstairs Neighbors and most recently one of the IO films (I'll share more about that project soon) 🥰 - and video continues to be my favorite form of input for Generative Ai workflows. If an image says more than a thousand words, a video says more than a million. And knowing that these capabilities are now in the hands of all of you, makes me very excited to see the new workflows and projects that will come out of it. What do you think of Omni so far?

Martin Nebelong

38,451 次观看 • 3 个月前

I've been a proponent of the idea of combining 3d and generative ai for a long time - as a way to get the best of both worlds: The precision and control of 3d - the detail, realism, lighting and speed of generative models. Here's another example of that idea, realized through a procedural system - entirely vibe-coded with Fable. Inspired by a photograph I took of an old tower in a town in southern France. In the first part of the video I'm using Krea AI's realtime model - later examples show how Nano Banana 2 can be used to take 3d stills and transform them to pretty much anything. What do you think of workflows like this compared to purely generating video or images from text, image or video input?
0:34

I've been a proponent of the idea of combining 3d and generative ai for a long time - as a way to get the best of both worlds: The precision and control of 3d - the detail, realism, lighting and speed of generative models. Here's another example of that idea, realized through a procedural system - entirely vibe-coded with Fable. Inspired by a photograph I took of an old tower in a town in southern France. In the first part of the video I'm using Krea AI's realtime model - later examples show how Nano Banana 2 can be used to take 3d stills and transform them to pretty much anything. What do you think of workflows like this compared to purely generating video or images from text, image or video input?

Martin Nebelong

20,265 次观看 • 1 个月前

Soon, generative AI will be a part of our 3d workflows.. and once that happens, we'll need 3d tools that are more well-suited for fast prototyping, more intuitive sculpting, and a departure from the overly complex and time-consuming workflows of most 3d tools today. Unbound is such a tool and here's a quick test I did today where I combined the output of it with Krea. Unbound is in early development and I recommend following the creators here: Unbound. #ai #art
0:38

Soon, generative AI will be a part of our 3d workflows.. and once that happens, we'll need 3d tools that are more well-suited for fast prototyping, more intuitive sculpting, and a departure from the overly complex and time-consuming workflows of most 3d tools today. Unbound is such a tool and here's a quick test I did today where I combined the output of it with Krea. Unbound is in early development and I recommend following the creators here: Unbound. #ai #art

Martin Nebelong

43,665 次观看 • 2 年前

I built these scenes from scratch on my PS5, and combined them with Google's new Omni Fast model. All of these were Input video + text prompts along the lines of: "Wrap this video in stark realism".
0:34

I built these scenes from scratch on my PS5, and combined them with Google's new Omni Fast model. All of these were Input video + text prompts along the lines of: "Wrap this video in stark realism".

Martin Nebelong

32,692 次观看 • 3 个月前

"Hah - generative ai can't even make an image of a hand with the right number of fingers.." "Stop pushing this slop" It's way past the point now where it must be clear to everyone, that generative ai is here to stay AND that the quality will continue to increase. I've been talking about this trajectory for years now, and I've been working towards finding ways to combine the strength of these models, with the best of what I love about "old-school" creation. Building with my hands, moving a pencil across the paper and seeing shapes emerge, moving a building slightly to the right to get just that composition I had in mind. Being fully immersed in a scene I'm building in VR, being inspired by the immersion to take the story in a new direction. For years I've been talking about how powerful the combination of 3d and generative ai is, be it traditional 3d, SDF volumes in Dreams or Gaussian splats - with experiments around using V2V as a "render pass" or with experiments around realtime ai. Enough talk you might think, where's the proof? It's all around us these days honestly and here's a small test I did during some OOO. Blender MPC + Fable - a pretty powerful combination! With a bit of Google Omni Fast on top as a "render" pass. What do you think of where this is heading? Hopeful, disheartened, inspired or the opposite? Can you imagine working with tools like this in a way where we still retain the human "spark" and the creative nerve that makes each persons creation unique?
0:25

"Hah - generative ai can't even make an image of a hand with the right number of fingers.." "Stop pushing this slop" It's way past the point now where it must be clear to everyone, that generative ai is here to stay AND that the quality will continue to increase. I've been talking about this trajectory for years now, and I've been working towards finding ways to combine the strength of these models, with the best of what I love about "old-school" creation. Building with my hands, moving a pencil across the paper and seeing shapes emerge, moving a building slightly to the right to get just that composition I had in mind. Being fully immersed in a scene I'm building in VR, being inspired by the immersion to take the story in a new direction. For years I've been talking about how powerful the combination of 3d and generative ai is, be it traditional 3d, SDF volumes in Dreams or Gaussian splats - with experiments around using V2V as a "render pass" or with experiments around realtime ai. Enough talk you might think, where's the proof? It's all around us these days honestly and here's a small test I did during some OOO. Blender MPC + Fable - a pretty powerful combination! With a bit of Google Omni Fast on top as a "render" pass. What do you think of where this is heading? Hopeful, disheartened, inspired or the opposite? Can you imagine working with tools like this in a way where we still retain the human "spark" and the creative nerve that makes each persons creation unique?

Martin Nebelong

49,395 次观看 • 1 个月前

Here's a look behind the scenes of how I used a 3d model, generated from an image, as a "driver" for ai animation. In the near future we'll see more 3d tools that support these types of workflows, with much more emphasis on enhancing user input instead of just letting the ai do everything. I used the 3d model to generate keyframes that I then animated in Luma Dream. By using a 3d model with a diffusion "layer" done with Krea, I got quite high quality frames with a relatively high consistency as well since the AI didn't have to "hallucinate" that much. I upscaled and detailed the frames using Magnific. Using semi-detailed 3d models as drivers for gen ai is very powerful, and it allows us to use models that are sculpted in a more free-flowing type of workflow that doesn't rely so much on high surface detailing or very time-consuming finish, but instead relies more on gestural 3d sculpting. #art #ai
0:24

Here's a look behind the scenes of how I used a 3d model, generated from an image, as a "driver" for ai animation. In the near future we'll see more 3d tools that support these types of workflows, with much more emphasis on enhancing user input instead of just letting the ai do everything. I used the 3d model to generate keyframes that I then animated in Luma Dream. By using a 3d model with a diffusion "layer" done with Krea, I got quite high quality frames with a relatively high consistency as well since the AI didn't have to "hallucinate" that much. I upscaled and detailed the frames using Magnific. Using semi-detailed 3d models as drivers for gen ai is very powerful, and it allows us to use models that are sculpted in a more free-flowing type of workflow that doesn't rely so much on high surface detailing or very time-consuming finish, but instead relies more on gestural 3d sculpting. #art #ai

Martin Nebelong

48,280 次观看 • 2 年前

NFINITY Studios presents: Spectre AI: “See It All. Make it yours.” $NFNT x $SPECT In partnership with SPECTRE AI, we’ve created one of our most groundbreaking videos yet. Spectre’s vision gave us the perfect opportunity to push the boundaries of our creative workflows. This video was brought to life using our new Workflow Automation system, a game-changer for video creation. With every project, our systems are getting sharper, faster, and closer to being released to the public.
1:30

NFINITY Studios presents: Spectre AI: “See It All. Make it yours.” $NFNT x $SPECT In partnership with SPECTRE AI, we’ve created one of our most groundbreaking videos yet. Spectre’s vision gave us the perfect opportunity to push the boundaries of our creative workflows. This video was brought to life using our new Workflow Automation system, a game-changer for video creation. With every project, our systems are getting sharper, faster, and closer to being released to the public.

NFINITY AI

26,786 次观看 • 1 年前

Google DeepMind CEO Demis Hassabis on "leaving AI in the lab for longer” (full question + answer in the video as I've seen him misquoted). Here's what he said: "For me, the best use case of AI was to improve human health and accelerate scientific discovery..." "Given how important AGI is and how transformative a technology is, maybe the most transformative one in human history, I thought it would be best to approach the sort of latter stages of building it, which we're in now, using the scientific method, very carefully, very precisely, very thoughtfully, and rigorously with all the best scientists, in my ideal world, collaborating on in CERN-like effort, on making sure each step we understood each step each as we got to the final goal of, of building AGI.... "While we're building AGI in this careful scientific way, humanity could benefit from the proceeds of that, like cures for cancer, or maybe new energy sources or new materials… “Looking at this from 20, 30 years ago when I started out on all of this, that would have been the ideal way for it to play out, in my opinion. “Now, it didn't happen like that because technology's unpredictable and in fact, it turns out that things like language were a lot easier than we were all expecting… “We were sort of playing around with that, so were the other leading labs, but of course with ChatGPT and fair play to OpenAI, they scaled it and then they put it out there. “And I think even they say it was kind of a research experiment. They didn't realize it would go so viral. And I think none of us did and we had sort of fairly equivalent systems at the time… “Now, the downside of it is, we're in this sort of ferocious commercial pressure race that everyone's sort of locked into currently. “And then on top of that, there's geopolitical issues like the US-China race and so on. So there's sort of multiple levels of pressure to sort of move fast. So the benefit of that, of course, you get faster progress, obviously. The progress is just at lightning speed these days. So that's good for all the good use cases. The second benefit is that everybody, all of the viewers out there, everyone, you're all getting to use the most cutting edge AI technology, perhaps only three to six months behind what is actually in the labs. So that's kind of mind blowing. “It's also great because I think it gives everyone a feeling for, it's democratizing AI. It's giving everyone a feeling for what it's like to interact with cutting edge AI and what it can do and what it can't do… “So I think there's positives and negatives about the way it's gone. It's not the way I dreamed about years ago where we would be sort of contemplating this philosophically and carefully considering each next step. We're not in that world. And I'm, although I'm a scientist first and foremost, I'm also a pragmatic engineer. So, we have to deal with the world as we find it and make the best of that. And we try to do that by advancing the frontier, but also trying to be as responsible as we can with doing that as we deploy these, you know, very powerful technologies, like Gemini and Alphafold.”
7:32

Google DeepMind CEO Demis Hassabis on "leaving AI in the lab for longer” (full question + answer in the video as I've seen him misquoted). Here's what he said: "For me, the best use case of AI was to improve human health and accelerate scientific discovery..." "Given how important AGI is and how transformative a technology is, maybe the most transformative one in human history, I thought it would be best to approach the sort of latter stages of building it, which we're in now, using the scientific method, very carefully, very precisely, very thoughtfully, and rigorously with all the best scientists, in my ideal world, collaborating on in CERN-like effort, on making sure each step we understood each step each as we got to the final goal of, of building AGI.... "While we're building AGI in this careful scientific way, humanity could benefit from the proceeds of that, like cures for cancer, or maybe new energy sources or new materials… “Looking at this from 20, 30 years ago when I started out on all of this, that would have been the ideal way for it to play out, in my opinion. “Now, it didn't happen like that because technology's unpredictable and in fact, it turns out that things like language were a lot easier than we were all expecting… “We were sort of playing around with that, so were the other leading labs, but of course with ChatGPT and fair play to OpenAI, they scaled it and then they put it out there. “And I think even they say it was kind of a research experiment. They didn't realize it would go so viral. And I think none of us did and we had sort of fairly equivalent systems at the time… “Now, the downside of it is, we're in this sort of ferocious commercial pressure race that everyone's sort of locked into currently. “And then on top of that, there's geopolitical issues like the US-China race and so on. So there's sort of multiple levels of pressure to sort of move fast. So the benefit of that, of course, you get faster progress, obviously. The progress is just at lightning speed these days. So that's good for all the good use cases. The second benefit is that everybody, all of the viewers out there, everyone, you're all getting to use the most cutting edge AI technology, perhaps only three to six months behind what is actually in the labs. So that's kind of mind blowing. “It's also great because I think it gives everyone a feeling for, it's democratizing AI. It's giving everyone a feeling for what it's like to interact with cutting edge AI and what it can do and what it can't do… “So I think there's positives and negatives about the way it's gone. It's not the way I dreamed about years ago where we would be sort of contemplating this philosophically and carefully considering each next step. We're not in that world. And I'm, although I'm a scientist first and foremost, I'm also a pragmatic engineer. So, we have to deal with the world as we find it and make the best of that. And we try to do that by advancing the frontier, but also trying to be as responsible as we can with doing that as we deploy these, you know, very powerful technologies, like Gemini and Alphafold.”

Cleo Abram

64,840 次观看 • 4 个月前

I'm playing around with generative AI tools and stitching them together into visual stories. Here I took the first few sentences of Pride and Prejudice and made it into a video. The gen stack used for this one: - Anthropic Claude took the first chapter, generated the scenes and the individual prompts to to the image generator. - Ideogram took the prompts and generate the images - Luma took the images and animated them - for narration - VEED | AI Video Creation to stitch it together (Many of these choices are just what I happened to use for this one while exploring a bunch of things). Anyway honestly it was pretty messy and there is a ton of copy pasting between all of the tools, and even this little video with 3 scenes took me about an hour. There is a huge storytelling opportunity here for whoever can make this convenient. Who is building the first 100% AI-native movie maker?
0:28

I'm playing around with generative AI tools and stitching them together into visual stories. Here I took the first few sentences of Pride and Prejudice and made it into a video. The gen stack used for this one: - Anthropic Claude took the first chapter, generated the scenes and the individual prompts to to the image generator. - Ideogram took the prompts and generate the images - Luma took the images and animated them - for narration - VEED | AI Video Creation to stitch it together (Many of these choices are just what I happened to use for this one while exploring a bunch of things). Anyway honestly it was pretty messy and there is a ton of copy pasting between all of the tools, and even this little video with 3 scenes took me about an hour. There is a huge storytelling opportunity here for whoever can make this convenient. Who is building the first 100% AI-native movie maker?

Andrej Karpathy

609,834 次观看 • 2 年前

I made this video in a few hours last week. Almost all of it - script, voice, image, and character animation - is AI. The product demos were filmed by me. I can't see how the future of video production doesn't include AI in most workflows. So easy, cheap, and fast!
1:53

I made this video in a few hours last week. Almost all of it - script, voice, image, and character animation - is AI. The product demos were filmed by me. I can't see how the future of video production doesn't include AI in most workflows. So easy, cheap, and fast!

Justine Moore

92,601 次观看 • 7 个月前

i’ve been fortunate enough to be mentored by some of the most experienced designers in tech early in my career. (thanks to the lads at LI.FI & ParagonsDAO) these learnings never made me compromise on the creative trust that i owed to every founder and team i worked with. for the last 3 years, every project came through referrals from clients and friends (ilu), so i never really felt the need to launch. but in a post ai era, i believe that peer-to-peer learning would be one of the most important aspects of growing as a designer or even as a studio. we've spent the last year experimenting obsessively with ai tools, workflows, storytelling systems, and new ways of creating. so it feels like the right time to share learnings and build in public. launching dapptales publicly - a creative studio specialising in tasteful storytelling for tech companies. and here's a little video about what we're best at :))
1:18

i’ve been fortunate enough to be mentored by some of the most experienced designers in tech early in my career. (thanks to the lads at LI.FI & ParagonsDAO) these learnings never made me compromise on the creative trust that i owed to every founder and team i worked with. for the last 3 years, every project came through referrals from clients and friends (ilu), so i never really felt the need to launch. but in a post ai era, i believe that peer-to-peer learning would be one of the most important aspects of growing as a designer or even as a studio. we've spent the last year experimenting obsessively with ai tools, workflows, storytelling systems, and new ways of creating. so it feels like the right time to share learnings and build in public. launching dapptales publicly - a creative studio specialising in tasteful storytelling for tech companies. and here's a little video about what we're best at :))

aamir

10,704 次观看 • 2 个月前

Here's a quick test of a new tool called Krea. It's like the Real-time latent consistency model tests I've done with my photoshop and Dreams creations, but instead in a single application with faster refresh rates and higher quality output. I find that it strays too far from the drawing, and the editor is still quite limited but it's interesting to see nonetheless. It's not too difficult to imagine tech like this built right into Photoshop. And yes, I do know how to draw even if it doesn't show here 😅, I've been sculpting, painting and drawing for most of my life and I think it's exciting that tools like this will eventually help us break down some of the technical walls between our imagination and the visuals we create. And I'm not necessarily talking about drawing alone here because I don't see that going away as an art form, and I wouldn't want it to. But a hybrid of different art forms merging into a new, immersive art medium where anything is possible. Want to change the camera angle of your painting? Sure, go ahead. Want to make an animation of moving the camera around and seeing the light change in the scene as you do? In the example here, being able to move things around, change lighting and see it all reflected in the resulting image, is pretty amazing. Like a combination of the benefits of working in 3D, but with 2d tools. The moving version at the end of the video was made using Runways Gen-2. #art #ai
0:30

Here's a quick test of a new tool called Krea. It's like the Real-time latent consistency model tests I've done with my photoshop and Dreams creations, but instead in a single application with faster refresh rates and higher quality output. I find that it strays too far from the drawing, and the editor is still quite limited but it's interesting to see nonetheless. It's not too difficult to imagine tech like this built right into Photoshop. And yes, I do know how to draw even if it doesn't show here 😅, I've been sculpting, painting and drawing for most of my life and I think it's exciting that tools like this will eventually help us break down some of the technical walls between our imagination and the visuals we create. And I'm not necessarily talking about drawing alone here because I don't see that going away as an art form, and I wouldn't want it to. But a hybrid of different art forms merging into a new, immersive art medium where anything is possible. Want to change the camera angle of your painting? Sure, go ahead. Want to make an animation of moving the camera around and seeing the light change in the scene as you do? In the example here, being able to move things around, change lighting and see it all reflected in the resulting image, is pretty amazing. Like a combination of the benefits of working in 3D, but with 2d tools. The moving version at the end of the video was made using Runways Gen-2. #art #ai

Martin Nebelong

1,668,679 次观看 • 2 年前

This is real-time.. The debate about Nvidia's latest neural rendering tech is ongoing. One side sees the potential of the technology to bridge the gap across the uncanny valley, to give us a much less rigid creative process than a to b to c.. the other side shouts "slop" and tries to shame any creative who dares to explore this new technology, any chance they get. Meanwhile, I'm over here saying "I told you so".. Gen Ai is here to stay, its becoming real-time and Nvidia now has the "RTX on" for AI which I've been talking about for years. We need creatives to help shape meaningful tools that combine this tech with much more granular creative control.. with tools where you as an artist see your creative thumbprint from beginning to end in the parts of the work that matters the most to you. If you're shaming artists or storytellers who embrace these tools, you're doing them a disservice in my opinion. If they don't learn to utilize these tools and workflows, to make them their own and to shape them, others will. There's no way this tech won't be a huge part of the creative industry and of the way we tell stories. It's happening around us as we speak. That said, at the core of it, do what makes you happy and create in any way you like. Don't let people tell you that "you can't do this" or "you have to do it like this".
0:36

This is real-time.. The debate about Nvidia's latest neural rendering tech is ongoing. One side sees the potential of the technology to bridge the gap across the uncanny valley, to give us a much less rigid creative process than a to b to c.. the other side shouts "slop" and tries to shame any creative who dares to explore this new technology, any chance they get. Meanwhile, I'm over here saying "I told you so".. Gen Ai is here to stay, its becoming real-time and Nvidia now has the "RTX on" for AI which I've been talking about for years. We need creatives to help shape meaningful tools that combine this tech with much more granular creative control.. with tools where you as an artist see your creative thumbprint from beginning to end in the parts of the work that matters the most to you. If you're shaming artists or storytellers who embrace these tools, you're doing them a disservice in my opinion. If they don't learn to utilize these tools and workflows, to make them their own and to shape them, others will. There's no way this tech won't be a huge part of the creative industry and of the way we tell stories. It's happening around us as we speak. That said, at the core of it, do what makes you happy and create in any way you like. Don't let people tell you that "you can't do this" or "you have to do it like this".

Martin Nebelong

43,165 次观看 • 5 个月前

In just 3 minutes, Ken Griffin (Citadel) & Larry Fink (BlackRock) explain the current state of government overspending, AI hype, & 2026 $600B data center CapEx "The world needs a savior, & the hope is that AI is the savior that we need for productivity." "The the area of recklessness is the spending of governments around the world, who are all, with little exception, all spending well beyond their means." Big issue: "Will AI create the productivity acceleration that is honestly just hoped for in Washington & in the halls of government around the world as a ways to overcome the profligate spending that we're currently engaged in?" Citadel BlackRock World Economic Forum . . . Ken Griffin "Let's take a step back and and talk about where we are right here right now. The the area of recklessness is the spending of governments around the world, who are all, with little exception, all spending well beyond their means. That's the recklessness of this moment in history. This is not a parallel to the 1920s in terms of the recklessness of the of the private capital markets. It's a story of the recklessness of government spending. Within the private sector there's a huge question as to where AI will take us. And I, I was carefully taking notes and listening to what Larry has to say, or to what Madame Lagarde has to say, because this is one of the big issues of our moment. Will AI create the productivity acceleration that is honestly just hoped for in Washington and in the halls of government around the world as a ways to overcome the profligate spending that we're currently engaged in? The world, the world needs a savior, and the hope is that AI is the savior that we need for productivity. And the challenge with this is it is, it may or may not be. We just don't know yet. Now there's a tremendous amount of hype around AI, and in some sense the large AI companies need to create that hype to raise the tens — or actually 100, hundreds of billions, right — of billions of dollars of investment that are going into the field. Like you wouldn't be able to raise hundreds of billions of dollars. We'll spend — and Larry can probably correct me on this — but roughly $600 billion this year in cap ex for data centers in the United States. Larry Fink "I think it could be larger." Andrew Ross Sorkin "But does that mean that it's getting hyped up too much, or it's just the the hype is required as a sales mechanism?" Larry Fink "First of all, so much of the data centers are being built for cloud, right? And and the big issue is gonna be in terms of monetization of of of the spend. The data centers are being built for AI requires more advanced chips. The question is what is the lifetime of that chip. If we have new technological changes in the lifetime of the chip in one year, then that spend is gonna be really a bad spend. If the lifetime as they expect it to be is 4 or 5 years and then those chips can be used for cloud, then then I think these investments are gonna prove to be good investments. So I think it's it's gonna be — you know, if the speed of technology changes and all these investments now, they're gonna be — it's gonna be a challenge. But I agree with Ken. I think we don't know enough, but I'm personally very optimistic on how AI is gonna affect the world economy."
3:10

In just 3 minutes, Ken Griffin (Citadel) & Larry Fink (BlackRock) explain the current state of government overspending, AI hype, & 2026 $600B data center CapEx "The world needs a savior, & the hope is that AI is the savior that we need for productivity." "The the area of recklessness is the spending of governments around the world, who are all, with little exception, all spending well beyond their means." Big issue: "Will AI create the productivity acceleration that is honestly just hoped for in Washington & in the halls of government around the world as a ways to overcome the profligate spending that we're currently engaged in?" Citadel BlackRock World Economic Forum . . . Ken Griffin "Let's take a step back and and talk about where we are right here right now. The the area of recklessness is the spending of governments around the world, who are all, with little exception, all spending well beyond their means. That's the recklessness of this moment in history. This is not a parallel to the 1920s in terms of the recklessness of the of the private capital markets. It's a story of the recklessness of government spending. Within the private sector there's a huge question as to where AI will take us. And I, I was carefully taking notes and listening to what Larry has to say, or to what Madame Lagarde has to say, because this is one of the big issues of our moment. Will AI create the productivity acceleration that is honestly just hoped for in Washington and in the halls of government around the world as a ways to overcome the profligate spending that we're currently engaged in? The world, the world needs a savior, and the hope is that AI is the savior that we need for productivity. And the challenge with this is it is, it may or may not be. We just don't know yet. Now there's a tremendous amount of hype around AI, and in some sense the large AI companies need to create that hype to raise the tens — or actually 100, hundreds of billions, right — of billions of dollars of investment that are going into the field. Like you wouldn't be able to raise hundreds of billions of dollars. We'll spend — and Larry can probably correct me on this — but roughly $600 billion this year in cap ex for data centers in the United States. Larry Fink "I think it could be larger." Andrew Ross Sorkin "But does that mean that it's getting hyped up too much, or it's just the the hype is required as a sales mechanism?" Larry Fink "First of all, so much of the data centers are being built for cloud, right? And and the big issue is gonna be in terms of monetization of of of the spend. The data centers are being built for AI requires more advanced chips. The question is what is the lifetime of that chip. If we have new technological changes in the lifetime of the chip in one year, then that spend is gonna be really a bad spend. If the lifetime as they expect it to be is 4 or 5 years and then those chips can be used for cloud, then then I think these investments are gonna prove to be good investments. So I think it's it's gonna be — you know, if the speed of technology changes and all these investments now, they're gonna be — it's gonna be a challenge. But I agree with Ken. I think we don't know enough, but I'm personally very optimistic on how AI is gonna affect the world economy."

Molly O’Shea

33,210 次观看 • 7 个月前

Excited to share that I'm joining fal as a Creative Technologist! I'll be building workflows with state-of-the-art generative image, video, and audio models, and producing content for the fal YouTube channel.
0:29

Excited to share that I'm joining fal as a Creative Technologist! I'll be building workflows with state-of-the-art generative image, video, and audio models, and producing content for the fal YouTube channel.

Matt Workman

46,244 次观看 • 4 个月前

Excited to show some surprising inventions on generative multiplayer games we made at Google with Stanford. We call the work MultiGen. I've always been inspired by early studios like id Software with Doom or Blizzard with Warcraft bringing networked video games to the next level. We are at the point in history where we can make strides like them, but for generative games. It's a strange feeling to be in the age of generative video games while still discovering how exactly to train the models and design the tools that make them useful. All of the tools that have been invented for classic game engines need to be redesigned for generative games. For example level and world design is not entirely possible with existing technology. We introduce editable memory to diffusion game engines that allow for design of new levels via a minimap. But we can easily imagine how this can be expanded with different creation tools. The end goal of this research direction is to allow game designers to be able to guide the generation process of their world, at the granularity that they prefer. Editable memory also allows us to add multiplayer to Generative Doom. We were amazed when we saw GameNGen some years ago, and now you can play it live with friends in real-time, on your couch or even online. Shared representations like our editable memory seem like the future for this type of experience. Models are, in some cases, expensive and approximate encoders but great interpolators and extrapolators. Leveraging their strengths lets you have completely new experiences that can be realized now and not in the distant future. This work was started at my previous team and continued in collaboration with Stanford. Congratulations to all for the discoveries.
0:40

Excited to show some surprising inventions on generative multiplayer games we made at Google with Stanford. We call the work MultiGen. I've always been inspired by early studios like id Software with Doom or Blizzard with Warcraft bringing networked video games to the next level. We are at the point in history where we can make strides like them, but for generative games. It's a strange feeling to be in the age of generative video games while still discovering how exactly to train the models and design the tools that make them useful. All of the tools that have been invented for classic game engines need to be redesigned for generative games. For example level and world design is not entirely possible with existing technology. We introduce editable memory to diffusion game engines that allow for design of new levels via a minimap. But we can easily imagine how this can be expanded with different creation tools. The end goal of this research direction is to allow game designers to be able to guide the generation process of their world, at the granularity that they prefer. Editable memory also allows us to add multiplayer to Generative Doom. We were amazed when we saw GameNGen some years ago, and now you can play it live with friends in real-time, on your couch or even online. Shared representations like our editable memory seem like the future for this type of experience. Models are, in some cases, expensive and approximate encoders but great interpolators and extrapolators. Leveraging their strengths lets you have completely new experiences that can be realized now and not in the distant future. This work was started at my previous team and continued in collaboration with Stanford. Congratulations to all for the discoveries.

Nataniel Ruiz

104,841 次观看 • 5 个月前

This is probably the most complex workflow I’ve ever built, only with open-source tools. It took my 4 days. It takes four inputs: author, title, and style; and generates a full visual animated story in one click in ComfyUI . I worked on it for four days. There are still some bugs, but here’s the first preview. Here’s a quick breakdown: - The four inputs are sent to LLMs with precise instructions to generate: first, prompts for images and image modifications; second, prompts for animations; third, prompts for generating music. - All voices are generated from the text and timed precisely, as they determine the length of each animation segment. - The first image and video are generated to serve as the title, but also as the guide for all other images created for the video. - Titles and subtitles are also added automatically in Comfy. - I also developed a lot of custom nodes for minor frame calculations, mostly to match audio and video. - The full system is a large loop that, for each line of text, generates an image and then a video from that image. The loop was the hardest part to build in this workflow, so it can process either a 20-second video or a 2-minute video with the same input. - There are multiple combinations of LLMs that try to understand the text in the best way to provide the best prompts for images and video. - The final video is assembled entirely within ComfyUI. - The music is generated based on the LLM output and matches the exact timing of the full animation. - Done! For reference, this workflow uses a lot of models and only works on an RTX 6000 Pro with plenty of RAM. My goal is not to replace humans, as I’ll try to explain later, this workflow is highly controlled and can be adapted or reworked at any point by real artists! My aim was to create a tool that can animate text in one go, allowing the AI some freedom while keeping a strict flow. I don’t know yet how I’ll share this workflow with people, I still need to polish it properly, but maybe through Patreon. Anyway, I hope you enjoy my research, and let’s always keep pushing further! :)
0:55

This is probably the most complex workflow I’ve ever built, only with open-source tools. It took my 4 days. It takes four inputs: author, title, and style; and generates a full visual animated story in one click in ComfyUI . I worked on it for four days. There are still some bugs, but here’s the first preview. Here’s a quick breakdown: - The four inputs are sent to LLMs with precise instructions to generate: first, prompts for images and image modifications; second, prompts for animations; third, prompts for generating music. - All voices are generated from the text and timed precisely, as they determine the length of each animation segment. - The first image and video are generated to serve as the title, but also as the guide for all other images created for the video. - Titles and subtitles are also added automatically in Comfy. - I also developed a lot of custom nodes for minor frame calculations, mostly to match audio and video. - The full system is a large loop that, for each line of text, generates an image and then a video from that image. The loop was the hardest part to build in this workflow, so it can process either a 20-second video or a 2-minute video with the same input. - There are multiple combinations of LLMs that try to understand the text in the best way to provide the best prompts for images and video. - The final video is assembled entirely within ComfyUI. - The music is generated based on the LLM output and matches the exact timing of the full animation. - Done! For reference, this workflow uses a lot of models and only works on an RTX 6000 Pro with plenty of RAM. My goal is not to replace humans, as I’ll try to explain later, this workflow is highly controlled and can be adapted or reworked at any point by real artists! My aim was to create a tool that can animate text in one go, allowing the AI some freedom while keeping a strict flow. I don’t know yet how I’ll share this workflow with people, I still need to polish it properly, but maybe through Patreon. Anyway, I hope you enjoy my research, and let’s always keep pushing further! :)

Lovis Odin

58,841 次观看 • 11 个月前