Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Google dropped a new AI paper called LUMIERE. It's remarkably flexible, supporting video inpainting, image-to-video, AND stylized video generation tasks. Say hello to “space-time diffusion” for video generation! Now what the heck does that mean exactly?! 🌐⏳ → TL;DR it utilizes a “Space-Time UNet” architecture that generates the full... duration of the video in one pass, rather than generating distant keyframes and interpolating between them like prior works. Because the computation is done in this “compressed space-time representation” to generate the full clip at once, it's far more temporally consistent. → Another benefit of generating the full video at once is that you can “direct” the video generation, making it easier to hand off to other models/tasks without having to stitch together partial solutions. You can condition generations on additional inputs, meaning you get the full stack of AI video capabilities – from video inpainting to image-to-video and beyond. → New SOTA for AI video generation? User study results in the paper suggest human evaluators preferred Lumiere over Runway Gen-2, Pika Labs, and Stable Video Diffusion in terms of quality, text alignment AND motion. But as always, we need to get hands-on with this tech when Google actually decides to ship it. → Could this end up inside YouTube? Y’all know i’m obsessed with blending reality and imagination – so it’s the video inpainting tech I'm most excited about. I really hope this model finds its way into YouTube's Generative AI efforts, and based on their prior announcements and the list of acknowledgments in the paper I think it might! 🤞🏽 Links: 🔗Paper: 🔗Project:show more

Bilawal Sidhu

106,927 subscribers

44,822 Aufrufe • vor 2 Jahren •via X (Twitter)

Wissenschaft & Technologie Nachrichten & Politik

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Do you use Runway Gen-2 to make AI videos? This is for you! I built a simple tool to help "extend" Runway videos beyond the 4 second clip limit. Just choose a video, and the tool will give you the final frame -- so that you can drop this back into Runway, and generate another AI video based on it. Let me know if this is useful to you! Link is at bottom of the video and it's live to try right now.

Do you use Runway Gen-2 to make AI videos? This is for you! I built a simple tool to help "extend" Runway videos beyond the 4 second clip limit. Just choose a video, and the tool will give you the final frame -- so that you can drop this back into Runway, and generate another AI video based on it. Let me know if this is useful to you! Link is at bottom of the video and it's live to try right now.

Benjamin De Kraker

68,394 Aufrufe • vor 3 Jahren

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Stability AI

1,024,795 Aufrufe • vor 2 Jahren

From product image to video with just one tool - Dzine As you may have noticed, this is one of my favorite tools. It is also very underrated, as probably 50% of my tutorials include some workflow. I was testing the new image-to-video option today, and I love it. Step - by step guide in comments 🔽 I can do 95% of a workflow without switching between apps. Image generation, Image to image with style reference, background removal, background generation, and 2 frames image to video. The only other app I have been using for this video is CapCut so that I can stitch it together. Step by step 🔽

From product image to video with just one tool - Dzine As you may have noticed, this is one of my favorite tools. It is also very underrated, as probably 50% of my tutorials include some workflow. I was testing the new image-to-video option today, and I love it. Step - by step guide in comments 🔽 I can do 95% of a workflow without switching between apps. Image generation, Image to image with style reference, background removal, background generation, and 2 frames image to video. The only other app I have been using for this video is CapCut so that I can stitch it together. Step by step 🔽

Teodora P L

28,523 Aufrufe • vor 2 Jahren

As always everyone is blind staring at the progress of LLMs for coding and chat But meanwhile the new SOTA video model Seedance 2.5 has been slowly rolling out and it's really quite exceptional It's made by ByteDance (TikTok) who of course have lots of training data With just a few reference pics, it can get quite close to how you look IRL and you can do quite professional video shots with just a prompt I'd say it's the first video model that's now at the level of image models with the level of character likeness, cracking that in image models also took about 3 years (2022-2025) Generating 15 seconds takes about 4 minutes I put it live now on Photo AI, you can use it under [ Make video ] from the sidebar with just a prompt and your model selected! So you don't need to take an AI photo first and then turn that into a video! Saves lots of time :D It's more expensive than but I kept the credits the same (30 for 1 video) It also works inside the new video editor and you can make changes in your video with [ Magic edit ] in both the main app and the video editor Also a message for my server guy Daniel Lockyer (it can do voice too and you can even submit a voice sample of yourself, but I didn't here)

As always everyone is blind staring at the progress of LLMs for coding and chat But meanwhile the new SOTA video model Seedance 2.5 has been slowly rolling out and it's really quite exceptional It's made by ByteDance (TikTok) who of course have lots of training data With just a few reference pics, it can get quite close to how you look IRL and you can do quite professional video shots with just a prompt I'd say it's the first video model that's now at the level of image models with the level of character likeness, cracking that in image models also took about 3 years (2022-2025) Generating 15 seconds takes about 4 minutes I put it live now on Photo AI, you can use it under [ Make video ] from the sidebar with just a prompt and your model selected! So you don't need to take an AI photo first and then turn that into a video! Saves lots of time :D It's more expensive than but I kept the credits the same (30 for 1 video) It also works inside the new video editor and you can make changes in your video with [ Magic edit ] in both the main app and the video editor Also a message for my server guy Daniel Lockyer (it can do voice too and you can even submit a voice sample of yourself, but I didn't here)

@levelsio

711,938 Aufrufe • vor 2 Monaten

An hour of live AI video generation now costs $10.80 At $0.003 a second, Vivix W1 generates a full minute for just 18 cents. That price matters because of what W1 is. It's a real-time interactive video model. Through the API, it keeps generating video while the experience is happening, and you can shape what comes next. With a normal clip generator, you pay once and you're done. Real-time interactive video keeps generating as you use it. Every second generated is another second you pay for. At higher per-second rates, that bill can grow quickly. It's hard to ship an interactive product when every user session costs a fortune. With W1, the math starts to look workable. And they didn't buy that price by cutting quality or speed. In Vivix’s supplied tests, eight-second videos averaged 7.7 seconds to generate from text and 8.4 seconds with one reference image. In the side-by-side comparisons, W1 holds its own against Seedance 2.5, Kling 3.0, and H3 Max on the same prompts. An exploding taxi, a skyline shot, a robot serving drinks on a plane. It's not perfect yet: • Cinematic fidelity still has gaps • Complex motion and dense scenes still need work • Full-scale training is still underway, so this is an early checkpoint How to try it: • Real-time generation is live through the API today • The browser playground does offline generation for now. Submit a prompt and you get a finished clip. Real-time browser tools are coming soon My take: I've seen a lot of interactive video demos that would cost a fortune to run in production. This looks priced for something you could actually ship. Once a minute of live video generation costs 18 cents, interactive stories, games, and characters could become an everyday medium instead of a tech demo. What I want to see next from Vivix.AI: better complex motion. Improve that and the use cases open up fast. Put your project’s runtime into the equation 👇

An hour of live AI video generation now costs $10.80 At $0.003 a second, Vivix W1 generates a full minute for just 18 cents. That price matters because of what W1 is. It's a real-time interactive video model. Through the API, it keeps generating video while the experience is happening, and you can shape what comes next. With a normal clip generator, you pay once and you're done. Real-time interactive video keeps generating as you use it. Every second generated is another second you pay for. At higher per-second rates, that bill can grow quickly. It's hard to ship an interactive product when every user session costs a fortune. With W1, the math starts to look workable. And they didn't buy that price by cutting quality or speed. In Vivix’s supplied tests, eight-second videos averaged 7.7 seconds to generate from text and 8.4 seconds with one reference image. In the side-by-side comparisons, W1 holds its own against Seedance 2.5, Kling 3.0, and H3 Max on the same prompts. An exploding taxi, a skyline shot, a robot serving drinks on a plane. It's not perfect yet: • Cinematic fidelity still has gaps • Complex motion and dense scenes still need work • Full-scale training is still underway, so this is an early checkpoint How to try it: • Real-time generation is live through the API today • The browser playground does offline generation for now. Submit a prompt and you get a finished clip. Real-time browser tools are coming soon My take: I've seen a lot of interactive video demos that would cost a fortune to run in production. This looks priced for something you could actually ship. Once a minute of live video generation costs 18 cents, interactive stories, games, and characters could become an everyday medium instead of a tech demo. What I want to see next from Vivix.AI: better complex motion. Improve that and the use cases open up fast. Put your project’s runtime into the equation 👇

Raja Chichulkar

18,899 Aufrufe • vor 3 Tagen

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page: Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page: Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.

AK

375,160 Aufrufe • vor 3 Jahren

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,849 Aufrufe • vor 1 Jahr

✨ Grok Imagine Video is now live on Photo AI It's hard to explain how impressive this is because of the speed that xAI got itself from literally nothing to the top of the leaderboards Six months ago Grok's video model was a joke, it wasn't even close to any of the video models out there, it looked cartoony and wasn't there and nobody took it seriously Now it's here and it's instantly the #1 video model out there now, it shot above Kling (which I used before on Photo AI and usually my favorite) and above Runway Gen 4.5 which was just launched 6 days ago! Mmore importantly it's now above xAI's biggest competitors' models: Google's Veo 3 and OpenAI's Sora 2 Being the best video model doesn't mean it's flawless: video is incredibly hard and actually because it looks so realistic now when it does make a mistakes it's even funnier One thing I noticed is that it still has a hard time with is voice, it does it well for a majority of the video but then slips up and produces unintelligible blabbering (which is really funny to hear) in both English (video 1: "it's where I find my naim", what's a "naim"?), and tested it in Portuguese too (video 3 at the end is unintelligible Portuguese I believe) In many ways Grok Imagine Video also reminds me of Sora, it has that weird but funny Sora conversation style But guys it's REALLY really really really close to getting perfect, we're so close to having full video productions being to be able done in AI, actually you already can if you just cut out the bad parts already Very exciting and I'm grateful I can experience this

✨ Grok Imagine Video is now live on Photo AI It's hard to explain how impressive this is because of the speed that xAI got itself from literally nothing to the top of the leaderboards Six months ago Grok's video model was a joke, it wasn't even close to any of the video models out there, it looked cartoony and wasn't there and nobody took it seriously Now it's here and it's instantly the #1 video model out there now, it shot above Kling (which I used before on Photo AI and usually my favorite) and above Runway Gen 4.5 which was just launched 6 days ago! Mmore importantly it's now above xAI's biggest competitors' models: Google's Veo 3 and OpenAI's Sora 2 Being the best video model doesn't mean it's flawless: video is incredibly hard and actually because it looks so realistic now when it does make a mistakes it's even funnier One thing I noticed is that it still has a hard time with is voice, it does it well for a majority of the video but then slips up and produces unintelligible blabbering (which is really funny to hear) in both English (video 1: "it's where I find my naim", what's a "naim"?), and tested it in Portuguese too (video 3 at the end is unintelligible Portuguese I believe) In many ways Grok Imagine Video also reminds me of Sora, it has that weird but funny Sora conversation style But guys it's REALLY really really really close to getting perfect, we're so close to having full video productions being to be able done in AI, actually you already can if you just cut out the bad parts already Very exciting and I'm grateful I can experience this

@levelsio

195,023 Aufrufe • vor 8 Monaten

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.

AK

25,465 Aufrufe • vor 2 Jahren

Today, we are adding Stable Video Diffusion, our foundation model for generative video to the Stability AI Developer Platform API. The model can generate 2 seconds of video, comprising of 25 generated frames and 24 frames of FILM interpolation, within an average time of 41 seconds. Developers interested in utilizing Stable Video Diffusion through an API can access it now on the Stability AI Developer Platform. Learn more here:

Today, we are adding Stable Video Diffusion, our foundation model for generative video to the Stability AI Developer Platform API. The model can generate 2 seconds of video, comprising of 25 generated frames and 24 frames of FILM interpolation, within an average time of 41 seconds. Developers interested in utilizing Stable Video Diffusion through an API can access it now on the Stability AI Developer Platform. Learn more here:

Stability AI

176,217 Aufrufe • vor 2 Jahren

seedance 2.0 + my v2 AI UGC prompting system is giving insane results i spent the last 24 hours generating over 200 seedance 2.0 videos to figure out the best prompting framework system for AI UGC this video was made with 1 prompt and 1 tool, no editing was done to the video this was just a prompt to a video this is by far the best model i've ever used and the craziest part is that it can be fully automated this is the first time we can actually automate high quality ai ugc at this level bytedance owns tiktok so this model is trained on millions of high quality ugc videos. you just need to know how to extract that and call it in your prompt. we are so early... it's insane

seedance 2.0 + my v2 AI UGC prompting system is giving insane results i spent the last 24 hours generating over 200 seedance 2.0 videos to figure out the best prompting framework system for AI UGC this video was made with 1 prompt and 1 tool, no editing was done to the video this was just a prompt to a video this is by far the best model i've ever used and the craziest part is that it can be fully automated this is the first time we can actually automate high quality ai ugc at this level bytedance owns tiktok so this model is trained on millions of high quality ugc videos. you just need to know how to extract that and call it in your prompt. we are so early... it's insane

Miko

81,676 Aufrufe • vor 8 Monaten

So remember that time machine I told you about? I managed to sneek out another video from my visit to the future.. this time I captured video of a 3d artist creating assets for a short commercial. They sculpted the bird in the video with nothing but two controllers which they used to manipulate generative matter in 3d. The AI model changed in realtime and adopted their style of work, both based on the work they did on the scene, and on reference images they gave the system. Seeing the 3d model transform in realtime as they were working on it was incredible. When it started moving and reacted to their instructions, I knew it was time to return to the present :D #art #ai

So remember that time machine I told you about? I managed to sneek out another video from my visit to the future.. this time I captured video of a 3d artist creating assets for a short commercial. They sculpted the bird in the video with nothing but two controllers which they used to manipulate generative matter in 3d. The AI model changed in realtime and adopted their style of work, both based on the work they did on the scene, and on reference images they gave the system. Seeing the 3d model transform in realtime as they were working on it was incredible. When it started moving and reacted to their instructions, I knew it was time to return to the present :D #art #ai

Martin Nebelong

22,044 Aufrufe • vor 2 Jahren

The next phase of AI video needs infrastructure, not more tools.. This isn’t just a product update. It’s a new category. Topview 4.0 is the world’s first AI video collaboration board, built for how creators and teams actually work. One shared workspace to create, review, and iterate together without switching tools. Prompt → Image → Video → Avatar in a single flow. Real-time collaboration. All top image and video models in one place. Creator-friendly pricing, cheaper than official APIs. If AI video is part of your workflow, this is the board it belongs on. TopviewAI

Kuria | AI

12,854 Aufrufe • vor 8 Monaten

Most AI video tools generate clips. The interesting ones help you build a complete creative workflow. MiniMax H3 stood out because it can use text, images, audio, and video together with Omni Reference. That makes it easier to keep the style, branding, and creative direction consistent from the first prompt to the final video. What caught my attention: • Text + image + audio + video inputs • Native 2K video generation • Better control over edits and refinements • Built for ads, e-commerce, gaming, and branded content AI video is moving beyond single prompts. It is becoming a creative workspace.MiniMax Design (H3) Try here 👇 #hailuoai #hailuo #MiniMaxH3 #H3 #AIVideo #GenerativeAI

Most AI video tools generate clips. The interesting ones help you build a complete creative workflow. MiniMax H3 stood out because it can use text, images, audio, and video together with Omni Reference. That makes it easier to keep the style, branding, and creative direction consistent from the first prompt to the final video. What caught my attention: • Text + image + audio + video inputs • Native 2K video generation • Better control over edits and refinements • Built for ads, e-commerce, gaming, and branded content AI video is moving beyond single prompts. It is becoming a creative workspace.MiniMax Design (H3) Try here 👇 #hailuoai #hailuo #MiniMaxH3 #H3 #AIVideo #GenerativeAI

Alina Davy

92,045 Aufrufe • vor 2 Monaten

AI Is Moving Beyond “Generating Videos” — Toward “Generating Worlds” Over the past two years, AI video models have advanced at an astonishing pace. From Runway and Pika to Sora and Veo, AI-generated videos have become increasingly realistic and more consistent with the physical laws of the real world. Many people believe the next objective is simply to generate videos that are longer, sharper, and more lifelike. But if we take a step back, we can see that the real transformation is not happening in video itself. It is happening in world models. What Is a World Model? In 1943, psychologist Kenneth Craik proposed an idea that would influence artificial intelligence research for decades. He argued that the human brain does not merely react to the outside world. Instead, it maintains an internal model of how the world works. Because we have this internal model, we can predict the outcome of an action before we actually take it. Before crossing a road, we estimate whether a car will pass by. Before catching a ball, we predict its trajectory. These abilities come from continuously simulating the world in our minds, rather than relying entirely on trial and error. This idea later became known by a more formal term: World Model. A world model does not describe a single image or a fixed video clip. It is an internal representation capable of continuously simulating the rules and dynamics of the real world. Why Is AI Research Turning Toward World Models? Because predicting “what comes next” is becoming increasingly central to how AI systems work. Language models predict the next token. Image models predict the next step in the denoising process. Video models predict the next frame. A world model, however, attempts to predict something broader: What should the world look like in the next moment? In 2018, David Ha and Jürgen Schmidhuber proposed in their paper World Models that an intelligent agent could first learn a model of the world, and then use that internal model to plan its actions. The Dreamer series later demonstrated that many complex tasks could be learned by training agents inside an “imagined world.” At the same time, the development of video models such as Sora and Veo led researchers to another realization: A model capable of continuously generating video has already learned, at least implicitly, many of the rules governing the real world. As a result, these two research directions have gradually begun to converge. But Video Is Not Yet a World This is where the distinction is often misunderstood. For a world model to support meaningful real-time interaction, it must solve several critical problems. Most video models today are essentially answering one question: What should the next frame look like? A true world model needs to answer much more: What happens if I take one step forward? If I walk behind a building and then return, will the building still be there? If I suddenly change the camera angle, will the entire space remain consistent? If I enter a command such as: “Summon a dragon.” Will the world respond immediately? In other words, a world model must do more than generate content. It must understand space. It must understand time. It must understand causality. And it must understand interaction. Moving from watching to participating is where the real difficulty of world models begins. World Models Are Entering the Interactive Era One of the latest attempts in this direction is Alaya World, recently open-sourced by Alaya World, or Alaya Lab. Instead of generating a fixed video clip, it generates a world that users can explore in real time. Users can begin with text, an image, or a video, enter the generated scene, move freely through it, and introduce new prompts at any moment during generation. The world responds immediately. According to the publicly released information, Alaya World provides: Real-time streaming generation at 720p and 24 FPS Stable continuous exploration for more than one minute The ability to switch prompts and trigger skills or events during generation Model weights and inference code released under the Apache 2.0 License Training code and datasets planned for future release What makes these capabilities important is not simply the technical specifications. It is that the generated “world” can now support continuous interaction. The official demo shows that users can genuinely control, transform, and explore the generated environment. AI Is Evolving From a Tool Into an Environment Over the past few years, most discussions around AI have focused on content generation. Generating text. Generating images. Generating videos. But world models raise a fundamentally different question: Can AI generate an environment that people can inhabit, explore, and continuously evolve? If the answer is yes, the impact will extend far beyond video generation. Game development, robotics training, embodied intelligence, digital twins, virtual production, and many other fields could be transformed by the development of world models. World models are still at a very early stage. Yet from Craik’s proposal of an internal mental model more than eighty years ago to the emergence of today’s interactive world-generation systems, a clear evolutionary path is beginning to take shape. Perhaps what AI is ultimately learning has never been limited to images, videos, or language. Perhaps it is learning the world itself. References GitHub: Technical Report:

AI Is Moving Beyond “Generating Videos” — Toward “Generating Worlds” Over the past two years, AI video models have advanced at an astonishing pace. From Runway and Pika to Sora and Veo, AI-generated videos have become increasingly realistic and more consistent with the physical laws of the real world. Many people believe the next objective is simply to generate videos that are longer, sharper, and more lifelike. But if we take a step back, we can see that the real transformation is not happening in video itself. It is happening in world models. What Is a World Model? In 1943, psychologist Kenneth Craik proposed an idea that would influence artificial intelligence research for decades. He argued that the human brain does not merely react to the outside world. Instead, it maintains an internal model of how the world works. Because we have this internal model, we can predict the outcome of an action before we actually take it. Before crossing a road, we estimate whether a car will pass by. Before catching a ball, we predict its trajectory. These abilities come from continuously simulating the world in our minds, rather than relying entirely on trial and error. This idea later became known by a more formal term: World Model. A world model does not describe a single image or a fixed video clip. It is an internal representation capable of continuously simulating the rules and dynamics of the real world. Why Is AI Research Turning Toward World Models? Because predicting “what comes next” is becoming increasingly central to how AI systems work. Language models predict the next token. Image models predict the next step in the denoising process. Video models predict the next frame. A world model, however, attempts to predict something broader: What should the world look like in the next moment? In 2018, David Ha and Jürgen Schmidhuber proposed in their paper World Models that an intelligent agent could first learn a model of the world, and then use that internal model to plan its actions. The Dreamer series later demonstrated that many complex tasks could be learned by training agents inside an “imagined world.” At the same time, the development of video models such as Sora and Veo led researchers to another realization: A model capable of continuously generating video has already learned, at least implicitly, many of the rules governing the real world. As a result, these two research directions have gradually begun to converge. But Video Is Not Yet a World This is where the distinction is often misunderstood. For a world model to support meaningful real-time interaction, it must solve several critical problems. Most video models today are essentially answering one question: What should the next frame look like? A true world model needs to answer much more: What happens if I take one step forward? If I walk behind a building and then return, will the building still be there? If I suddenly change the camera angle, will the entire space remain consistent? If I enter a command such as: “Summon a dragon.” Will the world respond immediately? In other words, a world model must do more than generate content. It must understand space. It must understand time. It must understand causality. And it must understand interaction. Moving from watching to participating is where the real difficulty of world models begins. World Models Are Entering the Interactive Era One of the latest attempts in this direction is Alaya World, recently open-sourced by Alaya World, or Alaya Lab. Instead of generating a fixed video clip, it generates a world that users can explore in real time. Users can begin with text, an image, or a video, enter the generated scene, move freely through it, and introduce new prompts at any moment during generation. The world responds immediately. According to the publicly released information, Alaya World provides: Real-time streaming generation at 720p and 24 FPS Stable continuous exploration for more than one minute The ability to switch prompts and trigger skills or events during generation Model weights and inference code released under the Apache 2.0 License Training code and datasets planned for future release What makes these capabilities important is not simply the technical specifications. It is that the generated “world” can now support continuous interaction. The official demo shows that users can genuinely control, transform, and explore the generated environment. AI Is Evolving From a Tool Into an Environment Over the past few years, most discussions around AI have focused on content generation. Generating text. Generating images. Generating videos. But world models raise a fundamentally different question: Can AI generate an environment that people can inhabit, explore, and continuously evolve? If the answer is yes, the impact will extend far beyond video generation. Game development, robotics training, embodied intelligence, digital twins, virtual production, and many other fields could be transformed by the development of world models. World models are still at a very early stage. Yet from Craik’s proposal of an internal mental model more than eighty years ago to the emergence of today’s interactive world-generation systems, a clear evolutionary path is beginning to take shape. Perhaps what AI is ultimately learning has never been limited to images, videos, or language. Perhaps it is learning the world itself. References GitHub: Technical Report:

雪踏乌云

113,347 Aufrufe • vor 2 Monaten

MiniMax H3 is a serious upgrade for AI video creation. Instead of relying on just text prompts, you can combine images, videos, and audio references to control the character, motion, camera, and sound. What stands out: → Up to 9 image + 3 video + 3 audio references → Native synced voice, music & sound effects → 2K video generation up to 15 seconds → Instruction-based video editing → Motion transfer → Better character, camera & voice control The interesting part is that you can build a full scene from references instead of endlessly regenerating until the result feels right. 🔗 Now is a great time to try MiniMax H3 on Magnific, especially with the current 2K offer.

MiniMax H3 is a serious upgrade for AI video creation. Instead of relying on just text prompts, you can combine images, videos, and audio references to control the character, motion, camera, and sound. What stands out: → Up to 9 image + 3 video + 3 audio references → Native synced voice, music & sound effects → 2K video generation up to 15 seconds → Instruction-based video editing → Motion transfer → Better character, camera & voice control The interesting part is that you can build a full scene from references instead of endlessly regenerating until the result feels right. 🔗 Now is a great time to try MiniMax H3 on Magnific, especially with the current 2K offer.

Tanvir Anjum

32,271 Aufrufe • vor 1 Monat

Brand system in video generation I created this video using 1 brand guide, 1 brand Skill, and 1 video model AI is now turning product videos into commodity, so brands can capture attention more effectively However, harnessing video generation isn't straightforward without specifically preparing your video model to generate brand-locked content I outline this step-by-step process in the Article below - read on and save it for later:

AmirMušić

294,957 Aufrufe • vor 1 Monat

I just launched a brand new channel with an INSANE strategy. I'm calling it the Launch Loop. I've been working on this channel for over a year, and took the actually massive risk to hold off on posting until I had all 5 launch videos ready. Each idea was carefully selected for this strategy, and each one was carefully written to execute it. I view this channel launch, and these videos, not as individual videos but as a season of a new show. The plan was this: come up with video ideas that fit together, and write the end of each video to smoothly blend into the next. I treated the conclusion of each video as if it was the intro of the next video, so that if people enjoyed what I made and wanted to see more, they could click the next video on the End Card This was inspired by work I've done in the past: a few years ago on my YouTube strategy channel, I posted a video about Ryan Trahan's thumbnail strategy which has gotten over 2M lifetime views. But at the end of that video, I carefully wrote a transition showing video my comprehensive breakdown of 21 thumbnail formats. The End Screen video CTR on that video is 29.4% 🤯. I'm convinced that some of those 2M views must have come from the success of that end screen. I've come to believe that it's really important to bring in an audience to your CHANNEL, not just one video. That was a mistake I made in another channel I worked on before! Getting virality on individual videos, but not keeping viewers for the long term or growing an actualy community. So I thought, what if I launched a channel where every video pointed to the next, in a loop? Maybe there will be a flywheel where success on one brings success on the next, and the next, and the next... But for this plan to work the videos I made have to actually be good 😅

Tyler Fleischman ⏸️

73,315 Aufrufe • vor 17 Tagen