Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

I am blown away 🤯. Check this out! CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models TL;DR: "To enable broader exploration of dynamic scenes, our model can generate new video clips of the same scene based on previously generated content and user-provided camera trajectories. This approach maintains... dynamic capabilities, accurate camera control, and scene consistency throughout the extended exploration." "Our model enables precise camera control across diverse scenarios while preserving dynamic scene elements, e.g." "Our method can generate videos with strong 3D consistency, which enables high-quality 3D reconstruction using the camera-controlled videos." Contributions: 1) A systematic data curation pipeline for constructing a dynamic video dataset with camera trajectory annotations; 2) A lightweight camera control injection module and corresponding training strategy that preserves dynamic video generation capabilities while adding camera control effect; 3) A clip-wise autoregressive generation recipe that enables extended range exploration of generated scenes.show more

MrNeRF

16,849 subscribers

12,633 views • 1 year ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,849 views • 1 year ago

AI video is moving beyond "generate a clip". We're entering the era of directing entire scenes. With MiniMax H3 on Magnific, you can combine: → Text → Images → Videos → Audio in a single workflow. Then control: → Character → Camera → Motion → Sound with far more consistency than a prompt alone. That's what makes this model interesting. 50% OFF until September 1.

AI video is moving beyond "generate a clip". We're entering the era of directing entire scenes. With MiniMax H3 on Magnific, you can combine: → Text → Images → Videos → Audio in a single workflow. Then control: → Character → Camera → Motion → Sound with far more consistency than a prompt alone. That's what makes this model interesting. 50% OFF until September 1.

Bishal Nandi

53,240 views • 11 days ago

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,062 views • 1 year ago

MiniMax H3 is a serious upgrade for AI video creation. Instead of relying on just text prompts, you can combine images, videos, and audio references to control the character, motion, camera, and sound. What stands out: → Up to 9 image + 3 video + 3 audio references → Native synced voice, music & sound effects → 2K video generation up to 15 seconds → Instruction-based video editing → Motion transfer → Better character, camera & voice control The interesting part is that you can build a full scene from references instead of endlessly regenerating until the result feels right. 🔗 Now is a great time to try MiniMax H3 on Magnific, especially with the current 2K offer.

MiniMax H3 is a serious upgrade for AI video creation. Instead of relying on just text prompts, you can combine images, videos, and audio references to control the character, motion, camera, and sound. What stands out: → Up to 9 image + 3 video + 3 audio references → Native synced voice, music & sound effects → 2K video generation up to 15 seconds → Instruction-based video editing → Motion transfer → Better character, camera & voice control The interesting part is that you can build a full scene from references instead of endlessly regenerating until the result feels right. 🔗 Now is a great time to try MiniMax H3 on Magnific, especially with the current 2K offer.

Tanvir Anjum

31,373 views • 9 days ago

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.

MrNeRF

21,346 views • 1 year ago

🔥 VIDU Multi-Entity Consistency Give Vidu 2/3 images and it’ll turn them into a video—it’s pure magic! ✨ Your own characters interacting with objects and in the exact environment you want! Ads, movies… endless possibilities, and this is just the beginning! Thanks @Viduforhuman The future is a carrot! 🥕 Plus, how about grabbing any frame from a Vidu-generated video "from scratch" and throwing it into another AI video or image tool to push your project even further? For now, check out the comment below: I scaled up a frame with Magnific.ai and fed it into Runway to create a dynamic shot using full camera control. But fingers crossed I can soon use #ReCapture by Bisho & team to generate new shots from the same video!

🔥 VIDU Multi-Entity Consistency Give Vidu 2/3 images and it’ll turn them into a video—it’s pure magic! ✨ Your own characters interacting with objects and in the exact environment you want! Ads, movies… endless possibilities, and this is just the beginning! Thanks @Viduforhuman The future is a carrot! 🥕 Plus, how about grabbing any frame from a Vidu-generated video "from scratch" and throwing it into another AI video or image tool to push your project even further? For now, check out the comment below: I scaled up a frame with Magnific.ai and fed it into Runway to create a dynamic shot using full camera control. But fingers crossed I can soon use #ReCapture by Bisho & team to generate new shots from the same video!

Hungry Donkey 🥕

37,561 views • 1 year ago

Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering discuss: The correct insertion of virtual objects in images of real-world scenes requires a deep understanding of the scene's lighting, geometry and materials, as well as the image formation process. While recent large-scale diffusion models have shown strong generative and inpainting capabilities, we find that current models do not sufficiently "understand" the scene shown in a single picture to generate consistent lighting effects (shadows, bright reflections, etc.) while preserving the identity and details of the composited object. We propose using a personalized large diffusion model as guidance to a physically based inverse rendering process. Our method recovers scene lighting and tone-mapping parameters, allowing the photorealistic composition of arbitrary virtual objects in single frames or videos of indoor or outdoor scenes. Our physically based pipeline further enables automatic materials and tone-mapping refinement.

Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering discuss: The correct insertion of virtual objects in images of real-world scenes requires a deep understanding of the scene's lighting, geometry and materials, as well as the image formation process. While recent large-scale diffusion models have shown strong generative and inpainting capabilities, we find that current models do not sufficiently "understand" the scene shown in a single picture to generate consistent lighting effects (shadows, bright reflections, etc.) while preserving the identity and details of the composited object. We propose using a personalized large diffusion model as guidance to a physically based inverse rendering process. Our method recovers scene lighting and tone-mapping parameters, allowing the photorealistic composition of arbitrary virtual objects in single frames or videos of indoor or outdoor scenes. Our physically based pipeline further enables automatic materials and tone-mapping refinement.

AK

19,101 views • 2 years ago

📖THE BEST SEEDANCE 2.0 WORKFLOW STARTS INSIDE CHATGPT IMAGE 2 One creator can now go from storyboard to cinematic short film without a camera, actors, or a production crew Most creators use Seedance 2.0 as a video generator. The real power comes from combining ChatGPT Image 2 + Seedance 2.0 into a production pipeline. Here’s the workflow: 1.Create a story idea in ChatGPT. 2.Generate a shot-by-shot storyboard. 3. Build character sheets to lock consistency. 4.Generate every scene in ChatGPT Image 2. 5.Define camera movements and actions. 6.Animate each shot in Seedance 2.0. 7.Stitch the clips together into a finished short film. Why this works: • Consistent characters across scenes • Better storytelling • Precise camera control • Faster iteration • Professional-looking results • One person can do the work of an entire production team Use cases: ⁃Viral AI shorts ⁃YouTube animations ⁃Brand commercials ⁃Educational content ⁃Story-driven ads ⁃Social media content The future of AI video isn’t prompting. It’s building production pipelines. 📥Tomorrow I am sharing a workflow that almost nobody is using but absolutely should be. 🔖 The full breakdown from setup to final export is waiting in the pinned article.

Zentrix⌚️

50,553 views • 2 months ago

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

MrNeRF

27,428 views • 1 year ago

Video diffusion models have strong implicit representations of 3D shape, material, and lighting, but controlling them with language is cumbersome, and control is critical for artists and animators. GenLit connects these implicit representations with a continuous 5D control signal describing the direction and intensity of a point light source. This enables single-image near-field relighting of an image using a video diffusion model. We use a ControlNet-like approach and show that, with a small amount of synthetic data, GenLit generalizes to complex real-world images. Given a single image and the 5D lighting signal, GenLit creates a video of a moving light source that is inside the scene. It moves around and behind scene objects, producing effects such as shading, cast shadows, secularities, and interreflections with a realism that is hard to obtain with traditional inverse rendering methods. GenLit shows that it is possible to get continuous control over implicit physical processes within a video model. I think this is just the beginning and promises to make such models much more practical for creators. Shrisha Bharadwaj will present today at SIGGRAPH Asia Room: S423/S424, Level 4 @ 13:50 on 15 of Dec.

Video diffusion models have strong implicit representations of 3D shape, material, and lighting, but controlling them with language is cumbersome, and control is critical for artists and animators. GenLit connects these implicit representations with a continuous 5D control signal describing the direction and intensity of a point light source. This enables single-image near-field relighting of an image using a video diffusion model. We use a ControlNet-like approach and show that, with a small amount of synthetic data, GenLit generalizes to complex real-world images. Given a single image and the 5D lighting signal, GenLit creates a video of a moving light source that is inside the scene. It moves around and behind scene objects, producing effects such as shading, cast shadows, secularities, and interreflections with a realism that is hard to obtain with traditional inverse rendering methods. GenLit shows that it is possible to get continuous control over implicit physical processes within a video model. I think this is just the beginning and promises to make such models much more practical for creators. Shrisha Bharadwaj will present today at SIGGRAPH Asia Room: S423/S424, Level 4 @ 13:50 on 15 of Dec.

Michael Black

22,182 views • 8 months ago

Break-A-Scene: Extracting Multiple Concepts from a Single Image introduce the task of textual scene decomposition: given a single image of a scene that may contain several concepts, we aim to extract a distinct text token for each concept, enabling fine-grained control over the generated scenes. To this end, we propose augmenting the input image with masks that indicate the presence of target concepts. These masks can be provided by the user or generated automatically by a pre-trained segmentation model. We then present a novel two-phase customization process that optimizes a set of dedicated textual embeddings (handles), as well as the model weights, striking a delicate balance between accurately capturing the concepts and avoiding overfitting. We employ a masked diffusion loss to enable handles to generate their assigned concepts, complemented by a novel loss on cross-attention maps to prevent entanglement. We also introduce union-sampling, a training strategy aimed to improve the ability of combining multiple concepts in generated images. We use several automatic metrics to quantitatively compare our method against several baselines, and further affirm the results using a user study. Finally, we showcase several applications of our method paper page:

Break-A-Scene: Extracting Multiple Concepts from a Single Image introduce the task of textual scene decomposition: given a single image of a scene that may contain several concepts, we aim to extract a distinct text token for each concept, enabling fine-grained control over the generated scenes. To this end, we propose augmenting the input image with masks that indicate the presence of target concepts. These masks can be provided by the user or generated automatically by a pre-trained segmentation model. We then present a novel two-phase customization process that optimizes a set of dedicated textual embeddings (handles), as well as the model weights, striking a delicate balance between accurately capturing the concepts and avoiding overfitting. We employ a masked diffusion loss to enable handles to generate their assigned concepts, complemented by a novel loss on cross-attention maps to prevent entanglement. We also introduce union-sampling, a training strategy aimed to improve the ability of combining multiple concepts in generated images. We use several automatic metrics to quantitatively compare our method against several baselines, and further affirm the results using a user study. Finally, we showcase several applications of our method paper page:

AK

154,511 views • 3 years ago

1/ We've all been aware of the hype surrounding Dreamina Seedance 2.0, and I finally got early access to the tool. And wow, I'm blown away. This is by far the best. It is starting to make AI video feel less like "generate a clip" and more like "direct a scene." What stood out to me is the level of control: camera motion, pacing, visual consistency, and the ability to build from multiple references inside one workflow. Some of the prompt directions that feel especially strong: - a busy modern city square during daytime. Suddenly, time freezes completely - a single continuous camera movement through a natural landscape that transitions through all four seasons in one shot - an underwater bioluminescent city waking up at dawn The big shift is this: One Prompt, Viral Remade. Edit Videos as Easy as Editing Photos. Dreamina Seedance 2.0 feels like a real step toward AI-native directing rather than just AI generation. Here are some examples 🧵:

1/ We've all been aware of the hype surrounding Dreamina Seedance 2.0, and I finally got early access to the tool. And wow, I'm blown away. This is by far the best. It is starting to make AI video feel less like "generate a clip" and more like "direct a scene." What stood out to me is the level of control: camera motion, pacing, visual consistency, and the ability to build from multiple references inside one workflow. Some of the prompt directions that feel especially strong: - a busy modern city square during daytime. Suddenly, time freezes completely - a single continuous camera movement through a natural landscape that transitions through all four seasons in one shot - an underwater bioluminescent city waking up at dawn The big shift is this: One Prompt, Viral Remade. Edit Videos as Easy as Editing Photos. Dreamina Seedance 2.0 feels like a real step toward AI-native directing rather than just AI generation. Here are some examples 🧵:

Chubby♨️

76,142 views • 5 months ago

Honestly... this makes a lot of sense. I've lost count of how many times I've regenerated an AI video just because someone was standing in the wrong place or the camera angle felt off. If I can quickly lay out the scene first and then let the model generate from that, I'd much rather work that way than keep tweaking prompts over and over. Seems like a really useful addition for anyone making AI videos. Props to TopviewAI for building this.

Honestly... this makes a lot of sense. I've lost count of how many times I've regenerated an AI video just because someone was standing in the wrong place or the camera angle felt off. If I can quickly lay out the scene first and then let the model generate from that, I'd much rather work that way than keep tweaking prompts over and over. Seems like a really useful addition for anyone making AI videos. Props to TopviewAI for building this.

Gina Acosta

103,147 views • 1 month ago

I’ve used all the recent GenAI video models extensively & here’s my 2¢: 🎬 Runway Gen3 Alpha - best image quality & motion for text-to-video & embedded words. Great at prompt travel changes over the course of 10 sec. And I’m super bullish on how gen3 will evolve, hopefully adopting the features listed below. Kling - best quality for image-to-video with prompt control, like eating food. Great clip extension that accounts for character (ie walking stride) & camera movement (speed & angle), rather than just using final frame. But it’s limited availability & Chinese native language is limiting. Used for Spider-Man video below (via Midjourney). LumaLabs - best for keyframe start & end control (it can not be overstated how important this is. other services should add it ASAP!) and their high dynamic action movements are really fun. Luma was used in my viral Multiverse of Memes video. PikaLabs - they haven’t gotten as much attention as others lately. But they did update their video model a few weeks ago and it looks great. Also, they are notable for their unique & AWESOME features, like video in-painting & out-painting. My perfect AI video platform would have the following features: 1) Gen3’s quality, prompt control & text embedding. 2) KLing’s image-to-video quality, prompt control & clip extension quality. 3) Luma’s multi-keyframe control & dynamic movement ability. 4) Pika’s inpainting & outpainting ability. And a video-to-video (aka next-gen Runway gen1) could be a game changer, too. It’s an exciting time to be alive 🫶 Who will get there first? 🔉🔉

I’ve used all the recent GenAI video models extensively & here’s my 2¢: 🎬 Runway Gen3 Alpha - best image quality & motion for text-to-video & embedded words. Great at prompt travel changes over the course of 10 sec. And I’m super bullish on how gen3 will evolve, hopefully adopting the features listed below. Kling - best quality for image-to-video with prompt control, like eating food. Great clip extension that accounts for character (ie walking stride) & camera movement (speed & angle), rather than just using final frame. But it’s limited availability & Chinese native language is limiting. Used for Spider-Man video below (via Midjourney). LumaLabs - best for keyframe start & end control (it can not be overstated how important this is. other services should add it ASAP!) and their high dynamic action movements are really fun. Luma was used in my viral Multiverse of Memes video. PikaLabs - they haven’t gotten as much attention as others lately. But they did update their video model a few weeks ago and it looks great. Also, they are notable for their unique & AWESOME features, like video in-painting & out-painting. My perfect AI video platform would have the following features: 1) Gen3’s quality, prompt control & text embedding. 2) KLing’s image-to-video quality, prompt control & clip extension quality. 3) Luma’s multi-keyframe control & dynamic movement ability. 4) Pika’s inpainting & outpainting ability. And a video-to-video (aka next-gen Runway gen1) could be a game changer, too. It’s an exciting time to be alive 🫶 Who will get there first? 🔉🔉

Blaine Brown

26,535 views • 2 years ago

3D Gaussian Splatting for Real-Time Radiance Field Rendering paper page: Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.

3D Gaussian Splatting for Real-Time Radiance Field Rendering paper page: Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.

AK

633,674 views • 3 years ago

Blender just became a prompt box. With Kimi K3 connected through Blender MCP, you can describe a scene like this in plain English and let the model handle the ugly part: terrain, buildings, lighting, materials, camera movement, animation, and the Python scripts holding everything together. The interesting part is not the first render. Kimi can inspect the result, notice that the camera clips through a tree or the city looks suspiciously like plastic, then edit the actual Blender scene and render it again instead of restarting from zero. You still need taste, because “make it cinematic” remains one of humanity’s least useful instructions. But the distance between an empty Blender file and a fully editable 3D world just got embarrassingly small. A few years ago, creating this meant weeks of modeling, scripting, lighting, and animation. Now you can describe the world, watch Kimi build it, and spend your time fixing the final 10% instead of manually constructing the first 90%. Kimi K3 + Blender MCP is basically text-to-3D without trapping the result inside a useless generated video. Every object, material, light, camera, and keyframe stays editable.

Blender just became a prompt box. With Kimi K3 connected through Blender MCP, you can describe a scene like this in plain English and let the model handle the ugly part: terrain, buildings, lighting, materials, camera movement, animation, and the Python scripts holding everything together. The interesting part is not the first render. Kimi can inspect the result, notice that the camera clips through a tree or the city looks suspiciously like plastic, then edit the actual Blender scene and render it again instead of restarting from zero. You still need taste, because “make it cinematic” remains one of humanity’s least useful instructions. But the distance between an empty Blender file and a fully editable 3D world just got embarrassingly small. A few years ago, creating this meant weeks of modeling, scripting, lighting, and animation. Now you can describe the world, watch Kimi build it, and spend your time fixing the final 10% instead of manually constructing the first 90%. Kimi K3 + Blender MCP is basically text-to-3D without trapping the result inside a useless generated video. Every object, material, light, camera, and keyframe stays editable.

Rina

264,903 views • 1 month ago

This is some quietly impressive work on making video world models actually controllable in 4D space. VerseCrafter lets you take an input image, use something like Blender to animate the 3D camera path and object trajectories, then uses that to condition generation. Scribbling in 2D feels so crude in comparison. The authors represent everything in a shared 4D world state - static background as a point cloud, moving objects as 3D gaussian trajectories. The gaussians are an interesting choice because they capture position, shape, and orientation probabilistically rather than forcing rigid bounding boxes or category specific models like SMPL-X for human bodies. They bolt this onto frozen Wan2.1 with a lightweight adapter, so they get a strong video prior. They also built a pipeline to auto extract 4D annotations from real world videos to train this puppy. It doesn't look sexy yet, but IMO this is the interface video world models need - actual 3D authoring tools to exert control rather than crude scribbles and prompt incantations.

This is some quietly impressive work on making video world models actually controllable in 4D space. VerseCrafter lets you take an input image, use something like Blender to animate the 3D camera path and object trajectories, then uses that to condition generation. Scribbling in 2D feels so crude in comparison. The authors represent everything in a shared 4D world state - static background as a point cloud, moving objects as 3D gaussian trajectories. The gaussians are an interesting choice because they capture position, shape, and orientation probabilistically rather than forcing rigid bounding boxes or category specific models like SMPL-X for human bodies. They bolt this onto frozen Wan2.1 with a lightweight adapter, so they get a strong video prior. They also built a pipeline to auto extract 4D annotations from real world videos to train this puppy. It doesn't look sexy yet, but IMO this is the interface video world models need - actual 3D authoring tools to exert control rather than crude scribbles and prompt incantations.

Bilawal Sidhu

26,017 views • 7 months ago

A cinematic AI-powered visual experience showcasing how imagination can be transformed into stunning digital worlds From futuristic environments and creative fashion visuals to seamless character transformations every scene blends technology creativity, and storytelling into one immersive piece. Made with MiniMaxH3 on WeryAI Wery Prompt: A stylish young South Asian woman with long dark hair, natural glamorous makeup and an elegant modern outfit. Keep her face, hairstyle, outfit and appearance consistent throughout the entire video. Scene 1 — 0–4s: She sits at a sleek modern workstation in a futuristic studio. She looks directly into the camera and says naturally: “What if you could turn your ideas into reality in just a few clicks?” Natural facial expressions and hand gestures. Slow cinematic camera push-in. Perfect lip-sync. Scene 2 — 4–7s: She turns to her computer and opens WeryAI. She types a creative idea and says: “That’s exactly what I love about WeryAI.” The interface responds with elegant AI processing animations and glowing digital effects. Scene 3 — 7–11s: Her simple idea transforms into stunning AI-generated visuals: a fashion scene, futuristic city, cinematic character, premium product advertisement and social-media video. She looks impressed and says: “Give it an idea, and watch it come to life.” Use smooth cinematic transitions and dynamic camera movement. Scene 4 — 11–15s: She turns back toward the camera with a confident smile and says: “WeryAI. Your imagination, powered by AI.” The camera slowly pulls back as the WeryAI brand reveal appears with a clean futuristic glow. Audio: Natural confident female voice, warm energetic delivery, subtle futuristic background music and soft cinematic sound effects. Keep dialogue clear and prominent. Lip-sync: The woman must visibly speak every line with accurate lip synchronization, natural mouth movement, blinking and facial expressions. Visual quality: Photorealistic, cinematic 4K, realistic skin, smooth camera movement, premium lighting, shallow depth of field, consistent character. No subtitles, no distorted face, no warped hands, no extra fingers, no flickering, no random text, no watermark.

Calira

12,635 views • 7 days ago

Two cinematic prompts for Seedance 2.0 right here 👇 [STYLE + CAMERA + ATMOSPHERE] Ultra-photorealistic cinematic 15-second action sequence from a grounded action thriller film. Nighttime at an abandoned industrial train yard. Old trains, rusted tracks, broken platforms and warehouses. Strong moonlight and industrial lighting. The visual language is gritty and realistic with hyper-realistic practical destruction. Camera is handheld-dynamic with urgent moves. [IMAGE REFERENCES] No reference images provided. Generate a fully consistent lead character: a male fugitive in his mid-30s, short hair, intense expression. He wears a dark jacket, jeans and boots. Maintain exact same appearance and clothing throughout the clip. [TIMELINE SECOND BY SECOND] 0-3s: [Dynamic handheld tracking shot from side-rear] The man runs across the train yard as a freight train derails violently after hitting a collapsed section of track. Massive train cars tip over and crash into the ground, sending metal, wood and cargo flying. The camera tracks him urgently as the destruction spreads. 3-7s: [Slow continuous orbiting camera moving around the frozen destruction] At the peak of the derailment, everything freezes in perfect realistic physics. Huge train cars, twisted metal, wooden debris and clouds of dust hang motionless in the air with accurate weight and trajectories. The camera performs a smooth, continuous orbit through the frozen chaos, moving between large suspended train cars and passing close to sharp metal fragments while the man remains visible. 7-15s: [Dynamic tracking shot as time resumes] At the 7-second mark time snaps back to forward motion with a slight realistic temporal residue. Some outer debris shows a very subtle reverse lag before continuing. The man has used the frozen moment to dive behind a concrete barrier. When motion resumes, several large pieces of metal miss him. He stays low and runs toward an open warehouse as debris finally crashes down. The camera tracks with him through the dust. [STYLE & QUALITY BOOSTERS] Photorealistic 8K, hyper-realistic rigid body destruction with correct material properties (metal, wood, train parts), perfect frozen mid-air physics, seamless transition from freeze to resume, natural dust interaction, heavy cinematic motion blur on fast debris, stable character performance, movie-level practical destruction VFX quality.

Two cinematic prompts for Seedance 2.0 right here 👇 [STYLE + CAMERA + ATMOSPHERE] Ultra-photorealistic cinematic 15-second action sequence from a grounded action thriller film. Nighttime at an abandoned industrial train yard. Old trains, rusted tracks, broken platforms and warehouses. Strong moonlight and industrial lighting. The visual language is gritty and realistic with hyper-realistic practical destruction. Camera is handheld-dynamic with urgent moves. [IMAGE REFERENCES] No reference images provided. Generate a fully consistent lead character: a male fugitive in his mid-30s, short hair, intense expression. He wears a dark jacket, jeans and boots. Maintain exact same appearance and clothing throughout the clip. [TIMELINE SECOND BY SECOND] 0-3s: [Dynamic handheld tracking shot from side-rear] The man runs across the train yard as a freight train derails violently after hitting a collapsed section of track. Massive train cars tip over and crash into the ground, sending metal, wood and cargo flying. The camera tracks him urgently as the destruction spreads. 3-7s: [Slow continuous orbiting camera moving around the frozen destruction] At the peak of the derailment, everything freezes in perfect realistic physics. Huge train cars, twisted metal, wooden debris and clouds of dust hang motionless in the air with accurate weight and trajectories. The camera performs a smooth, continuous orbit through the frozen chaos, moving between large suspended train cars and passing close to sharp metal fragments while the man remains visible. 7-15s: [Dynamic tracking shot as time resumes] At the 7-second mark time snaps back to forward motion with a slight realistic temporal residue. Some outer debris shows a very subtle reverse lag before continuing. The man has used the frozen moment to dive behind a concrete barrier. When motion resumes, several large pieces of metal miss him. He stays low and runs toward an open warehouse as debris finally crashes down. The camera tracks with him through the dust. [STYLE & QUALITY BOOSTERS] Photorealistic 8K, hyper-realistic rigid body destruction with correct material properties (metal, wood, train parts), perfect frozen mid-air physics, seamless transition from freeze to resume, natural dust interaction, heavy cinematic motion blur on fast debris, stable character performance, movie-level practical destruction VFX quality.

TechHalla

16,316 views • 1 month ago