Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

I am blown away 🤯. Check this out! CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models TL;DR: "To enable broader exploration of dynamic scenes, our model can generate new video clips of the same scene based on previously generated content and user-provided camera trajectories. This approach maintains...

janusch_patas's profile picture

MrNeRF

16,849 subscribers

12,633 Aufrufe • vor 1 Jahr •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.
0:12

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,849 Aufrufe • vor 1 Jahr

AI video is moving beyond "generate a clip". We're entering the era of directing entire scenes. With MiniMax H3 on Magnific, you can combine: → Text → Images → Videos → Audio in a single workflow. Then control: → Character → Camera → Motion → Sound with far more consistency than a prompt alone. That's what makes this model interesting. 50% OFF until September 1.
0:13

AI video is moving beyond "generate a clip". We're entering the era of directing entire scenes. With MiniMax H3 on Magnific, you can combine: → Text → Images → Videos → Audio in a single workflow. Then control: → Character → Camera → Motion → Sound with far more consistency than a prompt alone. That's what makes this model interesting. 50% OFF until September 1.

Bishal Nandi

53,240 Aufrufe • vor 1 Monat

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.
0:18

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,062 Aufrufe • vor 1 Jahr

MiniMax H3 is a serious upgrade for AI video creation. Instead of relying on just text prompts, you can combine images, videos, and audio references to control the character, motion, camera, and sound. What stands out: → Up to 9 image + 3 video + 3 audio references → Native synced voice, music & sound effects → 2K video generation up to 15 seconds → Instruction-based video editing → Motion transfer → Better character, camera & voice control The interesting part is that you can build a full scene from references instead of endlessly regenerating until the result feels right. 🔗 Now is a great time to try MiniMax H3 on Magnific, especially with the current 2K offer.
0:12

MiniMax H3 is a serious upgrade for AI video creation. Instead of relying on just text prompts, you can combine images, videos, and audio references to control the character, motion, camera, and sound. What stands out: → Up to 9 image + 3 video + 3 audio references → Native synced voice, music & sound effects → 2K video generation up to 15 seconds → Instruction-based video editing → Motion transfer → Better character, camera & voice control The interesting part is that you can build a full scene from references instead of endlessly regenerating until the result feels right. 🔗 Now is a great time to try MiniMax H3 on Magnific, especially with the current 2K offer.

Tanvir Anjum

32,271 Aufrufe • vor 1 Monat

AI video generation has quietly crossed a threshold. I got early access to Kling 4.0. Three things stood out: Dynamic Motions — complex camera movement and high-speed action that actually holds. No drift, no blur mess. Native 30s generation — a full 30-second clip, one shot, no stitching required. Omni Reference — up to 15 inputs at once: images, video clips, subjects, audio. Your characters and objects stay consistent across every scene. The clip below is straight out of the model. Same vehicle through every angle and speed
0:15

AI video generation has quietly crossed a threshold. I got early access to Kling 4.0. Three things stood out: Dynamic Motions — complex camera movement and high-speed action that actually holds. No drift, no blur mess. Native 30s generation — a full 30-second clip, one shot, no stitching required. Omni Reference — up to 15 inputs at once: images, video clips, subjects, audio. Your characters and objects stay consistent across every scene. The clip below is straight out of the model. Same vehicle through every angle and speed

Iris

19,016 Aufrufe • vor 11 Tagen

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.
0:19

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.

MrNeRF

21,391 Aufrufe • vor 1 Jahr

🚨 AI VIDEO PROMPTING IS DEAD !! You no longer need to write a 500-word prompt just to tell AI how your video should look. 💀 I just tested Pippit’s 3D Director Studio + Seedance 2.5 And instead of explaining the entire scene with words… I literally BUILT the scene first. For my test, I created a cinematic fight scene with characters inside a 3D environment. I could control: 1️⃣ Where the characters stood 2️⃣ Which props they held 3️⃣ Where the camera was placed 4️⃣ What each character should do Then I could actually CHOREOGRAPH the action. → Run → Fight → Shoot → Fall → React → Move You can place these actions on the timeline, adjust when they happen, and even plan the camera movement and character trajectories. So instead of: PROMPT → GENERATE → PRAY 😂 It becomes: BUILD → DIRECT → GENERATE And that gives you WAY more control. Because you’re not asking AI to imagine the entire scene from a block of text. You’re literally showing it the: → Characters → Environment → Props → Positions → Actions → Camera Then Seedance 2.5 takes that 3D direction and turns it into the final video. That’s what makes this so interesting. It feels less like prompting an AI video model… And more like sitting in the DIRECTOR’S CHAIR. 🎬 Try Pippit: Pippitofficial #3DDirectorStudio #PippitAI #Seedance25 #PippitPartner
0:14

🚨 AI VIDEO PROMPTING IS DEAD !! You no longer need to write a 500-word prompt just to tell AI how your video should look. 💀 I just tested Pippit’s 3D Director Studio + Seedance 2.5 And instead of explaining the entire scene with words… I literally BUILT the scene first. For my test, I created a cinematic fight scene with characters inside a 3D environment. I could control: 1️⃣ Where the characters stood 2️⃣ Which props they held 3️⃣ Where the camera was placed 4️⃣ What each character should do Then I could actually CHOREOGRAPH the action. → Run → Fight → Shoot → Fall → React → Move You can place these actions on the timeline, adjust when they happen, and even plan the camera movement and character trajectories. So instead of: PROMPT → GENERATE → PRAY 😂 It becomes: BUILD → DIRECT → GENERATE And that gives you WAY more control. Because you’re not asking AI to imagine the entire scene from a block of text. You’re literally showing it the: → Characters → Environment → Props → Positions → Actions → Camera Then Seedance 2.5 takes that 3D direction and turns it into the final video. That’s what makes this so interesting. It feels less like prompting an AI video model… And more like sitting in the DIRECTOR’S CHAIR. 🎬 Try Pippit: Pippitofficial #3DDirectorStudio #PippitAI #Seedance25 #PippitPartner

SANI BULA

95,462 Aufrufe • vor 1 Monat

🔥 VIDU Multi-Entity Consistency Give Vidu 2/3 images and it’ll turn them into a video—it’s pure magic! ✨ Your own characters interacting with objects and in the exact environment you want! Ads, movies… endless possibilities, and this is just the beginning! Thanks @Viduforhuman The future is a carrot! 🥕 Plus, how about grabbing any frame from a Vidu-generated video "from scratch" and throwing it into another AI video or image tool to push your project even further? For now, check out the comment below: I scaled up a frame with Magnific.ai and fed it into Runway to create a dynamic shot using full camera control. But fingers crossed I can soon use #ReCapture by Bisho & team to generate new shots from the same video!
0:13

🔥 VIDU Multi-Entity Consistency Give Vidu 2/3 images and it’ll turn them into a video—it’s pure magic! ✨ Your own characters interacting with objects and in the exact environment you want! Ads, movies… endless possibilities, and this is just the beginning! Thanks @Viduforhuman The future is a carrot! 🥕 Plus, how about grabbing any frame from a Vidu-generated video "from scratch" and throwing it into another AI video or image tool to push your project even further? For now, check out the comment below: I scaled up a frame with Magnific.ai and fed it into Runway to create a dynamic shot using full camera control. But fingers crossed I can soon use #ReCapture by Bisho & team to generate new shots from the same video!

Hungry Donkey 🥕

37,561 Aufrufe • vor 1 Jahr

There’s a weird paradox in AI video right now: The more ambitious the shot, the less you want the model improvising everything. For a simple clip, “describe → generate” can be enough. But for a complex scene, it helps to define the world first. That’s the idea behind the GPT-6 Astra workflow I’ve been testing: build the 3D scene, establish spatial relationships, and define the camera and character paths in the Viewport. Then, manually bring everything into CapCut PC. Seedance 2.5 handles the video rendering, while CapCut PC takes the result through post-production. Spatial control before generation. Creative control after it. #CapCutPC #GPT6Astra #AIVideo #Seedance25
0:12

There’s a weird paradox in AI video right now: The more ambitious the shot, the less you want the model improvising everything. For a simple clip, “describe → generate” can be enough. But for a complex scene, it helps to define the world first. That’s the idea behind the GPT-6 Astra workflow I’ve been testing: build the 3D scene, establish spatial relationships, and define the camera and character paths in the Viewport. Then, manually bring everything into CapCut PC. Seedance 2.5 handles the video rendering, while CapCut PC takes the result through post-production. Spatial control before generation. Creative control after it. #CapCutPC #GPT6Astra #AIVideo #Seedance25

Fathela Esq

43,328 Aufrufe • vor 1 Monat

📖THE BEST SEEDANCE 2.0 WORKFLOW STARTS INSIDE CHATGPT IMAGE 2 One creator can now go from storyboard to cinematic short film without a camera, actors, or a production crew Most creators use Seedance 2.0 as a video generator. The real power comes from combining ChatGPT Image 2 + Seedance 2.0 into a production pipeline. Here’s the workflow: 1.Create a story idea in ChatGPT. 2.Generate a shot-by-shot storyboard. 3. Build character sheets to lock consistency. 4.Generate every scene in ChatGPT Image 2. 5.Define camera movements and actions. 6.Animate each shot in Seedance 2.0. 7.Stitch the clips together into a finished short film. Why this works: • Consistent characters across scenes • Better storytelling • Precise camera control • Faster iteration • Professional-looking results • One person can do the work of an entire production team Use cases: ⁃Viral AI shorts ⁃YouTube animations ⁃Brand commercials ⁃Educational content ⁃Story-driven ads ⁃Social media content The future of AI video isn’t prompting. It’s building production pipelines. 📥Tomorrow I am sharing a workflow that almost nobody is using but absolutely should be. 🔖 The full breakdown from setup to final export is waiting in the pinned article.
0:15

📖THE BEST SEEDANCE 2.0 WORKFLOW STARTS INSIDE CHATGPT IMAGE 2 One creator can now go from storyboard to cinematic short film without a camera, actors, or a production crew Most creators use Seedance 2.0 as a video generator. The real power comes from combining ChatGPT Image 2 + Seedance 2.0 into a production pipeline. Here’s the workflow: 1.Create a story idea in ChatGPT. 2.Generate a shot-by-shot storyboard. 3. Build character sheets to lock consistency. 4.Generate every scene in ChatGPT Image 2. 5.Define camera movements and actions. 6.Animate each shot in Seedance 2.0. 7.Stitch the clips together into a finished short film. Why this works: • Consistent characters across scenes • Better storytelling • Precise camera control • Faster iteration • Professional-looking results • One person can do the work of an entire production team Use cases: ⁃Viral AI shorts ⁃YouTube animations ⁃Brand commercials ⁃Educational content ⁃Story-driven ads ⁃Social media content The future of AI video isn’t prompting. It’s building production pipelines. 📥Tomorrow I am sharing a workflow that almost nobody is using but absolutely should be. 🔖 The full breakdown from setup to final export is waiting in the pinned article.

Zentrix⌚️

50,553 Aufrufe • vor 3 Monaten

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.
0:13

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

MrNeRF

27,428 Aufrufe • vor 2 Jahren

AI video is becoming less about generating a clip and more about controlling the entire scene. 🎬 MiniMax H3 is now available on Magnific, with a multimodal workflow that brings text, images, video and audio together. → Up to 9 image + 3 video + 3 audio references → 2K video up to 15 seconds → Native synced voice, music & sound effects → Motion transfer → Instruction-based video editing → Camera, character and voice control The interesting part is how much you can build from references. A moodboard, product, character, environment or motion reference can all become part of the same generation. And the sound is generated alongside the video, instead of being added afterward. MiniMax H3 on Magnific makes the whole process feel much closer to directing a scene than simply prompting a video.
0:12

AI video is becoming less about generating a clip and more about controlling the entire scene. 🎬 MiniMax H3 is now available on Magnific, with a multimodal workflow that brings text, images, video and audio together. → Up to 9 image + 3 video + 3 audio references → 2K video up to 15 seconds → Native synced voice, music & sound effects → Motion transfer → Instruction-based video editing → Camera, character and voice control The interesting part is how much you can build from references. A moodboard, product, character, environment or motion reference can all become part of the same generation. And the sound is generated alongside the video, instead of being added afterward. MiniMax H3 on Magnific makes the whole process feel much closer to directing a scene than simply prompting a video.

Mohini Shewale

63,995 Aufrufe • vor 1 Monat

Video diffusion models have strong implicit representations of 3D shape, material, and lighting, but controlling them with language is cumbersome, and control is critical for artists and animators. GenLit connects these implicit representations with a continuous 5D control signal describing the direction and intensity of a point light source. This enables single-image near-field relighting of an image using a video diffusion model. We use a ControlNet-like approach and show that, with a small amount of synthetic data, GenLit generalizes to complex real-world images. Given a single image and the 5D lighting signal, GenLit creates a video of a moving light source that is inside the scene. It moves around and behind scene objects, producing effects such as shading, cast shadows, secularities, and interreflections with a realism that is hard to obtain with traditional inverse rendering methods. GenLit shows that it is possible to get continuous control over implicit physical processes within a video model. I think this is just the beginning and promises to make such models much more practical for creators. Shrisha Bharadwaj will present today at SIGGRAPH Asia Room: S423/S424, Level 4 @ 13:50 on 15 of Dec.
0:13

Video diffusion models have strong implicit representations of 3D shape, material, and lighting, but controlling them with language is cumbersome, and control is critical for artists and animators. GenLit connects these implicit representations with a continuous 5D control signal describing the direction and intensity of a point light source. This enables single-image near-field relighting of an image using a video diffusion model. We use a ControlNet-like approach and show that, with a small amount of synthetic data, GenLit generalizes to complex real-world images. Given a single image and the 5D lighting signal, GenLit creates a video of a moving light source that is inside the scene. It moves around and behind scene objects, producing effects such as shading, cast shadows, secularities, and interreflections with a realism that is hard to obtain with traditional inverse rendering methods. GenLit shows that it is possible to get continuous control over implicit physical processes within a video model. I think this is just the beginning and promises to make such models much more practical for creators. Shrisha Bharadwaj will present today at SIGGRAPH Asia Room: S423/S424, Level 4 @ 13:50 on 15 of Dec.

Michael Black

22,353 Aufrufe • vor 9 Monaten

Break-A-Scene: Extracting Multiple Concepts from a Single Image introduce the task of textual scene decomposition: given a single image of a scene that may contain several concepts, we aim to extract a distinct text token for each concept, enabling fine-grained control over the generated scenes. To this end, we propose augmenting the input image with masks that indicate the presence of target concepts. These masks can be provided by the user or generated automatically by a pre-trained segmentation model. We then present a novel two-phase customization process that optimizes a set of dedicated textual embeddings (handles), as well as the model weights, striking a delicate balance between accurately capturing the concepts and avoiding overfitting. We employ a masked diffusion loss to enable handles to generate their assigned concepts, complemented by a novel loss on cross-attention maps to prevent entanglement. We also introduce union-sampling, a training strategy aimed to improve the ability of combining multiple concepts in generated images. We use several automatic metrics to quantitatively compare our method against several baselines, and further affirm the results using a user study. Finally, we showcase several applications of our method paper page:
0:12

Break-A-Scene: Extracting Multiple Concepts from a Single Image introduce the task of textual scene decomposition: given a single image of a scene that may contain several concepts, we aim to extract a distinct text token for each concept, enabling fine-grained control over the generated scenes. To this end, we propose augmenting the input image with masks that indicate the presence of target concepts. These masks can be provided by the user or generated automatically by a pre-trained segmentation model. We then present a novel two-phase customization process that optimizes a set of dedicated textual embeddings (handles), as well as the model weights, striking a delicate balance between accurately capturing the concepts and avoiding overfitting. We employ a masked diffusion loss to enable handles to generate their assigned concepts, complemented by a novel loss on cross-attention maps to prevent entanglement. We also introduce union-sampling, a training strategy aimed to improve the ability of combining multiple concepts in generated images. We use several automatic metrics to quantitatively compare our method against several baselines, and further affirm the results using a user study. Finally, we showcase several applications of our method paper page:

AK

154,511 Aufrufe • vor 3 Jahren

1/ We've all been aware of the hype surrounding Dreamina Seedance 2.0, and I finally got early access to the tool. And wow, I'm blown away. This is by far the best. It is starting to make AI video feel less like "generate a clip" and more like "direct a scene." What stood out to me is the level of control: camera motion, pacing, visual consistency, and the ability to build from multiple references inside one workflow. Some of the prompt directions that feel especially strong: - a busy modern city square during daytime. Suddenly, time freezes completely - a single continuous camera movement through a natural landscape that transitions through all four seasons in one shot - an underwater bioluminescent city waking up at dawn The big shift is this: One Prompt, Viral Remade. Edit Videos as Easy as Editing Photos. Dreamina Seedance 2.0 feels like a real step toward AI-native directing rather than just AI generation. Here are some examples 🧵:
0:15

1/ We've all been aware of the hype surrounding Dreamina Seedance 2.0, and I finally got early access to the tool. And wow, I'm blown away. This is by far the best. It is starting to make AI video feel less like "generate a clip" and more like "direct a scene." What stood out to me is the level of control: camera motion, pacing, visual consistency, and the ability to build from multiple references inside one workflow. Some of the prompt directions that feel especially strong: - a busy modern city square during daytime. Suddenly, time freezes completely - a single continuous camera movement through a natural landscape that transitions through all four seasons in one shot - an underwater bioluminescent city waking up at dawn The big shift is this: One Prompt, Viral Remade. Edit Videos as Easy as Editing Photos. Dreamina Seedance 2.0 feels like a real step toward AI-native directing rather than just AI generation. Here are some examples 🧵:

Chubby♨️

76,702 Aufrufe • vor 6 Monaten

Honestly... this makes a lot of sense. I've lost count of how many times I've regenerated an AI video just because someone was standing in the wrong place or the camera angle felt off. If I can quickly lay out the scene first and then let the model generate from that, I'd much rather work that way than keep tweaking prompts over and over. Seems like a really useful addition for anyone making AI videos. Props to TopviewAI for building this.
0:17

Honestly... this makes a lot of sense. I've lost count of how many times I've regenerated an AI video just because someone was standing in the wrong place or the camera angle felt off. If I can quickly lay out the scene first and then let the model generate from that, I'd much rather work that way than keep tweaking prompts over and over. Seems like a really useful addition for anyone making AI videos. Props to TopviewAI for building this.

Gina Acosta

103,147 Aufrufe • vor 3 Monaten

I’ve used all the recent GenAI video models extensively & here’s my 2¢: 🎬 Runway Gen3 Alpha - best image quality & motion for text-to-video & embedded words. Great at prompt travel changes over the course of 10 sec. And I’m super bullish on how gen3 will evolve, hopefully adopting the features listed below. Kling - best quality for image-to-video with prompt control, like eating food. Great clip extension that accounts for character (ie walking stride) & camera movement (speed & angle), rather than just using final frame. But it’s limited availability & Chinese native language is limiting. Used for Spider-Man video below (via Midjourney). LumaLabs - best for keyframe start & end control (it can not be overstated how important this is. other services should add it ASAP!) and their high dynamic action movements are really fun. Luma was used in my viral Multiverse of Memes video. PikaLabs - they haven’t gotten as much attention as others lately. But they did update their video model a few weeks ago and it looks great. Also, they are notable for their unique & AWESOME features, like video in-painting & out-painting. My perfect AI video platform would have the following features: 1) Gen3’s quality, prompt control & text embedding. 2) KLing’s image-to-video quality, prompt control & clip extension quality. 3) Luma’s multi-keyframe control & dynamic movement ability. 4) Pika’s inpainting & outpainting ability. And a video-to-video (aka next-gen Runway gen1) could be a game changer, too. It’s an exciting time to be alive 🫶 Who will get there first? 🔉🔉
0:16

I’ve used all the recent GenAI video models extensively & here’s my 2¢: 🎬 Runway Gen3 Alpha - best image quality & motion for text-to-video & embedded words. Great at prompt travel changes over the course of 10 sec. And I’m super bullish on how gen3 will evolve, hopefully adopting the features listed below. Kling - best quality for image-to-video with prompt control, like eating food. Great clip extension that accounts for character (ie walking stride) & camera movement (speed & angle), rather than just using final frame. But it’s limited availability & Chinese native language is limiting. Used for Spider-Man video below (via Midjourney). LumaLabs - best for keyframe start & end control (it can not be overstated how important this is. other services should add it ASAP!) and their high dynamic action movements are really fun. Luma was used in my viral Multiverse of Memes video. PikaLabs - they haven’t gotten as much attention as others lately. But they did update their video model a few weeks ago and it looks great. Also, they are notable for their unique & AWESOME features, like video in-painting & out-painting. My perfect AI video platform would have the following features: 1) Gen3’s quality, prompt control & text embedding. 2) KLing’s image-to-video quality, prompt control & clip extension quality. 3) Luma’s multi-keyframe control & dynamic movement ability. 4) Pika’s inpainting & outpainting ability. And a video-to-video (aka next-gen Runway gen1) could be a game changer, too. It’s an exciting time to be alive 🫶 Who will get there first? 🔉🔉

Blaine Brown

26,535 Aufrufe • vor 2 Jahren

3D Gaussian Splatting for Real-Time Radiance Field Rendering paper page: Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.
0:18

3D Gaussian Splatting for Real-Time Radiance Field Rendering paper page: Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (>= 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.

AK

634,252 Aufrufe • vor 3 Jahren