Accepted at #ECCV2024 🇮🇹 See you in Milan! PolyOculus:... Simultaneous Multi-view Image-based Novel View Synthesis Jason Yu Tristan Aumentado-Armstrong Fereshteh Forghani Marcus Brubaker and me Paper:show more

We're releasing OVIE, a novel view generation model trained... entirely on single images. No multi-view datasets needed. Given a single image, it generates novel views of any scene in real time, running orders of magnitude faster than competing approaches.show more

Bai et al., "Positional Encoding Field" Make your RoPE... encoding 3D by including a z axis, then manipulate your image by simply manipulating your positional encoding in 3D --> novel view synthesis. Neat idea.show more

SplatVoxel: History-Aware Novel View Streaming without Temporal Training Contributions:... • We propose a hybrid Splat-Voxel feed-forward reconstruction framework that leverages historical information to enable novel view streaming, without relying on multi-view video datasets for training. • We develop an efficient sparse voxel transformer with a coarse-to-fine voxel representation, outperforming existing feed-forward Gaussian splatting methods. • Experiment results demonstrate that our proposed framework enhances novel view synthesis for streaming scene reconstruction, providing better visual quality and reduced temporal artifacts through history-aware modeling.show more

We introduce TurboEdit -- simple text-based image editing in... 1/2 sec! We leverage few-step diffusion models, mapping real images into noise with an encoder. Please see our #ECCV2024 paper: Work w/ Zongze Wu, Nick Kolkin, Jon Brandt, Eli Shechtman 1/show more

Check out MultiDiff #CVPR2024! From a single RGB image,... MultiDiff enables scene-level novel view synthesis with free camera control. Great work by @normanisation Katja Schwarz Barbara Roessle, L Porzi, S Rota Bulò, P Kontschiedershow more

Adaptive and Temporally Consistent Gaussian Surfels for Multi-view Dynamic... Reconstruction Contributions: • A method for efficiently reconstructing dynamic surfaces from multi-view videos using Gaussian surfels. • A unified and gradient-aware densification strategy for optimizing dynamic 3D Gaussians with fine details. • A temporal consistency approach that ensures stable and coherent surface reconstructions across frames by enforcing consistency on curvature maps. • Extensive experiments that demonstrate our method’s advantages including fast training, high-fidelity novel view synthesis, and accurate surface geometry.show more

We are happy to announce that ComfyUI now natively... supports toad Hunyuan3D 2.0 and Hunyuan3D 2.0 MV (Multi-View) model series! 3 workflows to get started: 🔹Hunyuan3D-2 mv: multi-view to 3D 🔹Hunyuan3D-2 mv Turbo: accelerated multi-view 🔹Hunyuan3D-2: single image to 3Dshow more

whippin da benz me and bro ain’t got no... planz you see me in traffic we movin real fast you can see us in your rear view lensshow more

SpaceX Multi-cam view.

Multi-step synthesis in action 🎬

The Stable Video Diffusion model just dropped 🔥 The... new model supports: – Text-to-Video – Image-to-Video – 14 or 25 frames at 576 x 1024 – Multi-View Generation – Frame Interpolation – 3D Scene Understanding – Camera Control via LoRA Paper: Code: SVD model: SVD-XT model:show more

Introducing Multi View ✨ Drop in any image and... it generates multiple shots of it - close-ups, mood shots, lifestyle scenes, alternate perspectives. Every direction your image could go, all from a single input. Now live on OpenArt.show more

You see that view

AI is so trippy. This entire scene, including the... exterior, was created from a single image of the ovens in the interior. This is a multi-view reconstruction, not a video. No v2v used. Prompt image in reply.show more

The annual toilet paper game between William Penn and... Grand View.show more

Introducing VR-NeRF (SIGGRAPH Asia ➡️ Kuala Lumpur, Malaysia 2023):... 🤩 multi-camera rig for multi-view HDR capture 🤓 perceptual HDR optimization + level of detail 😎 real-time multi-GPU VR rendering Project: Paper: Dataset:show more

Wonderland: Navigating 3D Scenes from a Single Image Contributions:... • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.show more

Are you looking at me or the view? 😌✨