Check out MultiDiff #CVPR2024! From a single RGB image,... MultiDiff enables scene-level novel view synthesis with free camera control. Great work by @normanisation Katja Schwarz Barbara Roessle, L Porzi, S Rota Bulò, P Kontschiedershow more

Wonderland: Navigating 3D Scenes from a Single Image Contributions:... • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.show more

We're releasing OVIE, a novel view generation model trained... entirely on single images. No multi-view datasets needed. Given a single image, it generates novel views of any scene in real time, running orders of magnitude faster than competing approaches.show more

Bai et al., "Positional Encoding Field" Make your RoPE... encoding 3D by including a z axis, then manipulate your image by simply manipulating your positional encoding in 3D --> novel view synthesis. Neat idea.show more

Accepted at #ECCV2024 🇮🇹 See you in Milan! PolyOculus:... Simultaneous Multi-view Image-based Novel View Synthesis Jason Yu Tristan Aumentado-Armstrong Fereshteh Forghani Marcus Brubaker and me Paper:show more

Pixel3DMM is a new method that can reconstruct 3D... human faces from a single RGB image or multiple framesshow more

RIP Ken Burns effect? Post camera moves will never... be the same. Convert and image or video into a 3D scene and animate a virtual camera to create a new move. Camera control by Moonvalleyshow more

Super excited to announce our new work: Synthesizing Moving... People with 3D Control (3DHM)💡 Why is 3DHM unique? With 3D Control, 3DHM can animate a 𝗿𝗮𝗻𝗱𝗼𝗺 human photo with 𝗮𝗻𝘆 poses in a 𝟯𝟲𝟬-𝗱𝗲𝗴𝗿𝗲𝗲 camera view and 𝗮𝗻𝘆 camera azimuths from 𝗮𝗻𝘆 video!show more

Create anime from a single image. With Gen-4 and... ChatGPT, you can turn any image into a multi-scene cartoon. Here’s how 👇show more

SplatVoxel: History-Aware Novel View Streaming without Temporal Training Contributions:... • We propose a hybrid Splat-Voxel feed-forward reconstruction framework that leverages historical information to enable novel view streaming, without relying on multi-view video datasets for training. • We develop an efficient sparse voxel transformer with a coarse-to-fine voxel representation, outperforming existing feed-forward Gaussian splatting methods. • Experiment results demonstrate that our proposed framework enhances novel view synthesis for streaming scene reconstruction, providing better visual quality and reduced temporal artifacts through history-aware modeling.show more

AI is so trippy. This entire scene, including the... exterior, was created from a single image of the ovens in the interior. This is a multi-view reconstruction, not a video. No v2v used. Prompt image in reply.show more

(1/3) Can we turn text-to-image models into photorealistic 3D... generators? ViewDiff (#CVPR2024) produces realistic, multi-view consistent images of real-world 3D objects in authentic surroundings. Website Video How does it work?show more

DimensionX: Create Any 3D and 4D Scenes from a... Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.show more

WildGS-SLAM: Monocular Gaussian Splatting SLAM in Dynamic Environments TL;DR:... We present WildGS-SLAM, a robust monocular RGB SLAM system. - Utilizes uncertainty-aware tracking and mapping to handle dynamic scenes. - Leverages DINOv2-based uncertainty maps for dynamic object removal. - Improves tracking and mapping. - Enables high-quality view synthesis.show more

Here’s how to create this cool 3D scene from... a single image! 📸 Midjourney (isometric image generation) Trellis (Image to 3D Gaussian Splat) Browser Lab (3D Editor Splat Import)show more

This AI 3D tool is on another level. A... dragon head and a dungeon NPC. Both from a single image. Here is how:show more

The camera motion in Imagine is insanely next-level...woah! I've... been exploring a few prompts with the token 'image sequence'show more

Seedance 2.0 is insane. It created a full anime... edit from a single prompt. Camera work, timing, effects, impact — everything feels finished, like a professional edit. The level this AI has reached is unreal.show more

U P W A R D S P I... R A Lshow more

Check out this view of the Bank from Saturday’s... flyover by the MD National Guard!show more

R A W S A L E R T... S A better view.show more