Text-To-4D Dynamic Scene Generation Presents MAV3D (Make-A-Video3D), a method... for generating three-dimensional dynamic scenes from text descriptions. proj: abs:show more

Introducing Make-A-Video3D! Generating 3D dynamic (mini) scenes from input... text. That is, text --> 4D! Needs no 4D data (i.e., no dynamic 3D data), no static 3D data, no paired text-video data. Paper: Website:show more

✨ Any static 3D assets ➡️ 4D dynamic worlds.... Introducing CHORD, a universal framework for generating scene-level 4D dynamic motion from any static 3D inputs. It generalizes surprisingly well across a wide range of objects 🤯 and can even be used to learn robotics manipulation policy 🤖! Project page: Dive deeper in a 🧵: 1/nshow more

📢🚗✨ Excited to announce InfiniCube, our scalable generative model... for dynamic 3D driving scene generation with high fidelity and controllability! InfiniCube generates very large-scale (300m×400m ~ 100,000m^2), dynamic 3D driving scenes given HD maps, 3D bounding boxes, and text prompts as controls. Website: Props to Yifan Lu for leading this project!!!! [1/N]show more

For dynamic, explosive text-to-video, Kling is still an absolute masterclass.

📢DNF: Generating 4D animations with dictionary-based neural fields! Xinyi... Zhang presents a new dictionary-based neural field for unconditional 4D generation of deforming shapes -- generating motions with high-quality shape and temporal consistency.show more

Google presents MobileDiffusion Subsecond Text-to-Image Generation on Mobile Devices... paper page: MobileDiffusion achieves a remarkable sub-second inference speed for generating a 512 × 512 image on mobile devices, establishing a new state of the art.show more

I am blown away 🤯. Check this out! CameraCtrl... II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models TL;DR: "To enable broader exploration of dynamic scenes, our model can generate new video clips of the same scene based on previously generated content and user-provided camera trajectories. This approach maintains dynamic capabilities, accurate camera control, and scene consistency throughout the extended exploration." "Our model enables precise camera control across diverse scenarios while preserving dynamic scene elements, e.g." "Our method can generate videos with strong 3D consistency, which enables high-quality 3D reconstruction using the camera-controlled videos." Contributions: 1) A systematic data curation pipeline for constructing a dynamic video dataset with camera trajectory annotations; 2) A lightweight camera control injection module and corresponding training strategy that preserves dynamic video generation capabilities while adding camera control effect; 3) A clip-wise autoregressive generation recipe that enables extended range exploration of generated scenes.show more

GSTAR: Gaussian Surface Tracking and Reconstruction Contributions: • A... new framework for tracking and reconstructing dynamic scenes, combining 3D Gaussians and meshes to effectively manage changes in topology. • A method for Gaussian unbinding and surface re-meshing, allowing for the generation of new surfaces as topologies evolve. • A method for handling large or fast deformations of surfaces between frames using scene flow warping. Abstract (excerpt): However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians.show more

Excited to share our #CoRL2024 paper on humanoid motion... generation! Combining human motion priors with VLM feedback, Harmon generates natural, expressive, and text-aligned humanoid motion from freeform text descriptions. 👇(1/4)show more

Interesting LTX2.3 Cameraman LoRA. Transfers camera motion from reference... vids to new scenes. text-prompted generation, no trigger words.show more

This is such a cool interactive website to learn... how LLMs work. It’s combines step-by-step text explanations with dynamic 3D visualizations for an immersive learning experience. Available to everyone for free!show more

Google presents LayoutVLM a model that can generate 3D... layouts from text instructionsshow more

⚡️ CSS ::highlight() + TreeWalker for dynamic text search:... - No DOM manipulation - Pure CSS ::highlight() styling - Efficient TreeWalker traversal [Code Snippet 👇]show more

Experimenting with dynamic annotations Normally it's difficult to annotate... text with a digital pen because the words can reflow- but what if your annotations could reflow with it?show more

MoVieS Motion-Aware 4D Dynamic View Synthesis in One Second

DimensionX: Create Any 3D and 4D Scenes from a... Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.show more

this looks insane, LLaDA app just dropped by apolinario... (poli) A text generation method different from the traditional left-to-right approach a diffusion model with an unprecedented 8B scale, trained entirely from scratch, rivaling LLaMA3 8B in performanceshow more

We're excited to introduce a new dataset, 4D-DRESS, at... #CVPR2024. It features diverse human subjects, realistic clothing deformations, and dynamic body motion in 4D. (1/6) 🧵👇show more