We discovered that imposing a spatio-temporal weight space via... LoRAs on DIT-based video models unlocks powerful customization! It captures dynamic concepts with precision and even enables composition of multiple videos together!🎥✨show more

Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos... with Spatio-Temporal Diffusion Models Contributions: • We introduce Diffuman4D, a novel diffusion model that generates spatio-temporally consistent and high-resolution (1024p) human videos from sparse-view video inputs. • We propose a sliding iterative denoising mechanism that enhances both the spatial and temporal consistency of generated long-term videos while maintaining efficient inference. • We design a human pose conditioning scheme to enhance the appearance quality and motion accuracy of generated human videos. • We plan to release our processed version of the DNA-Rendering dataset, which we believe will benefit future research in this area.show more

🚀New paper out - We present Video-MSG (Multimodal Sketch... Guidance), a novel planning-based training-free guidance method for T2V models, improving control of spatial layout and object trajectories. 🔧 Key idea: • Generate a Video Sketch — a spatio-temporal plan with background, foreground, and motion in the pixel space. • Encode this structure directly into the latent space of the diffusion model during generation, which does not require fine-tuning or additional memory during inference. 🧵show more

Wonderland: Navigating 3D Scenes from a Single Image Contributions:... • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.show more

A dramatic moment captures a space rock striking the... Moon’s surface, creating a powerful and sudden impact. The object hits at incredible speed, producing a brief flash and disturbing the lunar surface. With no atmosphere to slow it down, the Moon is constantly exposed to impacts from space rocks, which have shaped its cratered landscape over billions of years. Events like this are happening all the time, but capturing one clearly is rare. This moment gives a glimpse into the dynamic nature of space and how even the Moon continues to change. A brief impact, but a powerful reminder of cosmic activity. #nasa #space #spacexshow more

A dramatic moment captures a space rock striking the... Moon’s surface, creating a powerful and sudden impact. The object hits at incredible speed, producing a brief flash and disturbing the lunar surface. With no atmosphere to slow it down, the Moon is constantly exposed to impacts from space rocks, which have shaped its cratered landscape over billions of years. Events like this are happening all the time, but capturing one clearly is rare. This moment gives a glimpse into the dynamic nature of space and how even the Moon continues to change. A brief impact, but a powerful reminder of cosmic activity. #nasa #space #spacexshow more

Break-A-Scene: Extracting Multiple Concepts from a Single Image introduce... the task of textual scene decomposition: given a single image of a scene that may contain several concepts, we aim to extract a distinct text token for each concept, enabling fine-grained control over the generated scenes. To this end, we propose augmenting the input image with masks that indicate the presence of target concepts. These masks can be provided by the user or generated automatically by a pre-trained segmentation model. We then present a novel two-phase customization process that optimizes a set of dedicated textual embeddings (handles), as well as the model weights, striking a delicate balance between accurately capturing the concepts and avoiding overfitting. We employ a masked diffusion loss to enable handles to generate their assigned concepts, complemented by a novel loss on cross-attention maps to prevent entanglement. We also introduce union-sampling, a training strategy aimed to improve the ability of combining multiple concepts in generated images. We use several automatic metrics to quantitatively compare our method against several baselines, and further affirm the results using a user study. Finally, we showcase several applications of our method paper page:show more

Depth Any Video with Scalable Synthetic Data AI physicists... and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.show more

NVIDIA just released a very impressive text-to-video paper. Video... Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:show more

DimensionX: Create Any 3D and 4D Scenes from a... Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.show more

ImmerseGen: Agent-Guided Immersive World Generation with Alpha-Textured Proxies Contributions:... 1) We propose ImmerseGen, a novel agent-guided 3D environment generation framework. It uses simplified geometric proxies with alpha-textured meshes to produce compact, photorealistic worlds ready for real-time mobile VR rendering. 2) We propose a novel RGBA texturing paradigm. It first synthesizes 8K terrain textures using a geometry-conditioned panorama generator via user-centric mapping, and then directly generates alpha-textured proxy assets, avoiding fidelity loss typically resulting from mesh decimation. 3) To automate scene creation from user prompts, we introduce VLM-based modeling agents equipped with a novel grid-based semantic analysis. This enables 3D spatial reasoning from 2D observations and ensures accurate asset placement. ImmerseGen further enhances immersion with dynamic effects and ambient audio for a multisensory experience. 4) Experiments on multiple scene-generation scenarios and live mobile VR applications show that ImmerseGen outperforms previous methods in visual quality, realism, spatial coherence, and rendering efficiency for immersive real-time VR experiences.show more

There's a lot of both 2D and 3D effort... that we're putting into Star Birds together with our friends at Toukana Interactive! Here's a quick behind the scenes look at some of the assets, 3D models, birb shenanigans and cutscenes that you might have already seen in the Early Access version! 🚀✨ Have you already joined us on this space adventure? Don't miss out! Star Birds in Early Access is available on Steam:show more

Introducing Magic Orb🔮 Step into a new era of... control with Magic Orb, an advanced tool that empowers alchemists to fine-tune AI generation settings to meet their specific needs. Designed for precision and adaptability, it allows unparalleled customization of outputs to ensure every creation aligns with your vision. Looking ahead, updates to the Multi-AI system will unlock granular control over individual AI configurations. Users will soon be able to manage and tweak multiple specialized AI entities within their applications, each optimized for a distinct function. Now, that’s Magic!🪄✨show more

Anyone telling you AI can one-prompt motion videos is... bullshitting you Here's what it really takes to produce this kind of work: - Ideation: we have multiple people with years of editing and animation experience come up with ideas, pacing etc - Design: each animation gets designed by multiple professional product designers - Production: absolutely goated motion designers work together on it & art director is making sure that everything is hitting the quality standard One-prompt this, sureshow more

In collaboration with the Iowa Tribe of Oklahoma, we... officially announce that we will open our doors to Harrahs Oklahoma for the first time on April 9, 2026, bringing the first-ever Caesars-managed experience to the state of Oklahoma. Located along historic Route 66, Harrah’s Oklahoma will offer guests a Las Vegas-style gaming, dining and entertainment experience in the Sooner state. The destination will feature a dynamic casino floor with 12 table games, 1,000 slot machines, a high-limit gaming area, multiple restaurants and bars and plans for a future on-site hotel and expanded entertainment space.show more

We’re excited to introduce Text-to-LoRA: a Hypernetwork that generates... task-specific LLM adapters (LoRAs) based on a text description of the task. Catch our presentation at #ICML2025! Paper: Code: Biological systems are capable of rapid adaptation, given limited sensory cues. For example, our human visual system can quickly adapt and tune its light sensitivity to our surroundings. While modern LLMs exhibit a wide variety of capabilities and knowledge, they remain rigid when adding task-specific capabilities. Traditionally, customizing these models requires gathering large datasets and performing often expensive, time-consuming fine-tuning for specific applications. To bypass these limitations, Text-to-LoRA (T2L) meta-learns a “hypernetwork” that takes in a text description of a desired task, as a prompt, and generates a task-specific LoRA that performs well on the task. In our experiments, we show that T2L can encode hundreds of existing LoRA adapters. While the compression is lossy, T2L maintains the performance of task-specifically tuned LoRA adapters. We also show that T2L can even generalize to unseen tasks given a natural language description of the tasks. Importantly, Text-to-LoRA is parameter-efficient. It generates LoRAs in a single, inexpensive step, based solely on a simple text description of the task. This approach is a step towards dramatically lowering the technical and computational barriers, allowing non-technical users to specialize foundation models using plain language, rather than needing deep technical expertise or large compute resources.show more

Video diffusion models have strong implicit representations of 3D... shape, material, and lighting, but controlling them with language is cumbersome, and control is critical for artists and animators. GenLit connects these implicit representations with a continuous 5D control signal describing the direction and intensity of a point light source. This enables single-image near-field relighting of an image using a video diffusion model. We use a ControlNet-like approach and show that, with a small amount of synthetic data, GenLit generalizes to complex real-world images. Given a single image and the 5D lighting signal, GenLit creates a video of a moving light source that is inside the scene. It moves around and behind scene objects, producing effects such as shading, cast shadows, secularities, and interreflections with a realism that is hard to obtain with traditional inverse rendering methods. GenLit shows that it is possible to get continuous control over implicit physical processes within a video model. I think this is just the beginning and promises to make such models much more practical for creators. Shrisha Bharadwaj will present today at SIGGRAPH Asia Room: S423/S424, Level 4 @ 13:50 on 15 of Dec.show more

Dynamic workflows are a generalization of harnesses, automations, loops,... routing, and graphs. It's the most powerful feature I have built into my agent orchestrator. Supports all kinds of patterns that leverage different agent backends (claude, codex, pi, hermes,...). It's a meta-harness approach that unlocks new forms of test-time compute. Example of use cases it supports: > LLM councils to get different perspectives from LLMs or plan more intensively > Dynamically routing tasks to different agents based on needs (e.g., cost efficiency and optimal intelligence) > Advisor/Judge + executor workflows and pretty much any complex graph-based pattern required by the task. I find it especially useful for long-running work and code reviewing. > Agent teams that talk to each other if needed for the task. I like to use this for AI editing, artifact creation, and other creative tasks. And I am sure it supports so many things that I haven't discovered yet. I got inspired by the dynamic workflow feature released by the Claude Code team. I had actually built it earlier this year but wanted to generalize it across different agent backends. I think this is going to become more popular in the coming days. I will share more of my findings soon.show more

The Sabotaging Practice of Over Supply and Sameness in... the NFT Space. The current zeitgeist of the NFT space is that the same artists are doing the same kind of work five times a year, with project after project leaving a trail of disappointment and discontent among collectors and all of us watching in disbelief as huge resources are extracted from the space over work that feels like it could be left as an "artist study." I understand that you can do what you want with your money as collectors, but we are killing the whole space with this incestuous practice. No artist is that prolific to be able to do 5 collections of 100+ pieces each every year and actually deliver innovation and some kind of creative evolution. Of course, they can pretend play that the work has something new, but there is no precedent nor proof that that has ever happened in the speed that it happens in the NFT space. Again, people are free to through away their resources on whatever they want but with this way of doing things, we more and more are going to start seeing the consequences. Oh! There are consequences? Yes. Maybe unintended, but there are. Let's see. Let's start with the loss of belief in the NFT space as somewhere where emerging artists can come and find support for their experiments. Why even bother to bring experiments, innovation, and new ways to think of art on the blockchain if the same people have all the collectors hypnotized with their magical flutes? Why even try to come to a space where taking risks and challenging the status quo (the mission of art!!!) is overlooked? This makes the NFT space a social club and not a space for art. I guess it is fine, but IMO it is a recipe for disaster. New collectors stay away because the art will slowly but surely become stale and un-challenging. Why even bother to come and see what is happening here if you can't, as a collector, see new weird and up-and-coming artists? The amount of noise emitted by the same artists doing the same art over and over, drowns out any new voices. Again. A recipe for disaster. The NFT space is becoming a space of disappointment and doubt. We think that collections going to zero one after the other, over and over, is not damaging? I feel we are kidding ourselves. Disappointment piles up, and again, the people who will hurt are the emerging artists, the new blood, the ones who are willing to risk the most and, in return, put fire in this cold space of sameness. I love this space—don't get me wrong—it has changed my life, and I believe it has a ton of potential, but things need to change for it to become a beacon of light in art. But we need to support new voices. We need to support new ideas. The challenge is huge. I hope to contribute all I can to this change. I hope more and more see how exciting it is to go out and try to discover what else is out there and move this space forward. But again, I understand the leaps of faith needed, but if there is a space that is based on that, it's the NFT space...so there is hope. We will see. 📺by Boldtronshow more

The Mother of all Aquamarines🙌💎✨ This might be one... of the best Aquamarine on Matrix pieces ever discovered. A massive single Aquamarine crystal towers over a bed of white prismatic Feldspar crystals. The main Aquamarine has a perfect stepped termination and a brisk, glacier blue tone. Even the Aquamarines radiating outward at the base are stunning with amazing clarity and aesthetic placement. This cluster weighs around 70 lb (32 kg) and was found at the famous Goand Mine in Skardu, Pakistan, a place known for its Aquamarines but rarely something of this caliber. It’s a miracle with its size and composition that it was found in such pristine condition with no damage or repair. This truly is one of the best in the world that could be the centerpiece of any museum or top collection!💎✨ Aquamarine on Feldspar Goand Mine, Skardu, Pakistan The Terra Collectionshow more

Yeaah with Kartel.ai and Jesse Wellens ! So training... Lora for character on Flux and also for Wan 2.1 14B ( using dataset video ) make a Huge difference. So here you are seeing a volumetric capture of Jesse Wellens that we did in at the spatial studio of Kartel.ai We use a a World Labs gaussian splatting to create a simple environment that we put with the volumetric capture in our own webgl viewer ( out soon for everyone ). After that, We use after this output in a ComfyUI workflow with Wan Fun control ( a bit similar Vace but its working for Wan 14B) with the double loras , the first to generate the first frame with flux, and the second one to guide the generation of Wan 2.1 Control. So we keep very good consistency of character and position, and creating amazing worlds :) ! Hope you will find it cool !show more