Introducing ⇆ Marigold-DC — our training-free zero-shot approach to... monocular Depth Completion with guided diffusion! If you have ever wondered how else a long denoising diffusion schedule can be useful, we have an answer for you! Details 🧵show more

DimensionX: Create Any 3D and 4D Scenes from a... Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.show more

Can you make a jigsaw puzzle with two different... solutions? Or an image that changes appearance when flipped? We can do that, and a lot more, by using diffusion models to generate optical illusions! Continue reading for more illusions and method details 🧵show more

Humans are social creatures and physical interaction with other... humans plays an important role in our everyday life. In this thread, we explain how such interactions can be reconstructed in 3D from a single image utilising a denoising diffusion probabilistic model. (1/n)show more

Chop the gradients ✂️! We found that truncating decoder... gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.show more

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page:... Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.show more

🧵1/5 Many of you have been asking for "the... metro" details... We don’t want to spoil everything but for now all we can say is collection supply will be 2024. 1024 for community, 1024 for other on-chain/art focused communities. It will be free to mint!show more

You asked (and asked, and asked… and asked) and... finally, we have an answer: for the first time ever, we designed an entire Jet Lag format that you can buy and play with friends, anywhere in the world:show more

👤: would it be accurate to say you have... zero desire to be a solo popstar? 🐯🐻: if i have zero desire, shouldn’t i stop being an artist? TAEHYUNG YOURE SO ICONIC FOR THIS RESPONSEshow more

🐈‍⬛ : how about... we make a promise to... be together forever? do you guys have a confident to keep that promise? "YESSSS" 🐈‍⬛ : then, i'll believe in you guys, and lets be together forever 🦊 : if you stay with us, we can be forevershow more

"The disease that creeps into your program is called... entitlement. If you can stomp it out with leadership, you can stay hungry. We have a saying around our place: we eat off the floor. If you're willing to eat off the floor, you can be special."show more

“Are you alone, Mr. Leech? I know how that... feels. We can be best friends, and we never have to be alone ever again” just end it all for meshow more

World modeling and imitation learning have largely been considered... two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -show more

"How long should I rest between sets?" Until you... can hit the same weight for the same reps with the same quality. That is the answer. That has always been the answer. The answer is not 60 seconds. The answer is not 90 seconds. The answer is not whatever the timer on your watch app says. If you are doing heavy compounds, that might be 3 to 5 minutes. Sometimes longer. The exercise does not care about your podcast schedule. It needs the central nervous system to be ready and the muscles to be loaded again. Rushing rest periods is one of the great unforced errors in training. You sacrifice load, sacrifice reps, sacrifice tension, and convince yourself you have done a hard session because you sweated. You have done a cardio session with weights. They are not the same. Sit down. Breathe. Let the heart settle. Then lift the next set heavy enough to justify the wait.show more

How can we more effectively leverage robot data from... different embodiments for skill transfer? Excited to share that our new work, RoVi-Aug, has been accepted to Conference on Robot Learning as an oral paper！ WIth RoVi-Aug, you can augment an existing robot dataset into a different robot and different viewpoints. A policy trained on the augmented dataset can zero-shot deploy on the unseen target robot with significantly different camera angles! 🧵👇 🔗 Check out our paper:show more

You can't walk on our footpaths or drive on... our roads ! We have a lot to be grateful for to our Govt Every day is an adventure !show more

Thank you from Caleb for everything you all have... done for our community. With all your help the last three years, we have gathered almost 40,000 pounds of food during the Holidays, with more to be counted. You can still donate here:show more

May God not put our food in a hard... place. If you don’t have to sell your soul for crumbs like this before you and your family can eat, you have no idea how God has been kind to you.show more