Introducing 🛹 RollingDepth 🛹 — a universal monocular depth... estimator for arbitrarily long videos! Our paper, “Video Depth without Video Models,” delivers exactly that, setting new standards in temporal consistency. Check out more details in the thread 🧵show more

Depth Any Video with Scalable Synthetic Data AI physicists... and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.show more

🤔Applying Depth Estimation models directly to videos can result... in inconsistency between frames. 💪Well, not anymore. 🔥ChronoDepth is a new approach to video depth estimation that focuses on achieving both accuracy within each frame and consistency across frames. 📜More👇show more

xAI absolutely cooked with the new Grok Imagine model!... The temporal consistency is rock-solid. The penguin's anatomy, lighting, and proportions stay locked in for the full 10 seconds with no weird morphing or melting. In less than 30 seconds, Grok created a video with snow spray building realistically, motion blur, and depth of field. The physics simulation rivals other top models.show more

🤯 Depth videos perfectly solve false flags on reference... videos — and can perfectly recreate both martial arts and dance! Turned a Guan Dao martial arts clip into a Depth motion reference, swapped the fighter for a market auntie, and moved the scene to a T-junction inside a local market 🤣 🌟 Workflow: 1. Pick a reference clip under 15 seconds 2. Convert it to Depth with Depth Anything V2 3. Generate a new character + scene 4. Feed everything into Seedance with the prompt below 🌟 Depth video conversion: You can build a local Depth converter with Codex — prompt in the comments. This goes way beyond trending dances. Martial arts, polearms, and other complex movements can all be transferred cleanly with Depth! Workflow + Prompt below 👇show more

For the facial animations in my game, I use... a technique that yields results similar to L.A. Noire. To keep it simple: I first generate a facial animation video using AI like LTX 2. Then, I create a video depth map from that animation and project it onto a face mask. Using vertex displacement, the depth map dynamically deforms the face as the character speaks. The result looks good in standalone VR. I actually created this workflow long time ago:show more

🚀New paper out - We present Video-MSG (Multimodal Sketch... Guidance), a novel planning-based training-free guidance method for T2V models, improving control of spatial layout and object trajectories. 🔧 Key idea: • Generate a Video Sketch — a spatio-temporal plan with background, foreground, and motion in the pixel space. • Encode this structure directly into the latent space of the diffusion model during generation, which does not require fine-tuning or additional memory during inference. 🧵show more

Return to Disney's The Lion King Realm in our... next free update - filled with 2 new friends and even more surprises. Check back tomorrow for another Developer Update Video for all the details… and maybe a launch date 👀show more

🔬 Built with an entirely new model architecture, our... diffusion-based approach uses 6B+ parameters and leverages the latest NVIDIA hardware. This is the most dynamic and wide-ranging video enhancing method we’ve ever created, setting a new standard for AI video restoration. Videos degrade due to compression artifacts, blurring, aliasing, noise, atmospheric distortion, missing pixels, etc. Each frame suffers from unique types of corruption, making AI video restoration a highly challenging task. Our technology solves this complexity by analyzing hundreds of frames to accurately restore details, delivering unmatched detail recovery combined with unparalleled temporal consistency.show more

A sneak peek of a new feature in the... #GeoAI Python package! 🎉 Now you can detect cars from georeferenced aerial imagery using deep learning—all with just a few lines of code. Stay tuned for an in-depth video tutorial coming soon! 🛠️ Explore the GitHub repository: 📚 Dive into the documentation: 📺 Check out the entire YouTube playlist:show more

🔥 Win a share of $50,000 in MORPHO rewards... 🔥 We’re celebrating the launch of Stablecoin Earn in collaboration with Morpho Labs. To enter: 🔹 Like & RT 🔹👇 Deposit USDT, USDC, or USDA into Morpho vaults via Stablecoin Earn Check out the thread for more details 🧵👇show more

Diffuman4D: 4D Consistent Human View Synthesis from Sparse-View Videos... with Spatio-Temporal Diffusion Models Contributions: • We introduce Diffuman4D, a novel diffusion model that generates spatio-temporally consistent and high-resolution (1024p) human videos from sparse-view video inputs. • We propose a sliding iterative denoising mechanism that enhances both the spatial and temporal consistency of generated long-term videos while maintaining efficient inference. • We design a human pose conditioning scheme to enhance the appearance quality and motion accuracy of generated human videos. • We plan to release our processed version of the DNA-Rendering dataset, which we believe will benefit future research in this area.show more

#ThrowbackThursday In their review, Jesse Cox went all out... for Control, a game that he labels as his Game of the Year. His 35-minute video goes IN DEPTH and is packed with nerdy references and features the Director herself. 📽️ Watch atshow more

Some updates on the multiview vistadream pipeline with Rerun!... Rerun came in extremely useful here, as being able to visualize depths at each stage of the pipeline allowed me to debug some nasty bugs. Since the last time, I was only working with a single image input. I've added in VGGT as my multiview pose + depth estimator. It works REALLY well for getting camera poses, but the depths are not that great. To try and fix that, I estimated depth maps from MoGeV2 for each of the views, and scale+shift aligned them so that they would match up to the confident sections of VGGT's depth predictions. You can see in the video just how much sharper the visualized 2d depth maps are! The biggest issue continues to be the multiview consistency 🫠 That's up next, along with actually training the Gaussian splat. Lots of work went into actually understanding inputs+outputs for VGGT. I had some funky bugs where the confidence values would all collapse to true I'm also really excited for this pipeline to use Difix3D+ Nvidia instead of Flux Inpainting, it seems like a better suited for a multiview pipeline.show more

#Kwiky is a NEW app, for hot content with... No Restrictions! ✨ #BabbieBia Naughty, short-form content that just works! Check out the link in our profile, for more!show more

Nano is a depth-aware atmospheric haze plugin that uses... ML depth estimation to add physically accurate fog and light scattering to your footage. Works *best* on log footage with visible light sources - it analyzes scene highlights then creates airlight (atmospheric scatter) and halation (light bloom) that responds to actual depth in the scene. Pretty clever approach to getting that cinematic haze look without having to pump a fog machine on set. Makes the OG Trapcode Shine look extremely dated (basically 2D light streaks masked by luminance values), and is yet way more controllable than the current crop of generative AI video-to-video tools.show more

Bring your videos to life with Akool’s Effect Enhancer.... ✨ Turn ordinary footage into dynamic, eye-catching content in seconds. With smarter visual effects and enhanced motion details, you can instantly elevate your videos without complex editing. Whether you're creating marketing content, storytelling videos, or social media clips, Effect Enhancer helps you add more depth, energy, and polish to every frame. Create smoother visuals. Add stronger impact. Make every video stand out with Akool.show more

These new AI models are so good! Dreamina AI... dropped Image 3.0 and Video 3.0 and now you'll have: · no more morphing · control over the camera · everything's into place · smart image reference Prompts, comparisons and a quick how-to guide in this thread 🧵👇show more

Can MLLMs actually track what's happening in a video?... Introducing VSTAT 🎯, our new benchmark for visual state tracking. The tasks are simple: count cups, read typed words, count page flips. Humans solve them easily. MLLMs don't. 🧵 [1/11]show more

Exciting Announcement! 📢 #PonderONE x #RealityMetaverse🤝 We’re excited to... announce our strategic partnership with Reality Metaverse, together we’ll be participating in AMAs, Cross-community Airdrops & more in depth integrations later on in our roadmap. Reality Metaverse revolutionizes gaming by allowing #NFT owners to monetize their assets in traditional games for the first time. By seamlessly connecting the real world with the Metaverse, it unlocks new opportunities in popular games like Landlord GO, setting a new standard for integrating digital assets with gaming experiences. We're delighted to have them in the #PonderEcosystem!show more

A preview of what's next, visualized with Rerun and... PlayCanvas supersplat ✨ (Also, feel free to send me a DM 📩; I’ll be in San Francisco from July 21–29, and I'm looking to meet like-minded folks!) I'm convinced that Gaussian Splats will be an integral part of any data engine as an underlying representation. So I've started putting together a repo that: 1. Given a single image, perform image outpainting 🖼️🖌️ 2. Estimate a monocular depth map on the outpainted image 📏 3. Train a Gaussian Splat initialized from the monocular depth 🎓✨ 4. Warp to new views, perform inpainting on the missing masks -> Train new splat 🔄🎨 This is going to be integrated into exo-egoforge, but I wanted to start with the simple single-image version before moving to a multi-video implementation There's some weirdness in the final rerun visualization, but the trained splat looks great 🎉! This is all based on the very cool VistaDream paper ( .github.io/) More on this next week!show more