Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

FlowRVS - segmentation as a continuous deformation, mapping video latents directly to masks via an ODE. Built on Wan’s T2V. - complex semantic understanding with temporal consistency. - no flickering

Wildminder

12,378 subscribers

26,643 views • 8 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

3 Comments

Try TLGen App | Brandon Starship 🇺🇸7 months ago

Does it work on cars? @grok?

Zanyi Wang7 months ago

So thrilled to see our work FlowRVS getting this kind of love! Massive thanks to @wildmindai and @Kijaidesign! Happy to answer any questions here.

Mindset Forge⚒️8 months ago

Leverage advanced tools. Consistent innovation fuels long-term vision. 💼📈

Related Videos

Tracking Anything with Decoupled Video Segmentation paper page: Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.

Tracking Anything with Decoupled Video Segmentation paper page: Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.

AK

305,683 views • 3 years ago

SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussians Contributions: • We propose SuperGSeg: a 3D segmentation method with neural Gaussians, designed to learn hierarchical instance segmentation features from 2D foundation models. • We introduce the concept of Super-Gaussian, a novel representation that integrates hierarchical instance segmentation features, enabling the embedding of high-dimensional language features. This approach addresses previously unfeasible challenges in representing complex scenes with rich semantic details. • Extensive experiments on the LERF-OVS and ScanNet datasets demonstrate the effectiveness of the proposed method, achieving significant improvements in open-vocabulary 3D object-level and scene-level semantic segmentation. It shows particular strength in capturing fine-grained scene details and dense pixel semantic segmentation tasks for the first time.

SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussians Contributions: • We propose SuperGSeg: a 3D segmentation method with neural Gaussians, designed to learn hierarchical instance segmentation features from 2D foundation models. • We introduce the concept of Super-Gaussian, a novel representation that integrates hierarchical instance segmentation features, enabling the embedding of high-dimensional language features. This approach addresses previously unfeasible challenges in representing complex scenes with rich semantic details. • Extensive experiments on the LERF-OVS and ScanNet datasets demonstrate the effectiveness of the proposed method, achieving significant improvements in open-vocabulary 3D object-level and scene-level semantic segmentation. It shows particular strength in capturing fine-grained scene details and dense pixel semantic segmentation tasks for the first time.

MrNeRF

13,594 views • 1 year ago

NVIDIA just dropped UniRelight, a handy video-relighting model - estimates albedo, relit video in a single pass - based on Cosmos-Predict1-7B - supports complex materials - temporal consistency - beats DiLightNet, NeuralGaffer

NVIDIA just dropped UniRelight, a handy video-relighting model - estimates albedo, relit video in a single pass - based on Cosmos-Predict1-7B - supports complex materials - temporal consistency - beats DiLightNet, NeuralGaffer

Wildminder

29,461 views • 6 months ago

Images, video, audio, actions — generative modeling has converged on one recipe: compress into continuous latents, generate in latent space, decode. Language is the lone exception: still generated token by token, as a long discrete stream. Latent Thought Flows: We compress 256 text tokens into 8 continuous latents, generate them with a one-step flow model, and read them out as text using an autoregressive decoder. Result: a better inference-compute vs. generation-quality Pareto frontier than a tuned autoregressive baseline — thanks to compression and one-step generation. Co-led w/ Zhengyang Geng 🧵

Images, video, audio, actions — generative modeling has converged on one recipe: compress into continuous latents, generate in latent space, decode. Language is the lone exception: still generated token by token, as a long discrete stream. Latent Thought Flows: We compress 256 text tokens into 8 continuous latents, generate them with a one-step flow model, and read them out as text using an autoregressive decoder. Result: a better inference-compute vs. generation-quality Pareto frontier than a tuned autoregressive baseline — thanks to compression and one-step generation. Co-led w/ Zhengyang Geng 🧵

Mihir Prabhudesai

83,020 views • 3 months ago

🔥Ultra-Long Video World Model up to 5min🔥 ✨ We introduce #LongVie2, an end-to-end autoregressive video world model that supports continuous video generation lasting up to 5min with: 🕹️ Strong Controllability 📷 Long-term Visual Fidelity 🔒 Temporal Consistency - Project: - Code: - Paper: . Thanks to AK !

🔥Ultra-Long Video World Model up to 5min🔥 ✨ We introduce #LongVie2, an end-to-end autoregressive video world model that supports continuous video generation lasting up to 5min with: 🕹️ Strong Controllability 📷 Long-term Visual Fidelity 🔒 Temporal Consistency - Project: - Code: - Paper: . Thanks to AK !

Ziwei Liu

82,798 views • 10 months ago

🕹️We are excited to introduce "ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation" ChronoEdit reframes image editing as a video generation task to encourage temporal consistency. It leverages a temporal reasoning stage that denoises with “video reasoning tokens” to "reason" on physically plausible edits. See the attached video for results. Project Page: Arxiv: Code and model are coming.

🕹️We are excited to introduce "ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation" ChronoEdit reframes image editing as a video generation task to encourage temporal consistency. It leverages a temporal reasoning stage that denoises with “video reasoning tokens” to "reason" on physically plausible edits. See the attached video for results. Project Page: Arxiv: Code and model are coming.

Huan Ling

37,048 views • 1 year ago

VSTAR Generative Temporal Nursing for Longer Dynamic Video Synthesis Despite tremendous progress in the field of text-to-video (T2V) synthesis, open-sourced T2V diffusion models struggle to generate longer videos with dynamically varying and evolving content. They tend to

VSTAR Generative Temporal Nursing for Longer Dynamic Video Synthesis Despite tremendous progress in the field of text-to-video (T2V) synthesis, open-sourced T2V diffusion models struggle to generate longer videos with dynamically varying and evolving content. They tend to

AK

37,003 views • 2 years ago

1/20 I trained two segmentation models using GPT-6 Astra Ultra as the orchestrator with no human labels supplied. My prompt was very lazy too as I had to catch a flight. RF-DETR-Seg-M predictions: Left: SAM3 training masks Right: Astra training masks Both videos held out.

1/20 I trained two segmentation models using GPT-6 Astra Ultra as the orchestrator with no human labels supplied. My prompt was very lazy too as I had to catch a flight. RF-DETR-Seg-M predictions: Left: SAM3 training masks Right: Astra training masks Both videos held out.

Satya Mallick

31,514 views • 1 month ago

Qwen Video Edit adapts Qwen Image Edit to do video editing. It edits video-VAE latents directly, avoiding a video pretrained transformer while supporting long and portrait clips.

Qwen Video Edit adapts Qwen Image Edit to do video editing. It edits video-VAE latents directly, avoiding a video pretrained transformer while supporting long and portrait clips.

AI Search

11,167 views • 2 months ago

Got tired of boring websites. Built a way to drop 3D agents directly onto standard HTML elements with zero complex math.

Got tired of boring websites. Built a way to drop 3D agents directly onto standard HTML elements with zero complex math.

nich

242,674 views • 5 months ago

Whilst we figure out a solution for temporal filtering on 3D Gaussian Splatting input frames. We can use a temporal filtering plugin on the screen captures or rendered images to act as a visualization tool to check what the future results might look like. "Processing and rendering a digital human with 3D Gaussian Splatting with a post process to simulate temporal filtering." Left video no temporal filtering. Right video with temporal filtering. R&D for archviz projects. #GaussianSplatting #ir #inria #sibr #aftereffects #temporalfiltering

Whilst we figure out a solution for temporal filtering on 3D Gaussian Splatting input frames. We can use a temporal filtering plugin on the screen captures or rendered images to act as a visualization tool to check what the future results might look like. "Processing and rendering a digital human with 3D Gaussian Splatting with a post process to simulate temporal filtering." Left video no temporal filtering. Right video with temporal filtering. R&D for archviz projects. #GaussianSplatting #ir #inria #sibr #aftereffects #temporalfiltering

Infinite-Realities

58,407 views • 3 years ago

Video Analysis and Generation via a Semantic Progress Function paper:

Video Analysis and Generation via a Semantic Progress Function paper:

AK

34,542 views • 6 months ago

What if you had nano-banana for audio? AudioChat is a multi-modal LM that performs fine-grained understanding, generation, and editing of multi-source scenes By diffusing continuous latents, it generates 48khz stereo edits with great input adherence:

What if you had nano-banana for audio? AudioChat is a multi-modal LM that performs fine-grained understanding, generation, and editing of multi-source scenes By diffusing continuous latents, it generates 48khz stereo edits with great input adherence:

William Chen

18,242 views • 8 months ago

Introducing Continuous Thought Machines New Blog: Modern AI is powerful, but it’s still distinct from human-like flexible intelligence. We believe neural timing is key. Our Continuous Thought Machine is built from the ground up to use neural dynamics as a powerful representation for intelligence. Thought takes time, and reasoning is a process. Biological brains inspire us with their complex neural activity, where neural timing is critical to intelligence. We’re exploring how to bring that power to AI. The Continuous Thought Machine (CTM) incorporates neuron-level temporal processing and neural synchronization, moving beyond current AI limitations. Our approach has two core innovations: (1) neuron-level temporal processing, where each neuron uses unique parameters to process a history of incoming signals for fine-grained temporal dynamics, and (2) neural synchronization, used as a direct latent representation to modulate data and produce outputs, encoding information directly in the timing of neural activity. Learn more about our approach: Interactive Report: Full Paper: GitHub :

Introducing Continuous Thought Machines New Blog: Modern AI is powerful, but it’s still distinct from human-like flexible intelligence. We believe neural timing is key. Our Continuous Thought Machine is built from the ground up to use neural dynamics as a powerful representation for intelligence. Thought takes time, and reasoning is a process. Biological brains inspire us with their complex neural activity, where neural timing is critical to intelligence. We’re exploring how to bring that power to AI. The Continuous Thought Machine (CTM) incorporates neuron-level temporal processing and neural synchronization, moving beyond current AI limitations. Our approach has two core innovations: (1) neuron-level temporal processing, where each neuron uses unique parameters to process a history of incoming signals for fine-grained temporal dynamics, and (2) neural synchronization, used as a direct latent representation to modulate data and produce outputs, encoding information directly in the timing of neural activity. Learn more about our approach: Interactive Report: Full Paper: GitHub :

Sakana AI

290,107 views • 1 year ago

One for all, and all for one 🧧 Introducing Ming-flash-omni-2.0: A specialist in every domain, unified as a capable generalist. A gift from Ling =) - Unified Acoustic Synthesis: Speech, audio, and music combined for unbounded creativity; - "Seeing" to "Knowing": Moving beyond input to true deep semantic understanding; - Native Visual Fusion: Seamless generation, editing, and segmentation;

One for all, and all for one 🧧 Introducing Ming-flash-omni-2.0: A specialist in every domain, unified as a capable generalist. A gift from Ling =) - Unified Acoustic Synthesis: Speech, audio, and music combined for unbounded creativity; - "Seeing" to "Knowing": Moving beyond input to true deep semantic understanding; - Native Visual Fusion: Seamless generation, editing, and segmentation;

Ant Ling

1,925,484 views • 8 months ago

'Rerender A Video' by Shuai Yang et al. uses a cluster of techniques to remove the flickering and temporal inconsistency that happens when using stable diffusion for video, while still being compatible with LoRA/ControlNet-driven content.

'Rerender A Video' by Shuai Yang et al. uses a cluster of techniques to remove the flickering and temporal inconsistency that happens when using stable diffusion for video, while still being compatible with LoRA/ControlNet-driven content.

Ben Ferns

726,935 views • 3 years ago

CoDeF: Content Deformation Fields for Temporally Consistent Video Processing abs: paper page: present the content deformation field CoDeF as a new type of video representation, which consists of a canonical content field aggregating the static contents in the entire video and a temporal deformation field recording the transformations from the canonical image (i.e., rendered from the canonical content field) to each individual frame along the time axis.Given a target video, these two fields are jointly optimized to reconstruct it through a carefully tailored rendering pipeline.We advisedly introduce some regularizations into the optimization process, urging the canonical content field to inherit semantics (e.g., the object shape) from the video.With such a design, CoDeF naturally supports lifting image algorithms for video processing, in the sense that one can apply an image algorithm to the canonical image and effortlessly propagate the outcomes to the entire video with the aid of the temporal deformation field.We experimentally show that CoDeF is able to lift image-to-image translation to video-to-video translation and lift keypoint detection to keypoint tracking without any training.More importantly, thanks to our lifting strategy that deploys the algorithms on only one image, we achieve superior cross-frame consistency in processed videos compared to existing video-to-video translation approaches, and even manage to track non-rigid objects like water and smog.

CoDeF: Content Deformation Fields for Temporally Consistent Video Processing abs: paper page: present the content deformation field CoDeF as a new type of video representation, which consists of a canonical content field aggregating the static contents in the entire video and a temporal deformation field recording the transformations from the canonical image (i.e., rendered from the canonical content field) to each individual frame along the time axis.Given a target video, these two fields are jointly optimized to reconstruct it through a carefully tailored rendering pipeline.We advisedly introduce some regularizations into the optimization process, urging the canonical content field to inherit semantics (e.g., the object shape) from the video.With such a design, CoDeF naturally supports lifting image algorithms for video processing, in the sense that one can apply an image algorithm to the canonical image and effortlessly propagate the outcomes to the entire video with the aid of the temporal deformation field.We experimentally show that CoDeF is able to lift image-to-image translation to video-to-video translation and lift keypoint detection to keypoint tracking without any training.More importantly, thanks to our lifting strategy that deploys the algorithms on only one image, we achieve superior cross-frame consistency in processed videos compared to existing video-to-video translation approaches, and even manage to track non-rigid objects like water and smog.

AK

153,321 views • 3 years ago

Wan2.2 is now natively supported in ComfyUI on Day 0! 🔹 A next-gen video model with MoE (Mixture of Experts) architecture with dual noise experts, under Apache 2.0 license! - Cinematic-level Aesthetic Control - Large-scale Complex Motion - Precise Semantic Compliance 📚 Versions available: - Wan2.2-TI2V-5B: FP16 - Wan2.2-I2V-14B: FP16/FP8 - Wan2.2-T2V-14B: FP16/FP8 💻 Down to 8GB VRAM requirement for the 5B version with ComfyUI auto-offloading.

Wan2.2 is now natively supported in ComfyUI on Day 0! 🔹 A next-gen video model with MoE (Mixture of Experts) architecture with dual noise experts, under Apache 2.0 license! - Cinematic-level Aesthetic Control - Large-scale Complex Motion - Precise Semantic Compliance 📚 Versions available: - Wan2.2-TI2V-5B: FP16 - Wan2.2-I2V-14B: FP16/FP8 - Wan2.2-T2V-14B: FP16/FP8 💻 Down to 8GB VRAM requirement for the 5B version with ComfyUI auto-offloading.

ComfyUI

84,680 views • 1 year ago

𝗠𝗲𝘁𝗮𝗖𝗮𝗻𝘃𝗮𝘀 🎨 lets MLLMs "𝘥𝘳𝘢𝘧𝘵 𝘢𝘯𝘥 𝘥𝘳𝘢𝘸 𝘰𝘯 𝘢 𝘤𝘢𝘯𝘷𝘢𝘴" to guide diffusion generators. - canvas: spatial & temporal latents - gains on 6 tasks: T2I, T/I2V, image/video editing, in-context vgen. 📄 🌐

𝗠𝗲𝘁𝗮𝗖𝗮𝗻𝘃𝗮𝘀 🎨 lets MLLMs "𝘥𝘳𝘢𝘧𝘵 𝘢𝘯𝘥 𝘥𝘳𝘢𝘸 𝘰𝘯 𝘢 𝘤𝘢𝘯𝘷𝘢𝘴" to guide diffusion generators. - canvas: spatial & temporal latents - gains on 6 tasks: T2I, T/I2V, image/video editing, in-context vgen. 📄 🌐

Ziqi Huang

11,054 views • 10 months ago