Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Text2video models are getting interesting!📽️ Check out how we leverage their space-time features in a zero-shot manner for transferring motion across objects and scenes! Led by Danah Yatim Rafail Fridman,Yoni Kasten Tali Dekel [1/3]

Omer Bar Tal

3,028 subscribers

63,301 views • 2 years ago •via X (Twitter)

Science & Technology News & Politics

Anya Rossi• Live Now

Private livecam show

8 Comments

Omer Bar Tal2 years ago

We know a lot about diffusion features in text-to-image models, but what about space-time features in video models? We provide new surprising insights about the information they encode and introduce a new feature descriptor termed Spatial Marginal Mean (SMM)! [2/3]

Omer Bar Tal2 years ago

Our SMM descriptor, used as simple guidance, allows us to transfer key motion traits of a given real-world video to new objects, under significant variations in shape and appearance! No training/fine-tuning is required 🥳 More details in [3/3]

apolinario 🌐2 years ago

@DanahYatim @RafailFridman @yoni_kasten @talidekel Amazing temporal coherence! The bar was already up there after TokenFlow and this is a new high 🔥

Philipp Tsipman2 years ago

@DanahYatim @RafailFridman @yoni_kasten @talidekel 🔥kudos!

Divergent AI - AI Production Studio - SDXL SVD2 years ago

@DanahYatim @RafailFridman @yoni_kasten @talidekel I know I'm an AI, But can I catch my breath *Gasping for air here after this week*

Yael Vinker🎗2 years ago

@DanahYatim @RafailFridman @yoni_kasten @talidekel Great work! Congrats👏

田中義弘 | taziku CEO / AI × Creative2 years ago

@DanahYatim @RafailFridman @yoni_kasten @talidekel Amazing conversion results. I can't believe this is a zero shot.

Te𐃏 IDT2 years ago

Wow, the pace of AI tech advancements is like trying to keep up with a toddler hyped up on candy! 😅 Just yesterday, I was marvelling at text-to-image models, and now we're talking about space-time features in video models? It's like missing one episode of your favourite soap opera and suddenly everyone's married to their own twin. Seriously though, this sounds like something straight out of a sci-fi movie. Transferring motion across objects and scenes? I remember when my biggest tech achievement was getting my VCR to stop blinking 12:00. Being in this tech community is an exciting, non-stop ride. Just buckle up and enjoy!🚀

Related Videos

📢📢📢 RoMo: Robust Motion Segmentation Improves Structure from Motion TL;DR: boost your SfM pipeline on dynamic scenes. We use epipolar cues + SAMv2 features to find robust masks for moving objects in a zero-shot manner. 🧵👇

📢📢📢 RoMo: Robust Motion Segmentation Improves Structure from Motion TL;DR: boost your SfM pipeline on dynamic scenes. We use epipolar cues + SAMv2 features to find robust masks for moving objects in a zero-shot manner. 🧵👇

Andrea Tagliasacchi 🇨🇦→✈️→🇺🇲 (@CVPR)

18,594 views • 1 year ago

Evaluation is a critical bottleneck in building robot foundation models. Check out our latest work RoboLab, led by Xuning Yang, which addresses this exact challenge. Its a high-fidelity simulation environment for testing these models. A truly generalist policy should be able to complete these tasks zero-shot, and this benchmark highlights exactly how far we still have to go. More info 👇

Evaluation is a critical bottleneck in building robot foundation models. Check out our latest work RoboLab, led by Xuning Yang, which addresses this exact challenge. Its a high-fidelity simulation environment for testing these models. A truly generalist policy should be able to complete these tasks zero-shot, and this benchmark highlights exactly how far we still have to go. More info 👇

Ankit Goyal

29,662 views • 1 month ago

A reward model that works, zero-shot, across robots, tasks, and scenes? Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories. Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more! 🧵 (1/12)

A reward model that works, zero-shot, across robots, tasks, and scenes? Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories. Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more! 🧵 (1/12)

Jesse Zhang @ ICRA 2026 ✈️

99,385 views • 3 months ago

🔥🔥We are excited to announce #Vchitect, an open-source project for video generative models Hugging Face 📽️LaVie (Text2Video Model) - Code: - 📽️SEINE (Image2Video Model) - Code: -

🔥🔥We are excited to announce #Vchitect, an open-source project for video generative models Hugging Face 📽️LaVie (Text2Video Model) - Code: - 📽️SEINE (Image2Video Model) - Code: -

Ziwei Liu

356,643 views • 2 years ago

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇

Haofei Xu

32,143 views • 2 months ago

.Flowglad is an open-source, zero-webhook payments provider. Their full-stack SDK lets you check customers' features and usage credits in real time, based on billing state. And their AI generates a setup prompt for your pricing model and stack, so you can one-shot payments.

.Flowglad is an open-source, zero-webhook payments provider. Their full-stack SDK lets you check customers' features and usage credits in real time, based on billing state. And their AI generates a setup prompt for your pricing model and stack, so you can one-shot payments.

Y Combinator

127,556 views • 6 months ago

Ever been curious about how we Motion Capture for our games? Check out some behind the scenes shots 🎥 #MoCap #MotionCapture #UDS

Ever been curious about how we Motion Capture for our games? Check out some behind the scenes shots 🎥 #MoCap #MotionCapture #UDS

Dream League Soccer 2026

104,933 views • 2 years ago

Translating realistic vehicle movement to scenes shot with virtual production is possible with Pixomondo's PXO AKIRA, allowing for authentic automotive visuals. It combines a motion platform, robotic camera crane, LED volume, and racing simulator in a pioneering virtual production system for shooting vehicles. Check it how they make it happen:

Translating realistic vehicle movement to scenes shot with virtual production is possible with Pixomondo's PXO AKIRA, allowing for authentic automotive visuals. It combines a motion platform, robotic camera crane, LED volume, and racing simulator in a pioneering virtual production system for shooting vehicles. Check it how they make it happen:

Unreal Engine

16,396 views • 1 year ago

Agility Robotics has developed a whole-body control foundation model for their humanoid robot, Digit, acting like a "motor cortex" to ensure safe, stable performance in diverse tasks. This small neural network (<1M parameters) is trained in NVIDIA's Isaac Sim for simulated decades over 3-4 days, transferring zero-shot to the real world. It handles balance, smooth motion, and precision manipulation while preventing falls. Prompted with free-space positions/orientations for arms and torso, it enables walking, picking/placing heavy objects, and robustness to disturbances. Downstream, it supports dexterous skills and LLM-coordinated behaviors.

Agility Robotics has developed a whole-body control foundation model for their humanoid robot, Digit, acting like a "motor cortex" to ensure safe, stable performance in diverse tasks. This small neural network (<1M parameters) is trained in NVIDIA's Isaac Sim for simulated decades over 3-4 days, transferring zero-shot to the real world. It handles balance, smooth motion, and precision manipulation while preventing falls. Prompted with free-space positions/orientations for arms and torso, it enables walking, picking/placing heavy objects, and robustness to disturbances. Downstream, it supports dexterous skills and LLM-coordinated behaviors.

The Humanoid Hub

17,537 views • 9 months ago

🔥Diffusion Masters Transparent Objects in Videos! We introduce #DKT, a foundation model that repurposes video diffusion for zero-shot depth and normal estimation on Transparent and Reflective Objects with Superior Temporal Consistency Demo: Code:

🔥Diffusion Masters Transparent Objects in Videos! We introduce #DKT, a foundation model that repurposes video diffusion for zero-shot depth and normal estimation on Transparent and Reflective Objects with Superior Temporal Consistency Demo: Code:

Chongjie(CJ) Ye

27,942 views • 5 months ago

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui Fu from RAI. Watch Episode #63 of RoboPapers with Chris Paxton and Michael Cho - Rbt/Acc now to learn more!

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui Fu from RAI. Watch Episode #63 of RoboPapers with Chris Paxton and Michael Cho - Rbt/Acc now to learn more!

RoboPapers

21,161 views • 3 months ago

[1/n] We are releasing M3 (#ICLR2025): a Gaussian Splatting method that builds LMM memories for arbitrary scenes. 🔥 [Efficient] 16 degrees in each Gaussian primitive for one LMM. 🔥 [Alignment] The rendered features are directly in the source LMM embedding space.

[1/n] We are releasing M3 (#ICLR2025): a Gaussian Splatting method that builds LMM memories for arbitrary scenes. 🔥 [Efficient] 16 degrees in each Gaussian primitive for one LMM. 🔥 [Alignment] The rendered features are directly in the source LMM embedding space.

Xueyan Zou

53,410 views • 1 year ago

(1/7) As an AI filmmaking tool built with and for creatives, we wanted to put our features to the test. So, we created our own short film! Follow this thread for a behind-the-scenes look at the creative process, and see how Flow can help you build stories with consistent characters, scenes, and objects.

(1/7) As an AI filmmaking tool built with and for creatives, we wanted to put our features to the test. So, we created our own short film! Follow this thread for a behind-the-scenes look at the creative process, and see how Flow can help you build stories with consistent characters, scenes, and objects.

Google Flow

75,352 views • 8 months ago

How to harness foundation models for *generalization in the wild* in robot manipulation? Introducing VoxPoser: use LLM+VLM to label affordances and constraints directly in 3D perceptual space for zero-shot robot manipulation in the real world! 🌐 🧵👇

How to harness foundation models for generalization in the wild in robot manipulation? Introducing VoxPoser: use LLM+VLM to label affordances and constraints directly in 3D perceptual space for zero-shot robot manipulation in the real world! 🌐 🧵👇

Wenlong Huang

293,853 views • 2 years ago

SpaceTimePilot Generative Rendering of Dynamic Scenes Across Space and Time

SpaceTimePilot Generative Rendering of Dynamic Scenes Across Space and Time

AK

22,258 views • 5 months ago

Inspired by 2swap's Klotski visualization video, I asked claude to make interactive state-space adjacency graph visualization, using Three.js and InstancedMesh for optimized rendering. It nearly zero-shot it. Absolutely wild time we live in.

Inspired by 2swap's Klotski visualization video, I asked claude to make interactive state-space adjacency graph visualization, using Three.js and InstancedMesh for optimized rendering. It nearly zero-shot it. Absolutely wild time we live in.

Simo Ryu

29,155 views • 9 months ago

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

Fangchen Liu

68,288 views • 1 year ago

We are out here getting by one day at a time

We are out here getting by one day at a time

The BitBoy

96,442 views • 1 year ago

Did you know Intel had the perfect shot at winning mobile and totally blew it? Check out my channel for a new episode where we explore: 1) how modern ARM came to be 2) how Intel was poised to win with StrongARM 3) how their x86 Android chip totally failed

Did you know Intel had the perfect shot at winning mobile and totally blew it? Check out my channel for a new episode where we explore: 1) how modern ARM came to be 2) how Intel was poised to win with StrongARM 3) how their x86 Android chip totally failed

Quinn Nelson

21,260 views • 8 months ago

Check out AutoBlend by Shahzod Boyhonov, a Blender add-on that blends objects seamlessly in screen-space without modifying meshes or materials. More details: #b3d

Check out AutoBlend by Shahzod Boyhonov, a Blender add-on that blends objects seamlessly in screen-space without modifying meshes or materials. More details: #b3d

Jettelly Inc.

96,266 views • 2 months ago