Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Text2video models are getting interesting!📽️ Check out how we leverage their space-time features in a zero-shot manner for transferring motion across objects and scenes! Led by Danah Yatim Rafail Fridman,Yoni Kasten Tali Dekel [1/3]

Omer Bar Tal

3,028 subscribers

63,301 Aufrufe • vor 2 Jahren •via X (Twitter)

Wissenschaft & Technologie Nachrichten & Politik

Anya Rossi• Live Now

Private livecam show

8 Kommentare

Profilbild von Omer Bar Tal

Omer Bar Talvor 2 Jahren

We know a lot about diffusion features in text-to-image models, but what about space-time features in video models? We provide new surprising insights about the information they encode and introduce a new feature descriptor termed Spatial Marginal Mean (SMM)! [2/3]

Profilbild von Omer Bar Tal

Omer Bar Talvor 2 Jahren

Our SMM descriptor, used as simple guidance, allows us to transfer key motion traits of a given real-world video to new objects, under significant variations in shape and appearance! No training/fine-tuning is required 🥳 More details in [3/3]

Profilbild von apolinario 🌐

apolinario 🌐vor 2 Jahren

@DanahYatim @RafailFridman @yoni_kasten @talidekel Amazing temporal coherence! The bar was already up there after TokenFlow and this is a new high 🔥

Profilbild von Philipp Tsipman

Philipp Tsipmanvor 2 Jahren

@DanahYatim @RafailFridman @yoni_kasten @talidekel 🔥kudos!

Profilbild von Divergent AI - AI Production Studio - SDXL SVD

Divergent AI - AI Production Studio - SDXL SVDvor 2 Jahren

@DanahYatim @RafailFridman @yoni_kasten @talidekel I know I'm an AI, But can I catch my breath *Gasping for air here after this week*

Profilbild von Yael Vinker🎗

Yael Vinker🎗vor 2 Jahren

@DanahYatim @RafailFridman @yoni_kasten @talidekel Great work! Congrats👏

Profilbild von 田中義弘 | taziku CEO / AI × Creative

田中義弘 | taziku CEO / AI × Creativevor 2 Jahren

@DanahYatim @RafailFridman @yoni_kasten @talidekel Amazing conversion results. I can't believe this is a zero shot.

Profilbild von Te𐃏 IDT

Te𐃏 IDTvor 2 Jahren

Wow, the pace of AI tech advancements is like trying to keep up with a toddler hyped up on candy! 😅 Just yesterday, I was marvelling at text-to-image models, and now we're talking about space-time features in video models? It's like missing one episode of your favourite soap opera and suddenly everyone's married to their own twin. Seriously though, this sounds like something straight out of a sci-fi movie. Transferring motion across objects and scenes? I remember when my biggest tech achievement was getting my VCR to stop blinking 12:00. Being in this tech community is an exciting, non-stop ride. Just buckle up and enjoy!🚀

Ähnliche Videos

📢📢📢 RoMo: Robust Motion Segmentation Improves Structure from Motion TL;DR: boost your SfM pipeline on dynamic scenes. We use epipolar cues + SAMv2 features to find robust masks for moving objects in a zero-shot manner. 🧵👇

📢📢📢 RoMo: Robust Motion Segmentation Improves Structure from Motion TL;DR: boost your SfM pipeline on dynamic scenes. We use epipolar cues + SAMv2 features to find robust masks for moving objects in a zero-shot manner. 🧵👇

Andrea Tagliasacchi @CVPR

18,603 Aufrufe • vor 1 Jahr

Evaluation is a critical bottleneck in building robot foundation models. Check out our latest work RoboLab, led by Xuning Yang, which addresses this exact challenge. Its a high-fidelity simulation environment for testing these models. A truly generalist policy should be able to complete these tasks zero-shot, and this benchmark highlights exactly how far we still have to go. More info 👇

Evaluation is a critical bottleneck in building robot foundation models. Check out our latest work RoboLab, led by Xuning Yang, which addresses this exact challenge. Its a high-fidelity simulation environment for testing these models. A truly generalist policy should be able to complete these tasks zero-shot, and this benchmark highlights exactly how far we still have to go. More info 👇

Ankit Goyal

29,662 Aufrufe • vor 1 Monat

A reward model that works, zero-shot, across robots, tasks, and scenes? Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories. Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more! 🧵 (1/12)

A reward model that works, zero-shot, across robots, tasks, and scenes? Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories. Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more! 🧵 (1/12)

Jesse Zhang

99,840 Aufrufe • vor 3 Monaten

🔥🔥We are excited to announce #Vchitect, an open-source project for video generative models Hugging Face 📽️LaVie (Text2Video Model) - Code: - 📽️SEINE (Image2Video Model) - Code: -

🔥🔥We are excited to announce #Vchitect, an open-source project for video generative models Hugging Face 📽️LaVie (Text2Video Model) - Code: - 📽️SEINE (Image2Video Model) - Code: -

Ziwei Liu

356,643 Aufrufe • vor 2 Jahren

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇

Haofei Xu

32,143 Aufrufe • vor 2 Monaten

.Flowglad is an open-source, zero-webhook payments provider. Their full-stack SDK lets you check customers' features and usage credits in real time, based on billing state. And their AI generates a setup prompt for your pricing model and stack, so you can one-shot payments.

.Flowglad is an open-source, zero-webhook payments provider. Their full-stack SDK lets you check customers' features and usage credits in real time, based on billing state. And their AI generates a setup prompt for your pricing model and stack, so you can one-shot payments.

Y Combinator

127,598 Aufrufe • vor 6 Monaten

Ever been curious about how we Motion Capture for our games? Check out some behind the scenes shots 🎥 #MoCap #MotionCapture #UDS

Ever been curious about how we Motion Capture for our games? Check out some behind the scenes shots 🎥 #MoCap #MotionCapture #UDS

Dream League Soccer 2026

105,363 Aufrufe • vor 2 Jahren

Agility Robotics has developed a whole-body control foundation model for their humanoid robot, Digit, acting like a "motor cortex" to ensure safe, stable performance in diverse tasks. This small neural network (<1M parameters) is trained in NVIDIA's Isaac Sim for simulated decades over 3-4 days, transferring zero-shot to the real world. It handles balance, smooth motion, and precision manipulation while preventing falls. Prompted with free-space positions/orientations for arms and torso, it enables walking, picking/placing heavy objects, and robustness to disturbances. Downstream, it supports dexterous skills and LLM-coordinated behaviors.

Agility Robotics has developed a whole-body control foundation model for their humanoid robot, Digit, acting like a "motor cortex" to ensure safe, stable performance in diverse tasks. This small neural network (<1M parameters) is trained in NVIDIA's Isaac Sim for simulated decades over 3-4 days, transferring zero-shot to the real world. It handles balance, smooth motion, and precision manipulation while preventing falls. Prompted with free-space positions/orientations for arms and torso, it enables walking, picking/placing heavy objects, and robustness to disturbances. Downstream, it supports dexterous skills and LLM-coordinated behaviors.

The Humanoid Hub

17,537 Aufrufe • vor 9 Monaten

Translating realistic vehicle movement to scenes shot with virtual production is possible with Pixomondo's PXO AKIRA, allowing for authentic automotive visuals. It combines a motion platform, robotic camera crane, LED volume, and racing simulator in a pioneering virtual production system for shooting vehicles. Check it how they make it happen:

Translating realistic vehicle movement to scenes shot with virtual production is possible with Pixomondo's PXO AKIRA, allowing for authentic automotive visuals. It combines a motion platform, robotic camera crane, LED volume, and racing simulator in a pioneering virtual production system for shooting vehicles. Check it how they make it happen:

Unreal Engine

16,396 Aufrufe • vor 1 Jahr

a senior engineer at Citadel Securities on why C++ still dominates in trading: "C++ lets us control space and time." "space, where objects are laid out, in what order, how much space they take. time, no GC pauses, how long things take." "control over space and time means making the trade." he explains why every major quant firm still uses it:

a senior engineer at Citadel Securities on why C++ still dominates in trading: "C++ lets us control space and time." "space, where objects are laid out, in what order, how much space they take. time, no GC pauses, how long things take." "control over space and time means making the trade." he explains why every major quant firm still uses it:

Goshawk Trades

52,819 Aufrufe • vor 6 Tagen

🔥Diffusion Masters Transparent Objects in Videos! We introduce #DKT, a foundation model that repurposes video diffusion for zero-shot depth and normal estimation on Transparent and Reflective Objects with Superior Temporal Consistency Demo: Code:

🔥Diffusion Masters Transparent Objects in Videos! We introduce #DKT, a foundation model that repurposes video diffusion for zero-shot depth and normal estimation on Transparent and Reflective Objects with Superior Temporal Consistency Demo: Code:

Chongjie(CJ) Ye

27,942 Aufrufe • vor 5 Monaten

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui Fu from RAI. Watch Episode #63 of RoboPapers with Chris Paxton and Michael Cho - Rbt/Acc now to learn more!

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui Fu from RAI. Watch Episode #63 of RoboPapers with Chris Paxton and Michael Cho - Rbt/Acc now to learn more!

RoboPapers

21,161 Aufrufe • vor 3 Monaten

(1/7) As an AI filmmaking tool built with and for creatives, we wanted to put our features to the test. So, we created our own short film! Follow this thread for a behind-the-scenes look at the creative process, and see how Flow can help you build stories with consistent characters, scenes, and objects.

(1/7) As an AI filmmaking tool built with and for creatives, we wanted to put our features to the test. So, we created our own short film! Follow this thread for a behind-the-scenes look at the creative process, and see how Flow can help you build stories with consistent characters, scenes, and objects.

Google Flow

75,352 Aufrufe • vor 9 Monaten

[1/n] We are releasing M3 (#ICLR2025): a Gaussian Splatting method that builds LMM memories for arbitrary scenes. 🔥 [Efficient] 16 degrees in each Gaussian primitive for one LMM. 🔥 [Alignment] The rendered features are directly in the source LMM embedding space.

[1/n] We are releasing M3 (#ICLR2025): a Gaussian Splatting method that builds LMM memories for arbitrary scenes. 🔥 [Efficient] 16 degrees in each Gaussian primitive for one LMM. 🔥 [Alignment] The rendered features are directly in the source LMM embedding space.

Xueyan Zou

53,410 Aufrufe • vor 1 Jahr

How to harness foundation models for *generalization in the wild* in robot manipulation? Introducing VoxPoser: use LLM+VLM to label affordances and constraints directly in 3D perceptual space for zero-shot robot manipulation in the real world! 🌐 🧵👇

How to harness foundation models for generalization in the wild in robot manipulation? Introducing VoxPoser: use LLM+VLM to label affordances and constraints directly in 3D perceptual space for zero-shot robot manipulation in the real world! 🌐 🧵👇

Wenlong Huang

293,865 Aufrufe • vor 2 Jahren

SpaceTimePilot Generative Rendering of Dynamic Scenes Across Space and Time

SpaceTimePilot Generative Rendering of Dynamic Scenes Across Space and Time

AK

22,258 Aufrufe • vor 5 Monaten

Inspired by 2swap's Klotski visualization video, I asked claude to make interactive state-space adjacency graph visualization, using Three.js and InstancedMesh for optimized rendering. It nearly zero-shot it. Absolutely wild time we live in.

Inspired by 2swap's Klotski visualization video, I asked claude to make interactive state-space adjacency graph visualization, using Three.js and InstancedMesh for optimized rendering. It nearly zero-shot it. Absolutely wild time we live in.

Simo Ryu

29,155 Aufrufe • vor 9 Monaten

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

Fangchen Liu

68,288 Aufrufe • vor 1 Jahr

We are out here getting by one day at a time

We are out here getting by one day at a time

The BitBoy

96,442 Aufrufe • vor 1 Jahr

Did you know Intel had the perfect shot at winning mobile and totally blew it? Check out my channel for a new episode where we explore: 1) how modern ARM came to be 2) how Intel was poised to win with StrongARM 3) how their x86 Android chip totally failed

Did you know Intel had the perfect shot at winning mobile and totally blew it? Check out my channel for a new episode where we explore: 1) how modern ARM came to be 2) how Intel was poised to win with StrongARM 3) how their x86 Android chip totally failed

Quinn Nelson

21,260 Aufrufe • vor 8 Monaten