Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware check out this SAM2 vs SAMURAI comparison! - paper: - code: - license: Apache-2.0

skalskip92's profile picture

SkalskiP

46,528 subscribers

124,358 görüntüleme • 1 yıl önce •via X (Twitter)

EğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

11 Yorum

SkalskiP profil fotoğrafı
SkalskiP1 yıl önce

- enhance the visual tracking accuracy of SAM 2 by incorporating motion information through motion modeling, to effectively handle the fast-moving and occluded objects - propose a motion-aware memory selection mechanism that reduces error in crowded scenes in contrast to the original fixed-window memory by selectively storing relevant frames decided by a mixture of motion and affinity scores

SkalskiP profil fotoğrafı
SkalskiP1 yıl önce

state-of-the-art performance on various VOT benchmarks, including GOT-10k, LaSOT-ext, and NeedForSpeed

SkalskiP profil fotoğrafı
SkalskiP1 yıl önce

can't wait to have some fun with SAMURAI as I did with SAM2

Rainmaker profil fotoğrafı
Rainmaker2 yıl önce

Can Machine Learning beat the market? Check out this post on my free Substack where I share code and commentary for an XGBoost model and a Random Forest model that both deliver powerful performances.

BensenHsu profil fotoğrafı
BensenHsu1 yıl önce

The researchers aim to enhance the visual object tracking capabilities of the Segment Anything Model 2 (SAM 2) by addressing its limitations in handling crowded scenes and managing occlusions. The proposed SAMURAI framework demonstrates significant improvements over existing methods on various visual object tracking benchmarks, such as LaSOT, LaSOT ext, and GOT-10k, without the need for additional training or fine-tuning. full paper:

Data profil fotoğrafı
Data1 yıl önce

Anyone who is against ML/AI tools should be locked in a room and forced to rotoscope this mask by hand. They will be e/acc when they are let out.

X Æ A-12 profil fotoğrafı
X Æ A-121 yıl önce

Amazing work ! 😍

Carlos Alarcón profil fotoğrafı
Carlos Alarcón1 yıl önce

This is insane, great work !!

Tekholms profil fotoğrafı
Tekholms1 yıl önce

Absolutely mind blowing! IDK how you keep improving so quickly?? Any experiments with these results on live video feeds?

Brede profil fotoğrafı
Brede1 yıl önce

This is very cool! Tracking is incredibly hard. Would love to see this applied to multi-object tracking

justboulatbek profil fotoğrafı
justboulatbek1 yıl önce

I fear this kind of instruments among others are gonna be used in drones in their last mile before chasing the running target

Benzer Videolar

MetaAI's SAM 2 struggles when things move fast or when there are crowded, fast-moving objects! Introducing SAMURAI: An adaptation of the Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory. 100% Open Source
0:40

MetaAI's SAM 2 struggles when things move fast or when there are crowded, fast-moving objects! Introducing SAMURAI: An adaptation of the Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory. 100% Open Source

Sumanth

324,580 görüntüleme • 1 yıl önce

Apache 2.0 license🔥 On-device deployment ready Extends SAM2 for tracking objects in videos 🔥 Click-to-segment support EdgeTAM by Meta !
0:47

Apache 2.0 license🔥 On-device deployment ready Extends SAM2 for tracking objects in videos 🔥 Click-to-segment support EdgeTAM by Meta !

Gradio

23,301 görüntüleme • 1 yıl önce

SAMURAI vs. MetaAI's SAM 2! Traditional visual object tracking struggles in crowded, fast-moving, or self-occluded scenes, as does SAM2. Meet SAMURAI: a completely open-source adaptation of the Segment Anything Model for zero-shot visual tracking! Here's why it's a game-changer: 🚫 No need for retraining or finetuning 🎯 Boosts success rate and precision 🤖 Motion-aware memory selection 💪 Zero-shot performance on diverse datasets But that's not all: 🔬 Refines mask selection 🔮 Predicts object motion effectively 📈 Gains: 7.1% AUC on LaSOT, 3.5% AO on GOT-10k 🏆 Competes with fully supervised methods without extra training Link to the GitHub repo in the next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights & tutorials on AI and Machine Learning.
0:40

SAMURAI vs. MetaAI's SAM 2! Traditional visual object tracking struggles in crowded, fast-moving, or self-occluded scenes, as does SAM2. Meet SAMURAI: a completely open-source adaptation of the Segment Anything Model for zero-shot visual tracking! Here's why it's a game-changer: 🚫 No need for retraining or finetuning 🎯 Boosts success rate and precision 🤖 Motion-aware memory selection 💪 Zero-shot performance on diverse datasets But that's not all: 🔬 Refines mask selection 🔮 Predicts object motion effectively 📈 Gains: 7.1% AUC on LaSOT, 3.5% AO on GOT-10k 🏆 Competes with fully supervised methods without extra training Link to the GitHub repo in the next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights & tutorials on AI and Machine Learning.

Akshay 🚀

363,328 görüntüleme • 1 yıl önce

SAMURAI gives SAM 2 motion-aware memory! And the results look mind-blowing. This is zero-shot 🤯 Links ⬇️d
0:33

SAMURAI gives SAM 2 motion-aware memory! And the results look mind-blowing. This is zero-shot 🤯 Links ⬇️d

Dreaming Tulpa 🥓👑

116,471 görüntüleme • 1 yıl önce

✨ CVPR 2025 highlight: A Distractor-Aware Memory for Visual Object Tracking with SAM2 the authors propose a new distractor-aware memory model for SAM2 and an introspection-based update strategy that jointly addresses the segmentation accuracy as well as tracking robustness 🏡 (1/n)🧵👇
0:34

✨ CVPR 2025 highlight: A Distractor-Aware Memory for Visual Object Tracking with SAM2 the authors propose a new distractor-aware memory model for SAM2 and an introspection-based update strategy that jointly addresses the segmentation accuracy as well as tracking robustness 🏡 (1/n)🧵👇

GeekyRakshit (e/mad)

32,669 görüntüleme • 1 yıl önce

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️
0:35

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

AI at Meta

97,755 görüntüleme • 1 yıl önce

I'm experimenting with using SAM2 for automatic tracking of NBA players; it's mind-blowing how well this model performs out of the box. we might even add a SAM2-based tracker to the trackers package at some point. trackers:
0:27

I'm experimenting with using SAM2 for automatic tracking of NBA players; it's mind-blowing how well this model performs out of the box. we might even add a SAM2-based tracker to the trackers package at some point. trackers:

SkalskiP

293,191 görüntüleme • 1 yıl önce

Big News! Meta just released Segment Anything, a new AI model that can "cut out" any object, in any image/video, with a single click. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks.
0:22

Big News! Meta just released Segment Anything, a new AI model that can "cut out" any object, in any image/video, with a single click. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks.

Lior Alexander

290,190 görüntüleme • 3 yıl önce

CLIVE VS NOCTIS COMPARISON Check out this side-by-side comparison of Clive and Noctis’ movesets, highlighting their similarities. Clive feels like a reworked Noctis 2.0 for #TEKKEN8. Do you agree? Video created and edited by Saleem Mughal Link:
2:25

CLIVE VS NOCTIS COMPARISON Check out this side-by-side comparison of Clive and Noctis’ movesets, highlighting their similarities. Clive feels like a reworked Noctis 2.0 for #TEKKEN8. Do you agree? Video created and edited by Saleem Mughal Link:

YellowMotion

55,858 görüntüleme • 1 yıl önce

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇
0:32

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇

Haofei Xu

32,233 görüntüleme • 3 ay önce

🤯⚡️In addition to being ~$0 per million tokens, the new Gemini 1.5 Flash-8B model in Google AI Studio is ridiculously fast - check out this head-to-head comparison with 1.5 Pro on code generation:
0:23

🤯⚡️In addition to being ~$0 per million tokens, the new Gemini 1.5 Flash-8B model in Google AI Studio is ridiculously fast - check out this head-to-head comparison with 1.5 Pro on code generation:

👩‍💻 Paige Bailey

10,660 görüntüleme • 1 yıl önce

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX and video intelligence alike). The fix? They basically gave it a sense of motion and smarter memory selection using our old friend the Kalman filter -- helps it remember where things were going instead of just where they are. No retraining needed and runs in realtime. Elegant solution that matches (and even beats!) a lot of purpose built tracking systems. Nice example of how sometimes you don't need fancy new architectures -- just smart ways to use what the model already knows, plus some tried & tested classical methods. Every time someone's like "we solved tracking!" I check the methods section and... yep, there's our boy Kalman, still crushing it after all these years. Sometimes the old school stuff just works.
0:27

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX and video intelligence alike). The fix? They basically gave it a sense of motion and smarter memory selection using our old friend the Kalman filter -- helps it remember where things were going instead of just where they are. No retraining needed and runs in realtime. Elegant solution that matches (and even beats!) a lot of purpose built tracking systems. Nice example of how sometimes you don't need fancy new architectures -- just smart ways to use what the model already knows, plus some tried & tested classical methods. Every time someone's like "we solved tracking!" I check the methods section and... yep, there's our boy Kalman, still crushing it after all these years. Sometimes the old school stuff just works.

Bilawal Sidhu

47,246 görüntüleme • 1 yıl önce

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch
0:22

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch

Bo Wang

178,481 görüntüleme • 1 yıl önce

I'm building a visual editor for vibe-generated backends that lets you sync changes back to code. Like Lovable's Visual Edit, but for backend logic and workflows. Check out this demo with Convex Chef :) Curious about what you think!
1:12

I'm building a visual editor for vibe-generated backends that lets you sync changes back to code. Like Lovable's Visual Edit, but for backend logic and workflows. Check out this demo with Convex Chef :) Curious about what you think!

Alessia

50,211 görüntüleme • 1 yıl önce

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:
0:42

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Ilir Aliu - eu/acc

72,522 görüntüleme • 11 ay önce

Wan2.2 is now natively supported in ComfyUI on Day 0! 🔹 A next-gen video model with MoE (Mixture of Experts) architecture with dual noise experts, under Apache 2.0 license! - Cinematic-level Aesthetic Control - Large-scale Complex Motion - Precise Semantic Compliance 📚 Versions available: - Wan2.2-TI2V-5B: FP16 - Wan2.2-I2V-14B: FP16/FP8 - Wan2.2-T2V-14B: FP16/FP8 💻 Down to 8GB VRAM requirement for the 5B version with ComfyUI auto-offloading.
0:55

Wan2.2 is now natively supported in ComfyUI on Day 0! 🔹 A next-gen video model with MoE (Mixture of Experts) architecture with dual noise experts, under Apache 2.0 license! - Cinematic-level Aesthetic Control - Large-scale Complex Motion - Precise Semantic Compliance 📚 Versions available: - Wan2.2-TI2V-5B: FP16 - Wan2.2-I2V-14B: FP16/FP8 - Wan2.2-T2V-14B: FP16/FP8 💻 Down to 8GB VRAM requirement for the 5B version with ComfyUI auto-offloading.

ComfyUI

84,423 görüntüleme • 1 yıl önce

🚨 Breaking: Google will soon launch Gemini CLI, a powerful open-source AI agent built for the terminal. - Built on Gemini 2.5 Pro - 1 million token context window - Free tier with 60 requests per minute and 1,000 per day - Google Search grounding for real-time context - Script and plugin support - Non-interactive mode for automation - Support for Model Context Protocol (MCP) - Integrated with Gemini Code Assist in VS Code - Fully open-source under Apache 2.0
0:22

🚨 Breaking: Google will soon launch Gemini CLI, a powerful open-source AI agent built for the terminal. - Built on Gemini 2.5 Pro - 1 million token context window - Free tier with 60 requests per minute and 1,000 per day - Google Search grounding for real-time context - Script and plugin support - Non-interactive mode for automation - Support for Model Context Protocol (MCP) - Integrated with Gemini Code Assist in VS Code - Fully open-source under Apache 2.0

AshutoshShrivastava

217,433 görüntüleme • 1 yıl önce

Check out this drone: Joshua Bird built this drone ($20) in his dorm! [open-source motion capture system⬇️] Built at low cost, a motion capture system for tracking & and flying drones autonomously, with millimeter-level precision at room-scale. The student used $1 PS3 Eye cameras with 150fps capability. The challenge? PID tuning! It took him 4 days of crashes to get the drone to hover, but it's still wobbly. Used a 3x nested PID loop for precise control. This project led to his dissertation on visual SLAM! Full details - Algorithms for camera positioning & obstacle triangulation, in his YouTube video: Code & 3D files on GitHub: More projects at
1:48

Check out this drone: Joshua Bird built this drone ($20) in his dorm! [open-source motion capture system⬇️] Built at low cost, a motion capture system for tracking & and flying drones autonomously, with millimeter-level precision at room-scale. The student used $1 PS3 Eye cameras with 150fps capability. The challenge? PID tuning! It took him 4 days of crashes to get the drone to hover, but it's still wobbly. Used a 3x nested PID loop for precise control. This project led to his dissertation on visual SLAM! Full details - Algorithms for camera positioning & obstacle triangulation, in his YouTube video: Code & 3D files on GitHub: More projects at

Ilir Aliu

101,689 görüntüleme • 1 yıl önce

Testing four AI video models from the same input image and prompt. The scene: two cybernetic soldiers dueling with energy swords in a rainy neon cyberpunk city. This thread breaks down scene richness, physics, and camera motion for each model. First, a side-by-side comparison video of PixVerse V6 and Seedance 2.0.
0:22

Testing four AI video models from the same input image and prompt. The scene: two cybernetic soldiers dueling with energy swords in a rainy neon cyberpunk city. This thread breaks down scene richness, physics, and camera motion for each model. First, a side-by-side comparison video of PixVerse V6 and Seedance 2.0.

Jibril

113,346 görüntüleme • 3 ay önce

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames - correspondences as nearest neighbors - no motion or deformation estimation Stable geometry and tracking, even under large expressions and viewpoint changes - check out our results! 🌐 ▶️ Great work by Umut Kocasarı, Simon Giebenhain, Richard Shaw
0:42

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames - correspondences as nearest neighbors - no motion or deformation estimation Stable geometry and tracking, even under large expressions and viewpoint changes - check out our results! 🌐 ▶️ Great work by Umut Kocasarı, Simon Giebenhain, Richard Shaw

Matthias Niessner

61,636 görüntüleme • 3 ay önce