正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware check out this SAM2 vs SAMURAI comparison! - paper: - code: - license: Apache-2.0

SkalskiP

47,793 subscribers

124,419 次观看 • 1 年前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

11 条评论

SkalskiP 的头像

SkalskiP1 年前

- enhance the visual tracking accuracy of SAM 2 by incorporating motion information through motion modeling, to effectively handle the fast-moving and occluded objects - propose a motion-aware memory selection mechanism that reduces error in crowded scenes in contrast to the original fixed-window memory by selectively storing relevant frames decided by a mixture of motion and affinity scores

SkalskiP 的头像

SkalskiP1 年前

state-of-the-art performance on various VOT benchmarks, including GOT-10k, LaSOT-ext, and NeedForSpeed

SkalskiP 的头像

SkalskiP1 年前

can't wait to have some fun with SAMURAI as I did with SAM2

Rainmaker 的头像

Rainmaker2 年前

Can Machine Learning beat the market? Check out this post on my free Substack where I share code and commentary for an XGBoost model and a Random Forest model that both deliver powerful performances.

BensenHsu 的头像

BensenHsu1 年前

The researchers aim to enhance the visual object tracking capabilities of the Segment Anything Model 2 (SAM 2) by addressing its limitations in handling crowded scenes and managing occlusions. The proposed SAMURAI framework demonstrates significant improvements over existing methods on various visual object tracking benchmarks, such as LaSOT, LaSOT ext, and GOT-10k, without the need for additional training or fine-tuning. full paper:

Data 的头像

Data1 年前

Anyone who is against ML/AI tools should be locked in a room and forced to rotoscope this mask by hand. They will be e/acc when they are let out.

X Æ A-12 的头像

X Æ A-121 年前

Amazing work ! 😍

Carlos Alarcón 的头像

Carlos Alarcón1 年前

This is insane, great work !!

Tekholms 的头像

Tekholms1 年前

Absolutely mind blowing! IDK how you keep improving so quickly?? Any experiments with these results on live video feeds?

Brede 的头像

Brede1 年前

This is very cool! Tracking is incredibly hard. Would love to see this applied to multi-object tracking

justboulatbek 的头像

justboulatbek1 年前

I fear this kind of instruments among others are gonna be used in drones in their last mile before chasing the running target

相关视频

MetaAI's SAM 2 struggles when things move fast or when there are crowded, fast-moving objects! Introducing SAMURAI: An adaptation of the Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory. 100% Open Source

MetaAI's SAM 2 struggles when things move fast or when there are crowded, fast-moving objects! Introducing SAMURAI: An adaptation of the Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory. 100% Open Source

Sumanth

324,618 次观看 • 1 年前

SAMURAI vs. MetaAI's SAM 2! Traditional visual object tracking struggles in crowded, fast-moving, or self-occluded scenes, as does SAM2. Meet SAMURAI: a completely open-source adaptation of the Segment Anything Model for zero-shot visual tracking! Here's why it's a game-changer: 🚫 No need for retraining or finetuning 🎯 Boosts success rate and precision 🤖 Motion-aware memory selection 💪 Zero-shot performance on diverse datasets But that's not all: 🔬 Refines mask selection 🔮 Predicts object motion effectively 📈 Gains: 7.1% AUC on LaSOT, 3.5% AO on GOT-10k 🏆 Competes with fully supervised methods without extra training Link to the GitHub repo in the next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights & tutorials on AI and Machine Learning.

SAMURAI vs. MetaAI's SAM 2! Traditional visual object tracking struggles in crowded, fast-moving, or self-occluded scenes, as does SAM2. Meet SAMURAI: a completely open-source adaptation of the Segment Anything Model for zero-shot visual tracking! Here's why it's a game-changer: 🚫 No need for retraining or finetuning 🎯 Boosts success rate and precision 🤖 Motion-aware memory selection 💪 Zero-shot performance on diverse datasets But that's not all: 🔬 Refines mask selection 🔮 Predicts object motion effectively 📈 Gains: 7.1% AUC on LaSOT, 3.5% AO on GOT-10k 🏆 Competes with fully supervised methods without extra training Link to the GitHub repo in the next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights & tutorials on AI and Machine Learning.

Akshay 🚀

363,468 次观看 • 1 年前

SAMURAI gives SAM 2 motion-aware memory! And the results look mind-blowing. This is zero-shot 🤯 Links ⬇️d

SAMURAI gives SAM 2 motion-aware memory! And the results look mind-blowing. This is zero-shot 🤯 Links ⬇️d

Dreaming Tulpa 🥓👑

116,495 次观看 • 1 年前

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

AI at Meta

97,755 次观看 • 2 年前

I'm experimenting with using SAM2 for automatic tracking of NBA players; it's mind-blowing how well this model performs out of the box. we might even add a SAM2-based tracker to the trackers package at some point. trackers:

I'm experimenting with using SAM2 for automatic tracking of NBA players; it's mind-blowing how well this model performs out of the box. we might even add a SAM2-based tracker to the trackers package at some point. trackers:

SkalskiP

293,362 次观看 • 1 年前

Big News! Meta just released Segment Anything, a new AI model that can "cut out" any object, in any image/video, with a single click. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks.

Big News! Meta just released Segment Anything, a new AI model that can "cut out" any object, in any image/video, with a single click. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks.

Lior Alexander

290,330 次观看 • 3 年前

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle both optical flow and point tracking in a unified structure, delivering incredibly strong results. 🏆 SOTA zero-shot performance across multiple benchmarks and in-the-wild videos. Check out the links below to see it in action! 👇

Haofei Xu

32,299 次观看 • 5 个月前

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX and video intelligence alike). The fix? They basically gave it a sense of motion and smarter memory selection using our old friend the Kalman filter -- helps it remember where things were going instead of just where they are. No retraining needed and runs in realtime. Elegant solution that matches (and even beats!) a lot of purpose built tracking systems. Nice example of how sometimes you don't need fancy new architectures -- just smart ways to use what the model already knows, plus some tried & tested classical methods. Every time someone's like "we solved tracking!" I check the methods section and... yep, there's our boy Kalman, still crushing it after all these years. Sometimes the old school stuff just works.

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX and video intelligence alike). The fix? They basically gave it a sense of motion and smarter memory selection using our old friend the Kalman filter -- helps it remember where things were going instead of just where they are. No retraining needed and runs in realtime. Elegant solution that matches (and even beats!) a lot of purpose built tracking systems. Nice example of how sometimes you don't need fancy new architectures -- just smart ways to use what the model already knows, plus some tried & tested classical methods. Every time someone's like "we solved tracking!" I check the methods section and... yep, there's our boy Kalman, still crushing it after all these years. Sometimes the old school stuff just works.

Bilawal Sidhu

47,287 次观看 • 1 年前

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: Highlights: 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch

Bo Wang

178,579 次观看 • 2 年前

I'm building a visual editor for vibe-generated backends that lets you sync changes back to code. Like Lovable's Visual Edit, but for backend logic and workflows. Check out this demo with Convex Chef :) Curious about what you think!

I'm building a visual editor for vibe-generated backends that lets you sync changes back to code. Like Lovable's Visual Edit, but for backend logic and workflows. Check out this demo with Convex Chef :) Curious about what you think!

Alessia

50,218 次观看 • 1 年前

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Humanoids finally move like humans… and can do more than copy. [Details + demos in thread 👇] A new framework, BeyondMimic, shows how to learn naturalistic whole-body control from human motion. But then goes further by composing those skills into versatile, zero-shot policies. Two key pieces make it work: •Motion tracking pipeline: robustly turns human mocap into dynamic robot motions (jump spins, sprinting, cartwheels) on real hardware •Guided diffusion policy: distills motion primitives and enables zero-shot downstream tasks with simple cost functions The result: ✅ State-of-the-art motion quality from mocap tracking ✅ Versatile task execution; waypoint navigation, joystick teleop, obstacle avoidance ✅ Stable sim-to-real transfer for diverse humanoid skills Thank you for sharing, Qiayuan Liao! 📍 Paper: Video:

Ilir Aliu - eu/acc

72,522 次观看 • 1 年前

Wan2.2 is now natively supported in ComfyUI on Day 0! 🔹 A next-gen video model with MoE (Mixture of Experts) architecture with dual noise experts, under Apache 2.0 license! - Cinematic-level Aesthetic Control - Large-scale Complex Motion - Precise Semantic Compliance 📚 Versions available: - Wan2.2-TI2V-5B: FP16 - Wan2.2-I2V-14B: FP16/FP8 - Wan2.2-T2V-14B: FP16/FP8 💻 Down to 8GB VRAM requirement for the 5B version with ComfyUI auto-offloading.

Wan2.2 is now natively supported in ComfyUI on Day 0! 🔹 A next-gen video model with MoE (Mixture of Experts) architecture with dual noise experts, under Apache 2.0 license! - Cinematic-level Aesthetic Control - Large-scale Complex Motion - Precise Semantic Compliance 📚 Versions available: - Wan2.2-TI2V-5B: FP16 - Wan2.2-I2V-14B: FP16/FP8 - Wan2.2-T2V-14B: FP16/FP8 💻 Down to 8GB VRAM requirement for the 5B version with ComfyUI auto-offloading.

ComfyUI

84,552 次观看 • 1 年前

🚨 Breaking: Google will soon launch Gemini CLI, a powerful open-source AI agent built for the terminal. - Built on Gemini 2.5 Pro - 1 million token context window - Free tier with 60 requests per minute and 1,000 per day - Google Search grounding for real-time context - Script and plugin support - Non-interactive mode for automation - Support for Model Context Protocol (MCP) - Integrated with Gemini Code Assist in VS Code - Fully open-source under Apache 2.0

🚨 Breaking: Google will soon launch Gemini CLI, a powerful open-source AI agent built for the terminal. - Built on Gemini 2.5 Pro - 1 million token context window - Free tier with 60 requests per minute and 1,000 per day - Google Search grounding for real-time context - Script and plugin support - Non-interactive mode for automation - Support for Model Context Protocol (MCP) - Integrated with Gemini Code Assist in VS Code - Fully open-source under Apache 2.0

AshutoshShrivastava

217,433 次观看 • 1 年前

Check out this drone: Joshua Bird built this drone ($20) in his dorm! [open-source motion capture system⬇️] Built at low cost, a motion capture system for tracking & and flying drones autonomously, with millimeter-level precision at room-scale. The student used $1 PS3 Eye cameras with 150fps capability. The challenge? PID tuning! It took him 4 days of crashes to get the drone to hover, but it's still wobbly. Used a 3x nested PID loop for precise control. This project led to his dissertation on visual SLAM! Full details - Algorithms for camera positioning & obstacle triangulation, in his YouTube video: Code & 3D files on GitHub: More projects at

Check out this drone: Joshua Bird built this drone ($20) in his dorm! [open-source motion capture system⬇️] Built at low cost, a motion capture system for tracking & and flying drones autonomously, with millimeter-level precision at room-scale. The student used $1 PS3 Eye cameras with 150fps capability. The challenge? PID tuning! It took him 4 days of crashes to get the drone to hover, but it's still wobbly. Used a 3x nested PID loop for precise control. This project led to his dissertation on visual SLAM! Full details - Algorithms for camera positioning & obstacle triangulation, in his YouTube video: Code & 3D files on GitHub: More projects at

Ilir Aliu

101,793 次观看 • 1 年前

What if a $3,000 After Effects motion project could cost you just $3? TopviewAI just launched Topview Motion Studio, and it’s honestly pretty wild. • Cost: $3 generation vs. ~$3,000 for a traditional outsourced motion designer • Workflow: No complicated keyframes or timelines just AI-powered generation with Seedance 2.5 • Speed: Upload your logo, UI, and text requirements, and get a polished 15–30 second video in minutes. This makes a lot of sense for App promos, SaaS motion posters, social ads, and pretty much any short-form visual that normally takes hours to produce. The efficiency is the crazy part. Check out the video effects in the comparison below 👇

Harsh Hada

19,805 次观看 • 13 天前

Just added visual editing for any coding agent. Works with your: ✅ Claude Code CLI ✅ Codex CLI ✅ GitHub Copilot ✅ Cursor CLI Make visual tweaks -> view diffs -> commit, all in the browser! Works with your app framework of choice: ✅ Next.js ✅ React (Vite) ✅ Svelte ✅ Angular ✅ Other frameworks (.NET, Ruby, etc.) So if you liked Cursor's visual editor, check us out! It's live today as a VS Code extension called 'Wuunu'.

Just added visual editing for any coding agent. Works with your: ✅ Claude Code CLI ✅ Codex CLI ✅ GitHub Copilot ✅ Cursor CLI Make visual tweaks -> view diffs -> commit, all in the browser! Works with your app framework of choice: ✅ Next.js ✅ React (Vite) ✅ Svelte ✅ Angular ✅ Other frameworks (.NET, Ruby, etc.) So if you liked Cursor's visual editor, check us out! It's live today as a VS Code extension called 'Wuunu'.

Wesley

57,809 次观看 • 8 个月前