Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

MetaAI's SAM 2 struggles when things move fast or when there are crowded, fast-moving objects! Introducing SAMURAI: An adaptation of the Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory. 100% Open Source

Sumanth

76,665 subscribers

324,580 views • 1 year ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

11 Comments

Sumanth1 year ago

Github Repo:

Sumanth1 year ago

If you find this useful, RT to share it with your friends. Don't forget to follow me @Sumanth_077 for more such content and tutorials on Python, AI & ML!

Page to Pixel Publishing2 years ago

The Art of Flight is a homage to 80s/90s arcade action shmups with a fresh twist on the genre. Pilot multiple ships at the same time to take on oncoming waves of enemies in this fast paced space shooter. Wishlist on Steam today!

Rethynk AI1 year ago

That’s a brilliant upgrade! SAMURAI sounds like a game-changer for dynamic environments where traditional SAM models fall short. Motion-aware memory could make zero-shot visual tracking far more robust, especially in real-world applications like sports analysis or autonomous vehicles.

Sumanth1 year ago

Absolutely!

anarki🌟1 year ago

instant @arXivBangers haha let’s gooooo!!!!!

Pérry Odé 🇪🇺🇩🇪🇺🇦🇮🇱1 year ago

If I were a producer of self-shooting and AI-driven drones, SAMURAI would be preferable in this case. 👍

Machine Learning Community ⭐️1 year ago

Impressive!

Sumanth1 year ago

Indeed!

Appy Pie1 year ago

MetaAI's SAM 2 meets its match with fast motion and crowded scenes, but SAMURAI steps in! Motion-aware memory and zero-shot visual tracking make it a game-changer. Plus, it's 100% open source!

tarama1 year ago

@BenjaminDEKR fyi

Related Videos

SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware check out this SAM2 vs SAMURAI comparison! - paper: - code: - license: Apache-2.0

SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware check out this SAM2 vs SAMURAI comparison! - paper: - code: - license: Apache-2.0

SkalskiP

124,358 views • 1 year ago

SAMURAI gives SAM 2 motion-aware memory! And the results look mind-blowing. This is zero-shot 🤯 Links ⬇️d

SAMURAI gives SAM 2 motion-aware memory! And the results look mind-blowing. This is zero-shot 🤯 Links ⬇️d

Dreaming Tulpa 🥓👑

116,471 views • 1 year ago

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX and video intelligence alike). The fix? They basically gave it a sense of motion and smarter memory selection using our old friend the Kalman filter -- helps it remember where things were going instead of just where they are. No retraining needed and runs in realtime. Elegant solution that matches (and even beats!) a lot of purpose built tracking systems. Nice example of how sometimes you don't need fancy new architectures -- just smart ways to use what the model already knows, plus some tried & tested classical methods. Every time someone's like "we solved tracking!" I check the methods section and... yep, there's our boy Kalman, still crushing it after all these years. Sometimes the old school stuff just works.

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX and video intelligence alike). The fix? They basically gave it a sense of motion and smarter memory selection using our old friend the Kalman filter -- helps it remember where things were going instead of just where they are. No retraining needed and runs in realtime. Elegant solution that matches (and even beats!) a lot of purpose built tracking systems. Nice example of how sometimes you don't need fancy new architectures -- just smart ways to use what the model already knows, plus some tried & tested classical methods. Every time someone's like "we solved tracking!" I check the methods section and... yep, there's our boy Kalman, still crushing it after all these years. Sometimes the old school stuff just works.

Bilawal Sidhu

47,246 views • 1 year ago

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

AI at Meta

97,733 views • 1 year ago

Introducing CerebrasCoder! An open-source app that generates websites with Llama3.3-70b from as fast as you can type. 100% free and open-source.

Introducing CerebrasCoder! An open-source app that generates websites with Llama3.3-70b from as fast as you can type. 100% free and open-source.

Steve Krouse

240,119 views • 1 year ago

✨ CVPR 2025 highlight: A Distractor-Aware Memory for Visual Object Tracking with SAM2 the authors propose a new distractor-aware memory model for SAM2 and an introspection-based update strategy that jointly addresses the segmentation accuracy as well as tracking robustness 🏡 (1/n)🧵👇

✨ CVPR 2025 highlight: A Distractor-Aware Memory for Visual Object Tracking with SAM2 the authors propose a new distractor-aware memory model for SAM2 and an introspection-based update strategy that jointly addresses the segmentation accuracy as well as tracking robustness 🏡 (1/n)🧵👇

GeekyRakshit (e/mad)

32,669 views • 1 year ago

The Segment Anything Model (SAM) by Meta AI is a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from photos or videos + zero-shot transfer to other segmentation tasks. Try the demo ➡️

The Segment Anything Model (SAM) by Meta AI is a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from photos or videos + zero-shot transfer to other segmentation tasks. Try the demo ➡️

AI at Meta

186,348 views • 3 years ago

Introducing VT Tensiontrack a cylindrical tracking scenario with a mix of fast, short & long strafes. Uniquely made on @Aimlab with an indicator that informs the player when to increase tension or loosen up. Great for practicing tension control! ✏️ @Lowgravity56 | 🎯 VT Matty

Introducing VT Tensiontrack a cylindrical tracking scenario with a mix of fast, short & long strafes. Uniquely made on @Aimlab with an indicator that informs the player when to increase tension or loosen up. Great for practicing tension control! ✏️ @Lowgravity56 | 🎯 VT Matty

Voltaic

27,028 views • 2 years ago

It's out, and it comes with a demo🔥 Introducing Flux.2 [klein] Move ✨ > specialized move & resize LoRA for Flux.2 [klein] 9B > draw boxes manually or pick from auto-detected objects > works with fast 4-8 step inference 💨

It's out, and it comes with a demo🔥 Introducing Flux.2 [klein] Move ✨ > specialized move & resize LoRA for Flux.2 [klein] 9B > draw boxes manually or pick from auto-detected objects > works with fast 4-8 step inference 💨

Linoy Tsaban

59,059 views • 2 months ago

That's very impressive You can now use an audio-visual AI model in REAL TIME. So you're not just generating a one-shot video but you can: > interact with it > create a real time character > speak and receive replies Things are moving so fast

That's very impressive You can now use an audio-visual AI model in REAL TIME. So you're not just generating a one-shot video but you can: > interact with it > create a real time character > speak and receive replies Things are moving so fast

Paul Couvert

44,495 views • 20 days ago

You know that things are moving far too fast when even the 1990s feel as if they're a lifetime ago.

You know that things are moving far too fast when even the 1990s feel as if they're a lifetime ago.

Vinnie Sullivan

88,813 views • 12 days ago

Lots of things are happening. We are moving pretty fast.

Lots of things are happening. We are moving pretty fast.

SkriptkeeperElect

135,387 views • 1 year ago

I spent most of today preparing for CVPR 2024 "Matching Anything by Segmenting Anything" particularly caught my attention. Here are the fast open-vocabulary tracking examples (MASA + YOLO-World). link: ↓ read more

I spent most of today preparing for CVPR 2024 "Matching Anything by Segmenting Anything" particularly caught my attention. Here are the fast open-vocabulary tracking examples (MASA + YOLO-World). link: ↓ read more

SkalskiP

114,506 views • 2 years ago

✨ Introducing 𝐎𝐩𝐞𝐧𝐕𝐋𝐀 — an open-source vision-language-action model for robotics! 👐 - SOTA generalist policy - 7B params - outperforms Octo, RT-2-X on zero-shot evals 🦾 - trained on 970k episodes from OpenX dataset 🤖 - fully open: model/code/data all online 🤗 🧵👇

✨ Introducing 𝐎𝐩𝐞𝐧𝐕𝐋𝐀 — an open-source vision-language-action model for robotics! 👐 - SOTA generalist policy - 7B params - outperforms Octo, RT-2-X on zero-shot evals 🦾 - trained on 970k episodes from OpenX dataset 🤖 - fully open: model/code/data all online 🤗 🧵👇

Moo Jin Kim

226,991 views • 2 years ago

Finishing too fast can be a problem. But some things are better when they’re fast. Watch till the end & join our Gold Fast or Go Home challenge. 👀

Finishing too fast can be a problem. But some things are better when they’re fast. Watch till the end & join our Gold Fast or Go Home challenge. 👀

Bitget TradFi

14,772,242 views • 1 month ago

Introducing Chroma Cloud: an open-source serverless search database that is fast, cost-effective, scalable, and reliable.

Introducing Chroma Cloud: an open-source serverless search database that is fast, cost-effective, scalable, and reliable.

Chroma

127,662 views • 10 months ago

Remarkably lifelike motion and fluidity. BeyondMimic is a framework for training humanoid whole-body control from large mocap datasets. First, an open-source motion-tracking pipeline to reproduce diverse, highly dynamic human skills on real hardware, then distilling them into a guided state-action diffusion model for zero-shot, task-specific control. Project page:

Remarkably lifelike motion and fluidity. BeyondMimic is a framework for training humanoid whole-body control from large mocap datasets. First, an open-source motion-tracking pipeline to reproduce diverse, highly dynamic human skills on real hardware, then distilling them into a guided state-action diffusion model for zero-shot, task-specific control. Project page:

The Humanoid Hub

58,081 views • 10 months ago

"Things can move really fast when you make decisions." - Jon Moxley #AEWDynamite

"Things can move really fast when you make decisions." - Jon Moxley #AEWDynamite

All Elite Wrestling

103,062 views • 10 months ago