Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Track Anything: Segment Anything Meets Videos Track-Anything is a flexible and interactive tool for video object tracking and segmentation suitable for: - Video object tracking and segmentation with shot changes. - Visualized development and data annnotation for video object tracking and segmentation. - Object-centric downstream video tasks, such as... show more

AK

515,364 subscribers

578,661 views • 3 years ago •via X (Twitter)

Science & Technology News & Politics Education

Anya Rossi• Live Now

Private livecam show

10 Comments

AK3 years ago

demo:

AK3 years ago

Multiple Object Tracking and Segmentation (with XMem)

AK3 years ago

Video Object Tracking and Segmentation with Shot Changes (with XMem)

Smoke-away3 years ago

Neat 😎

Bhack3 years ago

@EMostaque There is an alternative version with AOT networks

Nicolai Klemke3 years ago

This is fine

Brett Adcock3 years ago

This is 😎

erictang.eth 🍕🍾3 years ago

Never too late to get into the video meme game. Looking forward to wasting more time.

Lior Golgher3 years ago

Could it be used for markerless pose estimation? DeepLabCut style?

FinnT7303 years ago

@Yokohara_h Ok, now to see if I can get working locally XD

Related Videos

Tracking Anything with Decoupled Video Segmentation paper page: Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.

Tracking Anything with Decoupled Video Segmentation paper page: Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.

AK

305,683 views • 2 years ago

The Segment Anything Model (SAM) by Meta AI is a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from photos or videos + zero-shot transfer to other segmentation tasks. Try the demo ➡️

The Segment Anything Model (SAM) by Meta AI is a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from photos or videos + zero-shot transfer to other segmentation tasks. Try the demo ➡️

AI at Meta

186,348 views • 3 years ago

Bussing plates after a healthy meal. Our robot only eats plastic fruits and veggies for now. Video segmentation accomplished using Track-Anything, an architecture combining Meta's SAM to generate a zero-shot segmentation prompt and Xmem to enable long-horizon, temporally consistent video segmentation masks.

Bussing plates after a healthy meal. Our robot only eats plastic fruits and veggies for now. Video segmentation accomplished using Track-Anything, an architecture combining Meta's SAM to generate a zero-shot segmentation prompt and Xmem to enable long-horizon, temporally consistent video segmentation masks.

Watney Robotics

14,856 views • 2 years ago

✨ CVPR 2025 highlight: A Distractor-Aware Memory for Visual Object Tracking with SAM2 the authors propose a new distractor-aware memory model for SAM2 and an introspection-based update strategy that jointly addresses the segmentation accuracy as well as tracking robustness 🏡 (1/n)🧵👇

✨ CVPR 2025 highlight: A Distractor-Aware Memory for Visual Object Tracking with SAM2 the authors propose a new distractor-aware memory model for SAM2 and an introspection-based update strategy that jointly addresses the segmentation accuracy as well as tracking robustness 🏡 (1/n)🧵👇

GeekyRakshit (e/mad)

32,669 views • 1 year ago

Here’s a sneak peek using Rerun and Gradio for data annotation. It uses Video Depth Anything and Segment Anything 2 under the hood to generate segmentation masks and depth maps/point clouds. More to share next week.

Here’s a sneak peek using Rerun and Gradio for data annotation. It uses Video Depth Anything and Segment Anything 2 under the hood to generate segmentation masks and depth maps/point clouds. More to share next week.

Pablo Vela

36,719 views • 1 year ago

SAM 2 from Meta FAIR is the first unified model for real-time, promptable object segmentation in images & videos. Using the model in our web-based demo you can segment, track and apply effects to objects in video in just a few clicks. Try SAM 2 ➡️

SAM 2 from Meta FAIR is the first unified model for real-time, promptable object segmentation in images & videos. Using the model in our web-based demo you can segment, track and apply effects to objects in video in just a few clicks. Try SAM 2 ➡️

AI at Meta

88,918 views • 2 years ago

🚀 The GeoAI QGIS Plugin is here 🔥 You can run Moondream vision-language models, object detection, image segmentation (SAM 3), and even train your own geospatial segmentation model end-to-end. Website: GitHub: Short demo: Full video tutorial: #QGIS #GeoAI #SAM3 #Geospatial #DeepLearning #ComputerVision #OpenSource #Python

🚀 The GeoAI QGIS Plugin is here 🔥 You can run Moondream vision-language models, object detection, image segmentation (SAM 3), and even train your own geospatial segmentation model end-to-end. Website: GitHub: Short demo: Full video tutorial: #QGIS #GeoAI #SAM3 #Geospatial #DeepLearning #ComputerVision #OpenSource #Python

Qiusheng Wu

11,738 views • 7 months ago

Three new open-source models just landed in ComfyUI natively: → Gemma 4 (Google DeepMind) - multimodal LLM handling text, image, audio, and video input with built-in step-by-step reasoning mode → VOID (Netflix) - video object removal that also erases shadows, reflections, and physical interactions caused by the removed subject → BiRefNet - high-res background & object segmentation, one of the most-used segmentation models in the ecosystem Workflows and blog linked below 👇

Three new open-source models just landed in ComfyUI natively: → Gemma 4 (Google DeepMind) - multimodal LLM handling text, image, audio, and video input with built-in step-by-step reasoning mode → VOID (Netflix) - video object removal that also erases shadows, reflections, and physical interactions caused by the removed subject → BiRefNet - high-res background & object segmentation, one of the most-used segmentation models in the ecosystem Workflows and blog linked below 👇

ComfyUI

218,749 views • 2 months ago

Agricultural Field Boundary Delineation (Instance Segmentation) with GeoAI Learn an end-to-end GeoAI workflow for agricultural field boundary delineation using instance segmentation and the Fields of the World dataset ( This workflow can be adapted to detect other object types (not just field boundaries) as long as training data is available. Video tutorial: Notebook: Web app: #geoai #geospatial #deeplearing

Agricultural Field Boundary Delineation (Instance Segmentation) with GeoAI Learn an end-to-end GeoAI workflow for agricultural field boundary delineation using instance segmentation and the Fields of the World dataset ( This workflow can be adapted to detect other object types (not just field boundaries) as long as training data is available. Video tutorial: Notebook: Web app: #geoai #geospatial #deeplearing

Qiusheng Wu

20,008 views • 4 months ago

NEW: Tracery. 25% Off until May 24. Automate color-based object tracking and generate sophisticated, customizable visual overlays. Effortlessly produce complex visual noise, data-informed diagrams, tracking graphics, and stylish motion elements derived directly from your video content. #aftereffects #aescripts

NEW: Tracery. 25% Off until May 24. Automate color-based object tracking and generate sophisticated, customizable visual overlays. Effortlessly produce complex visual noise, data-informed diagrams, tracking graphics, and stylish motion elements derived directly from your video content. #aftereffects #aescripts

aescripts+aeplugins

37,773 views • 1 year ago

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 views • 10 months ago

HOI-DETR (Hand-Object Interaction) is a new zero-shot framework that integrates hand-object and object-object interaction -Hand + tool detections -Interaction links (+secondary object) Huge for robotics training 🔥🔥

HOI-DETR (Hand-Object Interaction) is a new zero-shot framework that integrates hand-object and object-object interaction -Hand + tool detections -Interaction links (+secondary object) Huge for robotics training 🔥🔥

mirrash.eth 🦇🔊

301,255 views • 1 month ago

Segment Anything is fun and powerful but still requires full mask annotation for supervised training. Our new work MaskFreeVIS appearing at #CVPR2023 shows we can get accurate video instance segmentation WITHOUT any video or image mask annotations. More at

Segment Anything is fun and powerful but still requires full mask annotation for supervised training. Our new work MaskFreeVIS appearing at #CVPR2023 shows we can get accurate video instance segmentation WITHOUT any video or image mask annotations. More at

Fisher Yu

41,103 views • 3 years ago

.KeenTools released its geometry-based object and camera tracking solution GeoTracker for Houdini in open beta. Download:

.KeenTools released its geometry-based object and camera tracking solution GeoTracker for Houdini in open beta. Download:

80 LEVEL

18,352 views • 3 months ago

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:

I'm excited to share our new work Diffusion as Shader (DaS), a versatile controllable video generation method for various tasks: object manipulation, camera control, mesh-to-video, and motion transfer. Project page: Github:

Yuan Liu

33,467 views • 1 year ago

you can just steal this prompt and one shot a calorie tracking app in 9.12 minutes (full unedited video)

you can just steal this prompt and one shot a calorie tracking app in 9.12 minutes (full unedited video)

Anything

15,964 views • 9 months ago

Ask-Anything, tool for chatting about video with chatGPT, miniGPT4 and StableLM github: Gradio demo:

Ask-Anything, tool for chatting about video with chatGPT, miniGPT4 and StableLM github: Gradio demo:

AK

143,649 views • 3 years ago

InsertAnywhere Bridging 4D Scene Geometry and Diffusion Models for Realistic Video Object Insertion

InsertAnywhere Bridging 4D Scene Geometry and Diffusion Models for Realistic Video Object Insertion

AK

26,123 views • 7 months ago

Introducing Meta Segment Anything Model 2 (SAM 2) — the first unified model for real-time, promptable object segmentation in images & videos. SAM 2 is available today under Apache 2.0 so that anyone can use it to build their own experiences Details ➡️

Introducing Meta Segment Anything Model 2 (SAM 2) — the first unified model for real-time, promptable object segmentation in images & videos. SAM 2 is available today under Apache 2.0 so that anyone can use it to build their own experiences Details ➡️

AI at Meta

1,617,927 views • 2 years ago