Loading video...

Video Failed to Load

Go Home

Core AI: Realtime video image detection, tracking, and segmentation with SAM 3 and RF-DETR. All running on-device. I will share the code in the comments.

amos_gyamfi's profile picture

Amos Gyamfi

5,802 subscribers

15,373 views • 5 days ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

introducing: fruit xylophone a computer vision music experiment built with mediapipe hand tracking and roboflow RF-DETR object detection (running in realtime in the browser using transformers js)
0:15

introducing: fruit xylophone a computer vision music experiment built with mediapipe hand tracking and roboflow RF-DETR object detection (running in realtime in the browser using transformers js)

AA

48,020 views • 3 months ago

RF-DETR is now available in Hugging Face transformers state of the art in both detection and segmentation, outperforming YOLO architectures - checkpoints: - demo: - docs:
0:11

RF-DETR is now available in Hugging Face transformers state of the art in both detection and segmentation, outperforming YOLO architectures - checkpoints: - demo: - docs:

SkalskiP

79,277 views • 1 month ago

Gemma 4 watches raw video. Understands the scene. Then prompts SAM 3 to segment and RF-DETR to track. One AI directing two others. Fighter jets. Crowds. Aerial defense footage. All three models running locally on a MacBook. No cloud. What scene should I point this at next?
0:16

Gemma 4 watches raw video. Understands the scene. Then prompts SAM 3 to segment and RF-DETR to track. One AI directing two others. Fighter jets. Crowds. Aerial defense footage. All three models running locally on a MacBook. No cloud. What scene should I point this at next?

Maziyar PANAHI

368,647 views • 2 months ago

Sam 3 by Facebook now on MLX 🚀 Here is a realtime object tracking running on M3 Max 96GB.
0:16

Sam 3 by Facebook now on MLX 🚀 Here is a realtime object tracking running on M3 Max 96GB.

Prince Canuma

180,245 views • 3 months ago

smart parking demo - RF-DETR - license plate detection - OC-SORT - multi-object tracking - GLM-OCR - reading license plates on Monday I showed you trackers, yesterday GLM-OCR, today I'm putting everything together
0:17

smart parking demo - RF-DETR - license plate detection - OC-SORT - multi-object tracking - GLM-OCR - reading license plates on Monday I showed you trackers, yesterday GLM-OCR, today I'm putting everything together

SkalskiP

26,206 views • 3 months ago

Comparing AI video results: • Gen-3 • Kling AI 1.5 • Hailuo MiniMax • Luma Dream Machine I used a FLUX1.1 [pro] image as the input. Each model ran the same prompt 3 times, and I chose the best results: "Smooth side tracking shot of a woman jogging at twilight with the city skyline in the background, sharp focus on the woman as she pumps her arms with each stride as she jogs, depth of field, advertising commercial" Running, jogging, and walking are difficult for AI video models, and none of them produced a smooth result, but this is the worst they will be.
0:10

Comparing AI video results: • Gen-3 • Kling AI 1.5 • Hailuo MiniMax • Luma Dream Machine I used a FLUX1.1 [pro] image as the input. Each model ran the same prompt 3 times, and I chose the best results: "Smooth side tracking shot of a woman jogging at twilight with the city skyline in the background, sharp focus on the woman as she pumps her arms with each stride as she jogs, depth of field, advertising commercial" Running, jogging, and walking are difficult for AI video models, and none of them produced a smooth result, but this is the worst they will be.

Heather Cooper

57,611 views • 1 year ago

RF-DETR just landed to Hugging Face transformers 🥵🔥 sota real-time detection & segmentation models by Roboflow 💜 > play with our real-time demo > fine-tune the models on your use case with our tutorials (takes a toaster's VRAM) > or just hand them to your agents 😄
0:13

RF-DETR just landed to Hugging Face transformers 🥵🔥 sota real-time detection & segmentation models by Roboflow 💜 > play with our real-time demo > fine-tune the models on your use case with our tutorials (takes a toaster's VRAM) > or just hand them to your agents 😄

merve

56,682 views • 1 month ago

another experiment with fruit, computer vision, and projection mapping live object detections (roboflow RF-DETR model) are used to control the animated scene (threejs) i also added mediapipe segmentation to the pipeline to create collisions with my silhouette, but i haven't gotten a hang of it yet more experiments are underway
0:16

another experiment with fruit, computer vision, and projection mapping live object detections (roboflow RF-DETR model) are used to control the animated scene (threejs) i also added mediapipe segmentation to the pipeline to create collisions with my silhouette, but i haven't gotten a hang of it yet more experiments are underway

AA

21,581 views • 28 days ago

Today we're releasing the Segment Anything Model (SAM) — a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from any photo or video + zero-shot transfer to other segmentation tasks ➡️
0:15

Today we're releasing the Segment Anything Model (SAM) — a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from any photo or video + zero-shot transfer to other segmentation tasks ➡️

AI at Meta

3,570,391 views • 3 years ago

Worth mentioning to all the people on me about the pupil detection. It's a bug in the Comfy implementation and will get sorted.... this is day 1 testing remember. Here is the same video ran on Wan's website - near perfect pupil tracking.
0:10

Worth mentioning to all the people on me about the pupil detection. It's a bug in the Comfy implementation and will get sorted.... this is day 1 testing remember. Here is the same video ran on Wan's website - near perfect pupil tracking.

A.I.Warper

20,567 views • 9 months ago

FPS in top left corner 40fps realtime with video & 4.6gb VRAM ✅ 50fps realtime with image & 1.7gb VRAM ✅ Running locally on a 4090 o3mini w/ Cursor go brrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
0:16

FPS in top left corner 40fps realtime with video & 4.6gb VRAM ✅ 50fps realtime with image & 1.7gb VRAM ✅ Running locally on a 4090 o3mini w/ Cursor go brrrrrrrrrrrrrrrrrrrrrrrrrrrrrr

A.I.Warper

110,958 views • 1 year ago

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch
0:22

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch

Bo Wang

178,455 views • 1 year ago

From parked cars to an Airbus A321 at cruising altitude. Same laptop. MLX & Torch. SAM3 segments every vehicle on the ground. Yes, just cars. RF-DETR spots the Austrian Airlines jet overhead. Real-time detection. Two open-source models running locally. No cloud. No API.
0:16

From parked cars to an Airbus A321 at cruising altitude. Same laptop. MLX & Torch. SAM3 segments every vehicle on the ground. Yes, just cars. RF-DETR spots the Austrian Airlines jet overhead. Real-time detection. Two open-source models running locally. No cloud. No API.

Maziyar PANAHI

46,612 views • 2 months ago

GPT Image 2 + Seedance 2.0 Prompt Share Forest Rebirth Storyboard and video prompt in the replies. Created on Martini Art AI
0:15

GPT Image 2 + Seedance 2.0 Prompt Share Forest Rebirth Storyboard and video prompt in the replies. Created on Martini Art AI

Kōda

21,983 views • 1 month ago

Sidebar for AI application built with React and TypeScript 🚀 Code in the comments and within Atheros Pass 🧑‍💻
0:12

Sidebar for AI application built with React and TypeScript 🚀 Code in the comments and within Atheros Pass 🧑‍💻

David Mráz

65,412 views • 3 months ago

A few people asked if we show videos. No, we do not! It’s all realtime animation, created with code, running in the browser. We run CHROME to check if 1 pixel in the software is *exactly* 1 pixel on the hardware and if we get the right framerate.
0:17

A few people asked if we show videos. No, we do not! It’s all realtime animation, created with code, running in the browser. We run CHROME to check if 1 pixel in the software is *exactly* 1 pixel on the hardware and if we get the right framerate.

Leander Herzog

41,719 views • 2 years ago

Google’s Gemma 4 E2B running on-device on iPhone 17 Pro Gemma 4 is built from the same research as Gemini 3, has image understanding capabilities and can reason if needed Running at ~40tk/s with MLX optimized for Apple Silicon
0:12

Google’s Gemma 4 E2B running on-device on iPhone 17 Pro Gemma 4 is built from the same research as Gemini 3, has image understanding capabilities and can reason if needed Running at ~40tk/s with MLX optimized for Apple Silicon

Adrien Grondin

1,043,723 views • 2 months ago

i built an app to search old travel photos on my computer with natural language queries using free local AI models: - smolVLM to describe the scene and colours - roboflow RF-DETR to detect objects - chromaDB to store metadata labels and run semantic search - python + streamlit for the interface fast and free search has been achieved internally
0:19

i built an app to search old travel photos on my computer with natural language queries using free local AI models: - smolVLM to describe the scene and colours - roboflow RF-DETR to detect objects - chromaDB to store metadata labels and run semantic search - python + streamlit for the interface fast and free search has been achieved internally

AA

41,823 views • 3 months ago

The Automated TikTok & Instagram Farm (2.0) Complete guide live ↓ 1/ Device Management - Add unlimited devices - Create & manage device groups - Auto-scan screen coordinates to perform actions 2/ Post Analysis - Detects organic (non-sponsored) content - Identifies posts from your target audience - Analyzes video content, description, and comments 3/ Account Automation - Likes, saves, and comments based on post analysis - Posts AI-generated comments tailored to each video - Dynamic behavior to avoid detection Powered by OTG Chip + iMouse + OpenAI API + Lovable Link in 2nd Comment & BIO 🏴‍☠️
0:15

The Automated TikTok & Instagram Farm (2.0) Complete guide live ↓ 1/ Device Management - Add unlimited devices - Create & manage device groups - Auto-scan screen coordinates to perform actions 2/ Post Analysis - Detects organic (non-sponsored) content - Identifies posts from your target audience - Analyzes video content, description, and comments 3/ Account Automation - Likes, saves, and comments based on post analysis - Posts AI-generated comments tailored to each video - Dynamic behavior to avoid detection Powered by OTG Chip + iMouse + OpenAI API + Lovable Link in 2nd Comment & BIO 🏴‍☠️

Julian

37,926 views • 1 year ago

Pothole detection on the road in real time using Ultralytics YOLO26! 🕳️ Manual road inspections are slow, costly, and hard to scale. With object detection, potholes can be identified directly from street-level images or video feeds, enabling faster and more consistent road condition monitoring. How I built this demo: ✅ Trained a segmentation model on a custom dataset. ✅ Generated mask contours for each pothole. ✅ Leveraged the onnx-exported model for faster processing. #Pothole #RoadDamage #AI
0:15

Pothole detection on the road in real time using Ultralytics YOLO26! 🕳️ Manual road inspections are slow, costly, and hard to scale. With object detection, potholes can be identified directly from street-level images or video feeds, enabling faster and more consistent road condition monitoring. How I built this demo: ✅ Trained a segmentation model on a custom dataset. ✅ Generated mask contours for each pothole. ✅ Leveraged the onnx-exported model for faster processing. #Pothole #RoadDamage #AI

Muhammad Rizwan Munawar

30,677 views • 3 months ago