🚀 The Segment Anything Model (SAM) has been upgraded... to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearchshow more

Super excited to introduce SAM2 Studio! 🚀🤖 I've been... getting a lot of questions lately on supporting AI inference tailored for patient data and sensitive workflows. We optimized SAM2 to run completely on-device in real time for all of your medical segmentation workflows - including surgical video segmentation, radiographs, pathology slides and more!show more

🎉 The best way to start the week is... to find out that our MedSAM is finally published today in Nature Communications! **Segment anything in medical images** Paper: arXiv: Data & Code: MedSAM is the first promotable foundation model for medical image segmentation. **Highlights**: ⭐ Before its formal publication, we have received 220 citations and 1400+ GitHub stars 🙏🙏❤️‍🔥❤️‍🔥❤️‍🔥 📊 We curated a large-scale medical image dataset with 1,570,263 image-mask pairs, covering 10 imaging modalities and over 30 cancer types. 🚀 Built on top of SAM (AI at Meta ) with transfer learning, we have significantly enhanced its segmentation performance of medical images. 📈 Comprehensive evaluations of 86 internal validation tasks and 60 external validation tasks demonstrate its better accuracy and robustness than modality-wise specialist models. **What is Next? --- Clinical Translation!!** 🍕Our next goal is to make the model deployable on laptops (CPUs) or other edge devices without reliance on GPUs. We have distilled a lightweight model, LiteMedSAM, offering a speed boost of 10x while maintaining accuracy. Plus, we have integrated it into the 3D Slicer plugin, providing an efficient tool for medical image segmentation. 🌐 To further promote developments in this field, we organize a competition on #CVPR2026: Segment Anything in Medical Images on Laptop! An out-of-the-box baseline has been released to reduce the entry barriers. Welcome to join us to push the boundary further: 🙏 Massive thanks to MetaAI AI at Meta for their open-source project SAM and many reviewers/users for their invaluable feedback. A huge shoutout to my postdoc Jun Ma (JunMa) for his leadership on this project!! UHN AI Hub Vector Institute Peter Munk Cardiac Centre AI Department of Laboratory Medicine & Pathobiology U of T Department of Computer Science University of Toronto University Health Network Brad Wouters 🇨🇦 Barry Rubin MD, PhD, FRCSC Shaf Keshavjeeshow more

Exciting news! Our MedSAM is featured in Computer Vision... News! 🎉🎉🎉🙌 Check it out: We are not stopping at papers and aiming to take integration to the next level by incorporating MedSAM with 3D Slicer, a top-tier medical imaging viewer. 🏥 **Segment Anything in Medical Images!** Read the paper here: Get your hands on the code: Explore MedSAM@3Dslice: Huge shoutout to the lead contributor, Jun Ma (JunMa), and a big thank you to Reza Asakereh for his invaluable contributions to MedSAMSlicer! 👏 #MedSAM #MedicalImagingshow more

Tracking Anything with Decoupled Video Segmentation paper page: Training... data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.show more

Meta’s Segment Anything Model (SAM) 3.1 is now available... on Meta Model API, giving developers a fast and lightweight model for detection, segmentation and tracking in a single call on inference tuned for SAM 3.1's architecture. Use a short phrase to find objects in images and video. One API call returns detections, pixel-precise segmentation masks, and identity-preserving video tracks. Learn more and start building:show more

The Segment Anything Model (SAM) by Meta AI is... a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from photos or videos + zero-shot transfer to other segmentation tasks. Try the demo ➡️show more

Hitem3D V2.0 just launched. It's an AI 3D model... generation tool that does three things: - Photo or image to full 3D model - Model segmentation so you can isolate and refine parts - Image to STL for 3D printing Combine this with Google Nano Banana and you can build game assets, product prototypes, and printable designs in minutes. Here's what I tested. Sponsored by Hi3D 1/4show more

zero-shot image segmentation with Florence-2 + SAM-2 combo just... added new mode to my Hugging Face; now you can run open vocabulary detection with Florence-2 + box to mask with SAM2 link:show more

🚀 Smart Topology is LIVE in Image-to-3D! Generate 3D... assets with native part segmentation, clean and accurate, in ~10 seconds. Full control over poly count. Made for game dev workflows, real-time and web-based 3D. We'd love your feedback to help us refine the quality!show more

SAM 2 from Meta FAIR is the first unified... model for real-time, promptable object segmentation in images & videos. Using the model in our web-based demo you can segment, track and apply effects to objects in video in just a few clicks. Try SAM 2 ➡️show more

SuperGSeg: Open-Vocabulary 3D Segmentation with Structured Super-Gaussians Contributions: •... We propose SuperGSeg: a 3D segmentation method with neural Gaussians, designed to learn hierarchical instance segmentation features from 2D foundation models. • We introduce the concept of Super-Gaussian, a novel representation that integrates hierarchical instance segmentation features, enabling the embedding of high-dimensional language features. This approach addresses previously unfeasible challenges in representing complex scenes with rich semantic details. • Extensive experiments on the LERF-OVS and ScanNet datasets demonstrate the effectiveness of the proposed method, achieving significant improvements in open-vocabulary 3D object-level and scene-level semantic segmentation. It shows particular strength in capturing fine-grained scene details and dense pixel semantic segmentation tasks for the first time.show more

Meet SAM 3, a unified model that enables detection,... segmentation, and tracking of objects across images and videos. SAM 3 introduces some of our most highly requested features like text and exemplar prompts to segment all objects of a target category. Learnings from SAM 3 will help power new features in Instagram Edits and Vibes, bringing advanced segmentation capabilities directly to creators. 🔗 Learn more:show more

Google has launched Med-Gemini, an advanced AI fine-tuned for... medical Tasks. It significantly outperforms earlier models, including GPT-4, on most medical benchmarks. Achieves top scores, particularly on the MedQA-USMLE benchmark with a groundbreaking 91.1% accuracy.🚀 Demonstrates superior performance over GPT-4 by 44.5% on average across seven multimodal benchmarks. Excels in tasks such as medical summarization, generating doctor referrals, and simplifying medical documents. It is a preferred method over human expert analyses for complex text-based medical tasks. This marks a significant advancement in AI for healthcare, suggesting potential improvements in medical diagnostics and patient care. By choosing to harmonize with technology, we become more human, we become IRREPLACEABLE. Join the IRREPLACEABLE Academy and read the Book: #techforgood #medicine #aishow more

Stable Diffusion generates beautiful images, but can it be... used for open-world recognition? Try Demo! Our #CVPR2023 paper shows that the pre-trained diffusion model indeed is a good image parser, allows for open-vocabulary segmentation and detection.show more

Here’s a sneak peek using Rerun and Gradio for... data annotation. It uses Video Depth Anything and Segment Anything 2 under the hood to generate segmentation masks and depth maps/point clouds. More to share next week.show more

having worked on computer vision for +10 years ,... I have been asking myself lately how to solve EVERY computer vision problem with whatever LLM tech we have now. First step: image segmentation. meet LLMSeg, segmenting images with LLMs and B-splines loops. Basically no need for trained segmentation heads that do not generalize well, just use the MLLMs superior generalization to perform segmentation directly with no tools or CLI , just a simple pre-scripted forward loop with refinement process based on overlays. B-splines loops for base masks , and B-splines loops for positive and negative masks refinment ----> THEY ACTUALLY CONVERGE to more accurate masks 21 MLLMs tested , better "vision+reasoning" means better segmentation . any "decent" MLLM can do segmentation it turns out. website: paper: code: sounds on 🔊show more

🚀 I am very excited to release the SamGeo... QGIS plugin for geospatial image segmentation, powered by Meta’s Segment Anything Model (SAM 3) In this full tutorial, I’ll walk you through how to install, configure, and start segmenting satellite imagery in QGIS without writing a single line of code! 👉 Download the plugin here: 💻 Full video tutorial: #QGIS #SegmentAnything #SAM #GeoAI #RemoteSensingshow more

Most Powerful Spatial AI for 3D Segmentation in #GaussianSplatting... Major upgrade to our #SpatialAI! By defining just a few camera views, you can now segment extremely complex 3D objects with intricate geometries, including construction beams, industrial machinery, and beyond. ✅ No #3DGS retraining is required. Our spatial AI works seamlessly with any existing splats, enabling fast, on-the-fly 3D segmentation in photorealistic environments.show more