Segment Anything is fun and powerful but still requires... full mask annotation for supervised training. Our new work MaskFreeVIS appearing at #CVPR2023 shows we can get accurate video instance segmentation WITHOUT any video or image mask annotations. More atshow more

Mask3D 🎭 is now at #ICRA2023, great work Jonas... Schult! We use Mask Transformers for 3D Instance Segmentation on Point Clouds ~ 🥇 on ScanNet 📰Paper: 🛠️Project: 👨‍💻Code: @Pandoro89 Or Litany Siyu Tang @VLG-ETHZshow more

Tracking Anything with Decoupled Video Segmentation paper page: Training... data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.show more

Here’s a sneak peek using Rerun and Gradio for... data annotation. It uses Video Depth Anything and Segment Anything 2 under the hood to generate segmentation masks and depth maps/point clouds. More to share next week.show more

Stable Diffusion generates beautiful images, but can it be... used for open-world recognition? Try Demo! Our #CVPR2023 paper shows that the pre-trained diffusion model indeed is a good image parser, allows for open-vocabulary segmentation and detection.show more

New video: Mask - Live at Fice Gallery 05/29/2023... Full set: Listen to Mask:show more

Track Anything: Segment Anything Meets Videos Track-Anything is a... flexible and interactive tool for video object tracking and segmentation suitable for: - Video object tracking and segmentation with shot changes. - Visualized development and data annnotation for video object tracking and segmentation. - Object-centric downstream video tasks, such as video inpainting and editing. abs: github:show more

Accepted by #CVPR2023! X-Decoder is the FIRST generalist decoder... that supports all segmentation tasks (ins/sem/pano/ref) in OPEN VOCABULARY, both inter- AND intra-image VL tasks, and even helps instruct image inpainting/editing! New demo below and more atshow more

The Segment Anything Model (SAM) by Meta AI is... a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from photos or videos + zero-shot transfer to other segmentation tasks. Try the demo ➡️show more

Big News! Meta just released Segment Anything, a new... AI model that can "cut out" any object, in any image/video, with a single click. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks.show more

VideoMaMa from KAIST: A mask-to-matte model, converts coarse video... segmentation into pixel-accurate alpha mattes; looks quite good. Based on SVD+DINOv3.show more

ROANYER and 媛野 Fetizen Mask【中文版】 . Unboxing video of... my new Jane 2.0 mask from Roanyer. She looks a lovely mask. I am a bit sh1t at videos but I am wearing my ROANYER May mask with boobs to give the video some interest.show more

VideoJAM is our new framework for improved motion generation... from AI at Meta We show that video generators struggle with motion because the training objective favors appearance over dynamics. VideoJAM directly adresses this **without any extra data or scaling** 👇🧵show more

Monocular depth estimation is “impossible” because one image can’t... measure depth geometrically. Our iDisc #CVPR2023 can group pixels w/o supervision and learn depth inductive bias on groups. We get LiDAR-like (but denser) depth from single images! More:show more

Meta presents Video Editing via Factorized Diffusion Distillation We... introduce Emu Video Edit (EVE), a model that establishes a new state-of-the art in video editing without relying on any supervised video editing data. To develop EVE we separately train an image editingshow more

‘Lake Fever’ live from A National Celebration. The full... video is out now on YouTube. Watch at We heard your requests for more live video recordings, and we listened. Subscribe to our YouTube to get a new song every Thursday.show more

Training a Whole-Body Control Foundation Model -- new work... from my team at Agility Robotics A neural network for controlling our humanoid robots which is robust to disturbances, can handle heavy objects, and is a powerful platform for learning new whole-body skills learn more in our blog post ->show more

🚀 The Segment Anything Model (SAM) has been upgraded... to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearchshow more

Behind the cover of a mask, anything can happen.... Flirting is the game 🔥 Feb 7 Doors open at 9pm Dress Code: Masked Rugged Luxe Mask is highly encouraged but not required. 🎭 🟠 Unli Cocktails 🟠 Gogo Bears 🟠 Great Music Get your tickets here:show more

Excited to present UnScene3D coming to #CVPR24. David Rozenberszki... introduces the first fully unsupervised learned 3D instance segmentation! Using graph cuts over self-supervised features, and self-training for robust & dense masks. by David Rozenberszki, w/ Or Litanyshow more

Man Tries To Enter Store Without Mask & Is... Told By Security To Leave And Get A Mask Before Entering… But He Had Other Plans‼️show more