Segment Anything is fun and powerful but still requires... full mask annotation for supervised training. Our new work MaskFreeVIS appearing at #CVPR2023 shows we can get accurate video instance segmentation WITHOUT any video or image mask annotations. More atshow more

Mask3D 🎭 is now at #ICRA2023, great work Jonas... Schult! We use Mask Transformers for 3D Instance Segmentation on Point Clouds ~ 🥇 on ScanNet 📰Paper: 🛠️Project: 👨‍💻Code: @Pandoro89 Or Litany Siyu Tang @VLG-ETHZshow more

Tracking Anything with Decoupled Video Segmentation paper page: Training... data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.show more

VideoJAM is our new framework for improved motion generation... from AI at Meta We show that video generators struggle with motion because the training objective favors appearance over dynamics. VideoJAM directly adresses this **without any extra data or scaling** 👇🧵show more

🚀 The Segment Anything Model (SAM) has been upgraded... to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearchshow more

Training a Whole-Body Control Foundation Model -- new work... from my team at Agility Robotics A neural network for controlling our humanoid robots which is robust to disturbances, can handle heavy objects, and is a powerful platform for learning new whole-body skills learn more in our blog post ->show more

At this point, it's just ANYTHING-to-Video. Higgsfield Draw-to-Video gives... you a blank page to start off. You have TOTAL freedom over your result. Just drop ANY image, ANY text, or ANY product. The frame is YOURS. Retweet to get a full guide in DM. Check this out:show more

There’s a new way to bring your still images... to life on The Render Network You can now upload your own images – from renders and art to selfies, pets, or product shots & more using Image-to-Image or Image-to-Video Models on Render. Check it out at and start creating today!show more

Go from weeks of visual effects work to instant... visualization with Gen-4.5 Image to Video. Just drop an image in and ask for what you want. Looking for more tips? Check out our full collection of tutorial and guides at the link below.show more

A big thank you to the League Managers Association... for putting together this short video to update you on our work at the MacariFoundation The video was shown at this week's annual awards in London and reveals we can have a bit of fun despite the serious work...show more

The reception since we launched the first unified audio... + text + image to video model in production last week has been incredible! We wanted to take a moment and highlight some firsts: - the first omnimodal modal that jointly operates over text, image, and audio. Fun fact: we shipped our first audio-to-video model Character-1 last June! - Character-3 is generating 5 minute uncut clips, and capable of temporally stable longform prediction. - Character-3 handles realistic humans, cartoons, side profiles, dynamic backgrounds, anime, animals, and more. - Character-3 sets a new standard for fast & efficient video diffusion at HD and SD resolutions, leveraging our advancements in realtime video generation. We'll have an announcement soon, but you can read more about an earlier version of this line of work in our paper here:show more

"Getting that close and losing, it was not fun... at all. You can be more grateful, but our expectations are different. We use that for fuel...you can't get to June without going through all these other months. Worry about the now and get prepared." - Mikal Bridgesshow more

Introducing Agentic Object Detection! Given a text prompt like... “unripe strawberries” or “Kellogg’s branded cereal” and an image, we use an agentic workflow to reason at length and detect the specified objects. No need to label any training data. Watch the video for details.show more

Estimating our population in a changing world is complex,... in a new article we look at what long-term international migration is, and how we produce estimates that are as timely and accurate as possible. Watch this short video to find out more ⬇️show more

Still spending $500 per UGC ad? This was made... under $15 This golden period won't last long Capitalize when you can One product Image One character Image Full storyboarded video in <10 mins of work This is magic Comment "cooked" and I'll send the full workflow (must be following) P.S: This is just an AI production and in no way shows affiliation with any persons or products depicted in the mediashow more