Loading video...

Video Failed to Load

Go Home

Segment Anything is fun and powerful but still requires full mask annotation for supervised training. Our new work MaskFreeVIS appearing at #CVPR2023 shows we can get accurate video instance segmentation WITHOUT any video or image mask annotations. More at

DrFisherYu's profile picture

Fisher Yu

2,889 subscribers

41,103 views • 3 years ago •via X (Twitter)

EducationScience & Technology#CVPR2023
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Mask3D 🎭 is now at #ICRA2023, great work Jonas Schult! We use Mask Transformers for 3D Instance Segmentation on Point Clouds ~ 🥇 on ScanNet 📰Paper: 🛠️Project: 👨‍💻Code: @Pandoro89 Or Litany Siyu Tang @VLG-ETHZ
0:36

Mask3D 🎭 is now at #ICRA2023, great work Jonas Schult! We use Mask Transformers for 3D Instance Segmentation on Point Clouds ~ 🥇 on ScanNet 📰Paper: 🛠️Project: 👨‍💻Code: @Pandoro89 Or Litany Siyu Tang @VLG-ETHZ

Francis Engelmann ☀️ @CVPR'26

29,166 views • 3 years ago

Tracking Anything with Decoupled Video Segmentation paper page: Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.
0:32

Tracking Anything with Decoupled Video Segmentation paper page: Training data for video segmentation are expensive to annotate. This impedes extensions of end-to-end algorithms to new video segmentation tasks, especially in large-vocabulary settings. To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation. Due to this design, we only need an image-level model for the target task (which is cheaper to train) and a universal temporal propagation model which is trained once and generalizes across tasks. To effectively combine these two modules, we use bi-directional propagation for (semi-)online fusion of segmentation hypotheses from different frames to generate a coherent segmentation. We show that this decoupled formulation compares favorably to end-to-end approaches in several data-scarce tasks including large-vocabulary video panoptic segmentation, open-world video segmentation, referring video segmentation, and unsupervised video object segmentation.

AK

305,667 views • 2 years ago

VideoJAM is our new framework for improved motion generation from AI at Meta We show that video generators struggle with motion because the training objective favors appearance over dynamics. VideoJAM directly adresses this **without any extra data or scaling** 👇🧵
0:46

VideoJAM is our new framework for improved motion generation from AI at Meta We show that video generators struggle with motion because the training objective favors appearance over dynamics. VideoJAM directly adresses this **without any extra data or scaling** 👇🧵

Hila Chefer

168,657 views • 1 year ago

‘Lake Fever’ live from A National Celebration. The full video is out now on YouTube. Watch at We heard your requests for more live video recordings, and we listened. Subscribe to our YouTube to get a new song every Thursday.
0:29

‘Lake Fever’ live from A National Celebration. The full video is out now on YouTube. Watch at We heard your requests for more live video recordings, and we listened. Subscribe to our YouTube to get a new song every Thursday.

The Tragically Hip

11,897 views • 2 years ago

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch
0:22

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch

Bo Wang

178,539 views • 2 years ago

Training a Whole-Body Control Foundation Model -- new work from my team at Agility Robotics A neural network for controlling our humanoid robots which is robust to disturbances, can handle heavy objects, and is a powerful platform for learning new whole-body skills learn more in our blog post ->
1:56

Training a Whole-Body Control Foundation Model -- new work from my team at Agility Robotics A neural network for controlling our humanoid robots which is robust to disturbances, can handle heavy objects, and is a powerful platform for learning new whole-body skills learn more in our blog post ->

Chris Paxton

45,068 views • 11 months ago

At this point, it's just ANYTHING-to-Video. Higgsfield Draw-to-Video gives you a blank page to start off. You have TOTAL freedom over your result. Just drop ANY image, ANY text, or ANY product. The frame is YOURS. Retweet to get a full guide in DM. Check this out:
0:42

At this point, it's just ANYTHING-to-Video. Higgsfield Draw-to-Video gives you a blank page to start off. You have TOTAL freedom over your result. Just drop ANY image, ANY text, or ANY product. The frame is YOURS. Retweet to get a full guide in DM. Check this out:

Higgsfield AI 🧩

5,137,299 views • 11 months ago

🚨 🚨🚨 WE HAVE INCREASED THE CASH BOUNTY TO $1,500 🚨🚨🚨 Shirion Collective We have included newly revealed photos of her without a mask. If you have any information that can lead to identifying this young woman, please reach out. She was spotted at a counter protest at the Israel Day Parade in NYC holding her phone with a message attributed to the kidnapped hostages that read “They’re not coming home.” 🚨 🚨🚨 She was spotted after the counter protest was dispersed without a mask at a new counter protest. 🚨 🚨🚨 She was also spotted two days before at the Brooklyn Museum takeover by Nerdeen Kiswani and Within Our Lifetime, full face without a mask.
0:13

🚨 🚨🚨 WE HAVE INCREASED THE CASH BOUNTY TO $1,500 🚨🚨🚨 Shirion Collective We have included newly revealed photos of her without a mask. If you have any information that can lead to identifying this young woman, please reach out. She was spotted at a counter protest at the Israel Day Parade in NYC holding her phone with a message attributed to the kidnapped hostages that read “They’re not coming home.” 🚨 🚨🚨 She was spotted after the counter protest was dispersed without a mask at a new counter protest. 🚨 🚨🚨 She was also spotted two days before at the Brooklyn Museum takeover by Nerdeen Kiswani and Within Our Lifetime, full face without a mask.

Jew Hate Database

54,522 views • 2 years ago

There’s a new way to bring your still images to life on The Render Network You can now upload your own images – from renders and art to selfies, pets, or product shots & more using Image-to-Image or Image-to-Video Models on Render. Check it out at and start creating today!
0:37

There’s a new way to bring your still images to life on The Render Network You can now upload your own images – from renders and art to selfies, pets, or product shots & more using Image-to-Image or Image-to-Video Models on Render. Check it out at and start creating today!

The Render Network

38,029 views • 1 year ago

Go from weeks of visual effects work to instant visualization with Gen-4.5 Image to Video. Just drop an image in and ask for what you want. Looking for more tips? Check out our full collection of tutorial and guides at the link below.
0:35

Go from weeks of visual effects work to instant visualization with Gen-4.5 Image to Video. Just drop an image in and ask for what you want. Looking for more tips? Check out our full collection of tutorial and guides at the link below.

Runway

10,580 views • 6 months ago

A big thank you to the League Managers Association for putting together this short video to update you on our work at the MacariFoundation The video was shown at this week's annual awards in London and reveals we can have a bit of fun despite the serious work...
1:43

A big thank you to the League Managers Association for putting together this short video to update you on our work at the MacariFoundation The video was shown at this week's annual awards in London and reveals we can have a bit of fun despite the serious work...

Lou Macari

131,631 views • 3 years ago

The reception since we launched the first unified audio + text + image to video model in production last week has been incredible! We wanted to take a moment and highlight some firsts: - the first omnimodal modal that jointly operates over text, image, and audio. Fun fact: we shipped our first audio-to-video model Character-1 last June! - Character-3 is generating 5 minute uncut clips, and capable of temporally stable longform prediction. - Character-3 handles realistic humans, cartoons, side profiles, dynamic backgrounds, anime, animals, and more. - Character-3 sets a new standard for fast & efficient video diffusion at HD and SD resolutions, leveraging our advancements in realtime video generation. We'll have an announcement soon, but you can read more about an earlier version of this line of work in our paper here:
0:55

The reception since we launched the first unified audio + text + image to video model in production last week has been incredible! We wanted to take a moment and highlight some firsts: - the first omnimodal modal that jointly operates over text, image, and audio. Fun fact: we shipped our first audio-to-video model Character-1 last June! - Character-3 is generating 5 minute uncut clips, and capable of temporally stable longform prediction. - Character-3 handles realistic humans, cartoons, side profiles, dynamic backgrounds, anime, animals, and more. - Character-3 sets a new standard for fast & efficient video diffusion at HD and SD resolutions, leveraging our advancements in realtime video generation. We'll have an announcement soon, but you can read more about an earlier version of this line of work in our paper here:

Hedra

180,676 views • 1 year ago

"Getting that close and losing, it was not fun at all. You can be more grateful, but our expectations are different. We use that for fuel...you can't get to June without going through all these other months. Worry about the now and get prepared." - Mikal Bridges
0:49

"Getting that close and losing, it was not fun at all. You can be more grateful, but our expectations are different. We use that for fuel...you can't get to June without going through all these other months. Worry about the now and get prepared." - Mikal Bridges

Knicks Videos

37,047 views • 10 months ago

Introducing Agentic Object Detection! Given a text prompt like “unripe strawberries” or “Kellogg’s branded cereal” and an image, we use an agentic workflow to reason at length and detect the specified objects. No need to label any training data. Watch the video for details.
2:09

Introducing Agentic Object Detection! Given a text prompt like “unripe strawberries” or “Kellogg’s branded cereal” and an image, we use an agentic workflow to reason at length and detect the specified objects. No need to label any training data. Watch the video for details.

Andrew Ng

397,877 views • 1 year ago

Estimating our population in a changing world is complex, in a new article we look at what long-term international migration is, and how we produce estimates that are as timely and accurate as possible. Watch this short video to find out more ⬇️
2:02

Estimating our population in a changing world is complex, in a new article we look at what long-term international migration is, and how we produce estimates that are as timely and accurate as possible. Watch this short video to find out more ⬇️

Office for National Statistics (ONS)

11,575 views • 8 months ago

Still spending $500 per UGC ad? This was made under $15 This golden period won't last long Capitalize when you can One product Image One character Image Full storyboarded video in <10 mins of work This is magic Comment "cooked" and I'll send the full workflow (must be following) P.S: This is just an AI production and in no way shows affiliation with any persons or products depicted in the media
0:34

Still spending $500 per UGC ad? This was made under $15 This golden period won't last long Capitalize when you can One product Image One character Image Full storyboarded video in <10 mins of work This is magic Comment "cooked" and I'll send the full workflow (must be following) P.S: This is just an AI production and in no way shows affiliation with any persons or products depicted in the media

HeyOz

12,666 views • 2 months ago

INSIDE TRAINING: THE BRAZILIANS SET UP CAMP IN RUSTENBURG. 💪🏃‍ Get an inside look at the hard work going into preparations for the new campaign. 🙌 Watch the full video now on our YouTube channel. 📲 #Sundowns
0:30

INSIDE TRAINING: THE BRAZILIANS SET UP CAMP IN RUSTENBURG. 💪🏃‍ Get an inside look at the hard work going into preparations for the new campaign. 🙌 Watch the full video now on our YouTube channel. 📲 #Sundowns

Mamelodi Sundowns FC

18,067 views • 1 year ago