Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

sam.cpp 👀 Inference of Meta's Segment Anything Model on the CPU Project by Yavor Ivanov - powered by

ggerganov's profile picture

Georgi Gerganov

53,881 subscribers

264,542 Aufrufe • vor 2 Jahren •via X (Twitter)

BildungGamingWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Georgi Gerganov
Georgi Gerganovvor 2 Jahren

repo: The project is still in it's early stages, but @YavorGI has already done a great job with the implementation. Please check it out and let's help him bring SOTA image segmentation to our devices!

Profilbild von Georgi Gerganov
Georgi Gerganovvor 2 Jahren

Is this what the Terminator used 🤔

Profilbild von kache
kachevor 2 Jahren

@YavorGI amazing!

Profilbild von Kenneth Cassel
Kenneth Casselvor 2 Jahren

@YavorGI whoa looks pretty fast. we've been using FastSAM for faster SAM inference. but it has slightly worse perf than SAM excited to check this out 👀

Profilbild von Georgi Gerganov
Georgi Gerganovvor 2 Jahren

@YavorGI Speed is not there yet, but lots of room for improvement. Should by OK an Apple Silicon I think

Profilbild von bryan pratte
bryan prattevor 2 Jahren

@YavorGI What is the inference time per frame end to end?

Profilbild von Mevlut Cafer Arslan
Mevlut Cafer Arslanvor 2 Jahren

@YavorGI Vision Pro SDK + this + a model that can extract information from segmentations 🥹

Profilbild von 𝕏 NiNo
𝕏 NiNovor 2 Jahren

@yacineMTB @YavorGI @Scobleizer 👀 with AppleVisionPro and then SegmentationsInfos…. 🔥

Profilbild von Nikhila Ravi
Nikhila Ravivor 2 Jahren

@YavorGI Very cool! What's the breakdown in the time for 1) embedding extraction per image and 2) mask prediction per prompt?

Profilbild von Mayowa Daniel
Mayowa Danielvor 2 Jahren

@YavorGI Exciting! Just whipped this up and it got merged so quickly too! Trying to get my feet wet with the whole ai thing Been a minute since I touched c++ so I’m looking forward to good first issues and open to being a fly on the wall/pair 🤲🏽

Ähnliche Videos

Detecting airplanes at DFW Airport from a single text prompt. Meta's Segment Anything Model 3 and the geosam R package make powerful image detection tools accessible.
0:51

Detecting airplanes at DFW Airport from a single text prompt. Meta's Segment Anything Model 3 and the geosam R package make powerful image detection tools accessible.

Kyle Walker

31,475 Aufrufe • vor 5 Monaten

📣 MLPerf Inference v6.0 results are in. Learn how systems powered by NVIDIA Blackwell set the pace on inference, delivering the highest AI factory throughput. 🔗
1:02

📣 MLPerf Inference v6.0 results are in. Learn how systems powered by NVIDIA Blackwell set the pace on inference, delivering the highest AI factory throughput. 🔗

NVIDIA Data Center

33,408 Aufrufe • vor 3 Monaten

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!
0:56

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,622 Aufrufe • vor 2 Jahren

Introducing the NVIDIA Project DIGITS personal AI supercomputer, powered by the GB10 Grace Blackwell Superchip: Project DIGITS enables developers to prototype, fine-tune and inference models locally and seamlessly deploy at scale to the data center or cloud. #CES2025 #NVIDIAGraceBlackwell
0:23

Introducing the NVIDIA Project DIGITS personal AI supercomputer, powered by the GB10 Grace Blackwell Superchip: Project DIGITS enables developers to prototype, fine-tune and inference models locally and seamlessly deploy at scale to the data center or cloud. #CES2025 #NVIDIAGraceBlackwell

NVIDIA

51,991 Aufrufe • vor 1 Jahr

Unbiased AI -- at Scale Truly Open, inference at scale, running on Subnet 4. Powered by the Bittensor ecosystem. Full episode on Youtube and Spotify:
0:57

Unbiased AI -- at Scale Truly Open, inference at scale, running on Subnet 4. Powered by the Bittensor ecosystem. Full episode on Youtube and Spotify:

Openτensor Foundaτion

45,393 Aufrufe • vor 1 Jahr

Introducing AI Click powered by Segment Anything (SAM) for biological images, on Biodock. We’ve worked hard to make SAM much more useful for bio - read below to find out how!
0:44

Introducing AI Click powered by Segment Anything (SAM) for biological images, on Biodock. We’ve worked hard to make SAM much more useful for bio - read below to find out how!

Michael

57,577 Aufrufe • vor 3 Jahren

Introducing FOMO - FLock Open Model Offering The full-cycle decentralized AI model economy is now on Base, powered by FLock.
1:37

Introducing FOMO - FLock Open Model Offering The full-cycle decentralized AI model economy is now on Base, powered by FLock.

FLock.io

65,942 Aufrufe • vor 5 Monaten

Fund your Kalshi account with Bitcoin, trade on anything. Powered by @ZeroHashX
0:39

Fund your Kalshi account with Bitcoin, trade on anything. Powered by @ZeroHashX

Kalshi

86,126 Aufrufe • vor 1 Jahr

Super stoked that Jordão Bragantini 's napari plugin for Meta's Segment Anything Model 2: napari-sam2, as well as our cell tracking software ultrack were showcased during Chris Cox's Keynote at Meta's Connect 2024. Jordao and I were in the audience; it was great fun! Chan Zuckerberg Biohub Network
0:56

Super stoked that Jordão Bragantini 's napari plugin for Meta's Segment Anything Model 2: napari-sam2, as well as our cell tracking software ultrack were showcased during Chris Cox's Keynote at Meta's Connect 2024. Jordao and I were in the audience; it was great fun! Chan Zuckerberg Biohub Network

Loïc A. Royer 💻🔬⚗️

41,060 Aufrufe • vor 1 Jahr

The Segment Anything Model (SAM) by Meta AI is a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from photos or videos + zero-shot transfer to other segmentation tasks. Try the demo ➡️
0:51

The Segment Anything Model (SAM) by Meta AI is a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from photos or videos + zero-shot transfer to other segmentation tasks. Try the demo ➡️

AI at Meta

186,348 Aufrufe • vor 3 Jahren

THE TAKEDOWN: AOC losing to facts in today's Takedown segment powered by new search partner Perplexity
4:23

THE TAKEDOWN: AOC losing to facts in today's Takedown segment powered by new search partner Perplexity

The Ben Shapiro Show

169,095 Aufrufe • vor 1 Jahr

Private Uniswap is fully functional with The Anon Project. Powered by RAILGUN.
0:59

Private Uniswap is fully functional with The Anon Project. Powered by RAILGUN.

RAILGUN - Private Ethereum DeFi

22,248 Aufrufe • vor 1 Monat

Done. Long pending demand of people of #Koraput Parliament Segment: Gunupur - Therubali New Railway Line Project approved by the Central Government. 🙏
1:01

Done. Long pending demand of people of #Koraput Parliament Segment: Gunupur - Therubali New Railway Line Project approved by the Central Government. 🙏

Saptagiri Ulaka

14,151 Aufrufe • vor 1 Jahr

Releasing moondream-zig! It is a fast, implementation of moondream2 inference on the CPU written from-scratch in Zig :) moondream-zig provides 1.5-2x faster inference compared to huggingface on the same device. moondream vik
1:21

Releasing moondream-zig! It is a fast, implementation of moondream2 inference on the CPU written from-scratch in Zig :) moondream-zig provides 1.5-2x faster inference compared to huggingface on the same device. moondream vik

snow

29,245 Aufrufe • vor 1 Jahr

Glimpse of the model houses for resettlement of people who are affected by Parandur Greenfield Airport Project. A township just near the project site!!
1:30

Glimpse of the model houses for resettlement of people who are affected by Parandur Greenfield Airport Project. A township just near the project site!!

Chennai Updates

34,074 Aufrufe • vor 6 Monaten

RESIDENT MOUTHFUL Ashley Graham/POV Vore Animation. Apparently, she's more prepared to do anything to survive than we thought 🫦👀 Animation Collab with CEO of EatFun (Appdattta) Ashley Model by MokujinH🔞 Esophagus Model by meknes Stomach Model by GarrisenGameArt🔞🎮🎨
1:38

Sensitive content

RESIDENT MOUTHFUL Ashley Graham/POV Vore Animation. Apparently, she's more prepared to do anything to survive than we thought 🫦👀 Animation Collab with CEO of EatFun (Appdattta) Ashley Model by MokujinH🔞 Esophagus Model by meknes Stomach Model by GarrisenGameArt🔞🎮🎨

Paramouth Pictures

72,978 Aufrufe • vor 3 Jahren

A new breed, inspired by Sci-Fi and the shape of water, powered by technology. *Model shown is the XPENG G6 for the Chinese market. #XPENGG6 #TechnologyForAll
2:23

A new breed, inspired by Sci-Fi and the shape of water, powered by technology. *Model shown is the XPENG G6 for the Chinese market. #XPENGG6 #TechnologyForAll

XPENG

369,223 Aufrufe • vor 3 Jahren