正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

sam.cpp 👀 Inference of Meta's Segment Anything Model on the CPU Project by Yavor Ivanov - powered by

Georgi Gerganov

53,881 subscribers

264,542 次观看 • 2 年前 •via X (Twitter)

教育游戏科学技术

Anya Rossi• Live Now

Private livecam show

10 条评论

Georgi Gerganov 的头像

Georgi Gerganov2 年前

repo: The project is still in it's early stages, but @YavorGI has already done a great job with the implementation. Please check it out and let's help him bring SOTA image segmentation to our devices!

Georgi Gerganov 的头像

Georgi Gerganov2 年前

Is this what the Terminator used 🤔

kache 的头像

kache2 年前

@YavorGI amazing!

Kenneth Cassel 的头像

Kenneth Cassel2 年前

@YavorGI whoa looks pretty fast. we've been using FastSAM for faster SAM inference. but it has slightly worse perf than SAM excited to check this out 👀

Georgi Gerganov 的头像

Georgi Gerganov2 年前

@YavorGI Speed is not there yet, but lots of room for improvement. Should by OK an Apple Silicon I think

bryan pratte 的头像

bryan pratte2 年前

@YavorGI What is the inference time per frame end to end?

Mevlut Cafer Arslan 的头像

Mevlut Cafer Arslan2 年前

@YavorGI Vision Pro SDK + this + a model that can extract information from segmentations 🥹

𝕏 NiNo 的头像

𝕏 NiNo2 年前

@yacineMTB @YavorGI @Scobleizer 👀 with AppleVisionPro and then SegmentationsInfos…. 🔥

Nikhila Ravi 的头像

Nikhila Ravi2 年前

@YavorGI Very cool! What's the breakdown in the time for 1) embedding extraction per image and 2) mask prediction per prompt?

Mayowa Daniel 的头像

Mayowa Daniel2 年前

@YavorGI Exciting! Just whipped this up and it got merged so quickly too! Trying to get my feet wet with the whole ai thing Been a minute since I touched c++ so I’m looking forward to good first issues and open to being a fly on the wall/pair 🤲🏽

相关视频

Detecting airplanes at DFW Airport from a single text prompt. Meta's Segment Anything Model 3 and the geosam R package make powerful image detection tools accessible.

Detecting airplanes at DFW Airport from a single text prompt. Meta's Segment Anything Model 3 and the geosam R package make powerful image detection tools accessible.

Kyle Walker

31,475 次观看 • 5 个月前

📣 MLPerf Inference v6.0 results are in. Learn how systems powered by NVIDIA Blackwell set the pace on inference, delivering the highest AI factory throughput. 🔗

📣 MLPerf Inference v6.0 results are in. Learn how systems powered by NVIDIA Blackwell set the pace on inference, delivering the highest AI factory throughput. 🔗

NVIDIA Data Center

33,408 次观看 • 2 个月前

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,583 次观看 • 2 年前

Introducing the NVIDIA Project DIGITS personal AI supercomputer, powered by the GB10 Grace Blackwell Superchip: Project DIGITS enables developers to prototype, fine-tune and inference models locally and seamlessly deploy at scale to the data center or cloud. #CES2025 #NVIDIAGraceBlackwell

Introducing the NVIDIA Project DIGITS personal AI supercomputer, powered by the GB10 Grace Blackwell Superchip: Project DIGITS enables developers to prototype, fine-tune and inference models locally and seamlessly deploy at scale to the data center or cloud. #CES2025 #NVIDIAGraceBlackwell

NVIDIA

51,983 次观看 • 1 年前

#M5StackNew 🎊 The LLM630 Compute Kit is an #AI large language model (#LLM) inference development kit, powered by the #Axera #AX630C SoC with a 3.2 TOPs NPU, it delivers efficient AI inference for tasks like computer vision (CV) and LLM processing.

#M5StackNew 🎊 The LLM630 Compute Kit is an #AI large language model (#LLM) inference development kit, powered by the #Axera #AX630C SoC with a 3.2 TOPs NPU, it delivers efficient AI inference for tasks like computer vision (CV) and LLM processing.

M5Stack

16,383 次观看 • 1 年前

Unbiased AI -- at Scale Truly Open, inference at scale, running on Subnet 4. Powered by the Bittensor ecosystem. Full episode on Youtube and Spotify:

Unbiased AI -- at Scale Truly Open, inference at scale, running on Subnet 4. Powered by the Bittensor ecosystem. Full episode on Youtube and Spotify:

Openτensor Foundaτion

45,393 次观看 • 1 年前

Introducing AI Click powered by Segment Anything (SAM) for biological images, on Biodock. We’ve worked hard to make SAM much more useful for bio - read below to find out how!

Introducing AI Click powered by Segment Anything (SAM) for biological images, on Biodock. We’ve worked hard to make SAM much more useful for bio - read below to find out how!

Michael

57,576 次观看 • 3 年前

Introducing FOMO - FLock Open Model Offering The full-cycle decentralized AI model economy is now on Base, powered by FLock.

Introducing FOMO - FLock Open Model Offering The full-cycle decentralized AI model economy is now on Base, powered by FLock.

FLock.io

65,838 次观看 • 5 个月前

Super stoked that Jordão Bragantini 's napari plugin for Meta's Segment Anything Model 2: napari-sam2, as well as our cell tracking software ultrack were showcased during Chris Cox's Keynote at Meta's Connect 2024. Jordao and I were in the audience; it was great fun! Chan Zuckerberg Biohub Network

Super stoked that Jordão Bragantini 's napari plugin for Meta's Segment Anything Model 2: napari-sam2, as well as our cell tracking software ultrack were showcased during Chris Cox's Keynote at Meta's Connect 2024. Jordao and I were in the audience; it was great fun! Chan Zuckerberg Biohub Network

Loïc A. Royer 💻🔬⚗️

41,045 次观看 • 1 年前

The Segment Anything Model (SAM) by Meta AI is a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from photos or videos + zero-shot transfer to other segmentation tasks. Try the demo ➡️

The Segment Anything Model (SAM) by Meta AI is a step toward the first foundation model for image segmentation. SAM is capable of one-click segmentation of any object from photos or videos + zero-shot transfer to other segmentation tasks. Try the demo ➡️

AI at Meta

186,324 次观看 • 3 年前

Fund your Kalshi account with Bitcoin, trade on anything. Powered by @ZeroHashX

Fund your Kalshi account with Bitcoin, trade on anything. Powered by @ZeroHashX

Kalshi

86,126 次观看 • 1 年前

THE TAKEDOWN: AOC losing to facts in today's Takedown segment powered by new search partner Perplexity

THE TAKEDOWN: AOC losing to facts in today's Takedown segment powered by new search partner Perplexity

The Ben Shapiro Show

169,095 次观看 • 1 年前

Private Uniswap is fully functional with The Anon Project. Powered by RAILGUN.

Private Uniswap is fully functional with The Anon Project. Powered by RAILGUN.

RAILGUN - Private Ethereum DeFi

22,050 次观看 • 27 天前

Releasing moondream-zig! It is a fast, implementation of moondream2 inference on the CPU written from-scratch in Zig :) moondream-zig provides 1.5-2x faster inference compared to huggingface on the same device. moondream vik

Releasing moondream-zig! It is a fast, implementation of moondream2 inference on the CPU written from-scratch in Zig :) moondream-zig provides 1.5-2x faster inference compared to huggingface on the same device. moondream vik

snow

29,245 次观看 • 1 年前

Done. Long pending demand of people of #Koraput Parliament Segment: Gunupur - Therubali New Railway Line Project approved by the Central Government. 🙏

Done. Long pending demand of people of #Koraput Parliament Segment: Gunupur - Therubali New Railway Line Project approved by the Central Government. 🙏

Saptagiri Ulaka

14,151 次观看 • 1 年前

Glimpse of the model houses for resettlement of people who are affected by Parandur Greenfield Airport Project. A township just near the project site!!

Glimpse of the model houses for resettlement of people who are affected by Parandur Greenfield Airport Project. A township just near the project site!!

Chennai Updates

34,074 次观看 • 5 个月前

RESIDENT MOUTHFUL Ashley Graham/POV Vore Animation. Apparently, she's more prepared to do anything to survive than we thought 🫦👀 Animation Collab with CEO of EatFun (Appdattta) Ashley Model by MokujinH🔞 Esophagus Model by meknes Stomach Model by GarrisenGameArt🔞🎮🎨

Sensitive content

RESIDENT MOUTHFUL Ashley Graham/POV Vore Animation. Apparently, she's more prepared to do anything to survive than we thought 🫦👀 Animation Collab with CEO of EatFun (Appdattta) Ashley Model by MokujinH🔞 Esophagus Model by meknes Stomach Model by GarrisenGameArt🔞🎮🎨

Paramouth Pictures

72,978 次观看 • 3 年前

Create worlds at the speed of thought. Powered by World Engine. Waitlist coming soon. 👀

Create worlds at the speed of thought. Powered by World Engine. Waitlist coming soon. 👀

NeuralAI

23,991 次观看 • 7 个月前