Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Casually running a 180B parameter LLM on M2 Ultra

ggerganov's profile picture

Georgi Gerganov

61,741 subscribers

683,186 görüntüleme • 2 yıl önce •via X (Twitter)

Bilim & TeknolojiEğitimSanat
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

🌟 Introducing Falcon 180B: The World's Most Powerful Open LLM! 🚀 At #TII, we are continuing to push the boundaries of generative AI with our open access Falcon 180B AI model which has already soared to the top of the Hugging Face Leaderboard.
1:01

🌟 Introducing Falcon 180B: The World's Most Powerful Open LLM! 🚀 At #TII, we are continuing to push the boundaries of generative AI with our open access Falcon 180B AI model which has already soared to the top of the Hugging Face Leaderboard.

Technology Innovation Institute

1,296,137 görüntüleme • 2 yıl önce

Full F16 precision 34B Code Llama at >20 t/s on M2 Ultra
0:25

Full F16 precision 34B Code Llama at >20 t/s on M2 Ultra

Georgi Gerganov

1,157,414 görüntüleme • 2 yıl önce

The video is a Llama v1 7B model implemented in MLX and running on an M2 Ultra. More here: * Train a Transformer LM or fine-tune with LoRA * Text generation with Mistral * Image generation with Stable Diffusion * Speech recognition with Whisper
1:35

The video is a Llama v1 7B model implemented in MLX and running on an M2 Ultra. More here: * Train a Transformer LM or fine-tune with LoRA * Text generation with Mistral * Image generation with Stable Diffusion * Speech recognition with Whisper

Awni Hannun

66,565 görüntüleme • 2 yıl önce

TENCENT casually dropping a 13bn parameter open sourced video model WITH weights 🩵 Looks unreal and can't wait to test it
1:23

TENCENT casually dropping a 13bn parameter open sourced video model WITH weights 🩵 Looks unreal and can't wait to test it

A.I.Warper

156,356 görüntüleme • 1 yıl önce

Bro is casually out running a nuclear blast
1:21

Bro is casually out running a nuclear blast

Anime Posts

4,214,560 görüntüleme • 2 ay önce

What would it look like to combine Google search (shallow Knowledge Graph reasoning over an ultra-ultra-wide index) with LLM in-context learning (highly intelligent operations on a tiny index)?
2:49

What would it look like to combine Google search (shallow Knowledge Graph reasoning over an ultra-ultra-wide index) with LLM in-context learning (highly intelligent operations on a tiny index)?

Dwarkesh Patel

100,546 görüntüleme • 1 yıl önce

This looks like a test drive unit of the BMW M2. The driver is casually doing donuts on a busy public road in Noida, not cool, it’s reckless.
1:27

This looks like a test drive unit of the BMW M2. The driver is casually doing donuts on a busy public road in Noida, not cool, it’s reckless.

AARIZ RIZVI

747,357 görüntüleme • 11 ay önce

Agentic LLM NPCs with multi-user conversation all running on consumer-grade local hardware
1:31

Agentic LLM NPCs with multi-user conversation all running on consumer-grade local hardware

stochastics

24,261 görüntüleme • 1 yıl önce

What a way to close out the Ultra afterparty at M2 Miami B2B with the amazing Nicole Moudaber. That’s a wrap on Miami Music Week! 🖤
0:32

What a way to close out the Ultra afterparty at M2 Miami B2B with the amazing Nicole Moudaber. That’s a wrap on Miami Music Week! 🖤

Carl Cox

21,272 görüntüleme • 1 yıl önce

EMF Radiation Test Shows Sitting in an Electric Car is Like Standing Near a Running Microwave - Cell Phone: 120 mW/m2 - Microwave: 360 mW/m2 - Microwave (1-2’ Away): 100 mW/m2 - Inside Tesla: 100 mW/m2 Full Video:
1:43

EMF Radiation Test Shows Sitting in an Electric Car is Like Standing Near a Running Microwave - Cell Phone: 120 mW/m2 - Microwave: 360 mW/m2 - Microwave (1-2’ Away): 100 mW/m2 - Inside Tesla: 100 mW/m2 Full Video:

Chief Nerd

103,747 görüntüleme • 3 yıl önce

Finally found a meaningful scenario for running LLM in the browser, peering into the internals of LLM to see what's really happening inside
0:20

Finally found a meaningful scenario for running LLM in the browser, peering into the internals of LLM to see what's really happening inside

Broooooklyn

45,536 görüntüleme • 27 gün önce

Try now Hyperfast LLM running on custom built GPUs Answers in miliseconds, not seconds How? 🤯
0:39

Try now Hyperfast LLM running on custom built GPUs Answers in miliseconds, not seconds How? 🤯

@levelsio

602,378 görüntüleme • 2 yıl önce

An Ultra-Orthodox Jewish man casually takes a shit in the Armenian Quarter of Jerusalem.
0:30

Sensitive content

An Ultra-Orthodox Jewish man casually takes a shit in the Armenian Quarter of Jerusalem.

Clash Report

1,150,073 görüntüleme • 1 ay önce

casually running into Peeves bullying students on their way to class in #HogwartLegacy 🤣
0:48

casually running into Peeves bullying students on their way to class in #HogwartLegacy 🤣

TmarTn

199,414 görüntüleme • 3 yıl önce

Local voice AI with a 235 billion parameter LLM. ✅ - smart-turn v2 - MLX Whisper (large-v3-turbo-q4) - Qwen3-235B-A22B-Instruct-2507-3bit-DWQ - Kokoro All models running local on an M4 mac. Max RAM usage ~110GB. Voice-to-voice latency is ~950ms. There are a couple of relatively easy ways to carve another ~100ms off that number. But it's not a bad start!
3:24

Local voice AI with a 235 billion parameter LLM. ✅ - smart-turn v2 - MLX Whisper (large-v3-turbo-q4) - Qwen3-235B-A22B-Instruct-2507-3bit-DWQ - Kokoro All models running local on an M4 mac. Max RAM usage ~110GB. Voice-to-voice latency is ~950ms. There are a couple of relatively easy ways to carve another ~100ms off that number. But it's not a bad start!

kwindla

66,639 görüntüleme • 11 ay önce

I sent an AI computer to the edge of space! HAAID-1 was the first-of-its-kind high-altitude balloon mission. It reached an altitude of ~19 km and ran a 360M-parameter LLM on a Raspberry Pi at -68°C.
2:35

I sent an AI computer to the edge of space! HAAID-1 was the first-of-its-kind high-altitude balloon mission. It reached an altitude of ~19 km and ran a 360M-parameter LLM on a Raspberry Pi at -68°C.

Yatharth Mann

108,594 görüntüleme • 20 gün önce

good morning /v1/chat/completions This is a test we ran overnight on TensorRT-LLM with modified kernels serving a custom 1B parameter model we trained for a customer ~200ms end-to-end latency (not TTFB, full request). Beats their current Cerebras stack on latency and quality
0:23

good morning /v1/chat/completions This is a test we ran overnight on TensorRT-LLM with modified kernels serving a custom 1B parameter model we trained for a customer ~200ms end-to-end latency (not TTFB, full request). Beats their current Cerebras stack on latency and quality

Sam Hogan 🇺🇸

43,655 görüntüleme • 4 ay önce

Roman Reigns just casually whipping out Ultra Instinct in #WWE2K25 😮
0:31

Roman Reigns just casually whipping out Ultra Instinct in #WWE2K25 😮

GameVolt

105,899 görüntüleme • 6 ay önce

Okay, this is actually insane... You can now run LFM2.5-1.2B-Thinking (a 1.2B parameter LLM from @LiquidAI) at over 200 tokens per second directly in your browser on WebGPU! 🤯 Zero install. Fully private. Blazingly fast. Powered by Transformers.js and ONNX Runtime Web
0:45

Okay, this is actually insane... You can now run LFM2.5-1.2B-Thinking (a 1.2B parameter LLM from @LiquidAI) at over 200 tokens per second directly in your browser on WebGPU! 🤯 Zero install. Fully private. Blazingly fast. Powered by Transformers.js and ONNX Runtime Web

Xenova

103,001 görüntüleme • 4 ay önce