Загрузка видео...

Не удалось загрузить видео

На главную

PRIMA.CPP Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday Home Clusters

_akhaliq's profile picture

AK

408,893 subscribers

48,241 просмотров • 1 год назад •via X (Twitter)

Новости и политикаНаука и технологииОбразование
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Introducing DeepThought-8B: Transparent reasoning model built on LLaMA-3.1 with test-time compute scaling. - JSON-structured thought chains & controllable inference paths. - ~16GB VRAM, competitive w/ 70B models. - Open model weights, and inference scripts.
0:16

Introducing DeepThought-8B: Transparent reasoning model built on LLaMA-3.1 with test-time compute scaling. - JSON-structured thought chains & controllable inference paths. - ~16GB VRAM, competitive w/ 70B models. - Open model weights, and inference scripts.

Ruliad

219,315 просмотров • 1 год назад

The new low battery animation in Dynamic Island is satisfyingly smooth, speeding up then slowing down!
0:11

The new low battery animation in Dynamic Island is satisfyingly smooth, speeding up then slowing down!

Minimal Nerd

152,421 просмотров • 11 месяцев назад

BNBCHAIN inference campaign LIVE: easily access 30+ models top up the Bankr LLM Gateway with $5 USDT or any other token on BNB Chain, get +$5 bonus credits dropped into your balance.
0:11

BNBCHAIN inference campaign LIVE: easily access 30+ models top up the Bankr LLM Gateway with $5 USDT or any other token on BNB Chain, get +$5 bonus credits dropped into your balance.

Bankr

48,980 просмотров • 1 месяц назад

Llama 2: Now on Hugging Chat 🤗🦙 Try out the 70B Chat model for free with super fast inference, web search, and powered by open-source tools! 👉
0:11

Llama 2: Now on Hugging Chat 🤗🦙 Try out the 70B Chat model for free with super fast inference, web search, and powered by open-source tools! 👉

Hugging Face

403,581 просмотров • 3 лет назад

Microsoft killed the GPU mafia 🤯 They finally open-sourced their 1-bit LLM inference framework called bitnet.cpp. It lets you run 100B parameter models on your local CPU without GPUs. - 6.17x faster inference - 82.2% less energy on CPUs 100% Open Source.
0:13

Microsoft killed the GPU mafia 🤯 They finally open-sourced their 1-bit LLM inference framework called bitnet.cpp. It lets you run 100B parameter models on your local CPU without GPUs. - 6.17x faster inference - 82.2% less energy on CPUs 100% Open Source.

Oliver Prompts

1,628,066 просмотров • 5 месяцев назад

private inference is live on the bankr llm gateway. select models now run inside TEEs. prompts stay encrypted in use -- private, by design. pay with any coin you’re holding. get started below.
0:15

private inference is live on the bankr llm gateway. select models now run inside TEEs. prompts stay encrypted in use -- private, by design. pay with any coin you’re holding. get started below.

Bankr

26,998 просмотров • 14 дней назад

Poseidon needs voice data and reliable ground truth in low-resource languages to benchmark against. To ensure LLM transcript accuracy, we worked with linguists to audit Bengali outputs. For a language spoken by 280M people, the gaps we found point to a deeper issue: data ↓
0:16

Poseidon needs voice data and reliable ground truth in low-resource languages to benchmark against. To ensure LLM transcript accuracy, we worked with linguists to audit Bengali outputs. For a language spoken by 280M people, the gaps we found point to a deeper issue: data ↓

Poseidon

22,705 просмотров • 4 месяцев назад

🇻🇪 DAY 2: Everyday working normal Venezuelans stills stock up on food and water, then keep their heads low at home. Everyone is afraid of escalation, situation is unclear. There were minor protest in support of Maduro. The fully government of Maduro ia still in power.
0:15

🇻🇪 DAY 2: Everyday working normal Venezuelans stills stock up on food and water, then keep their heads low at home. Everyone is afraid of escalation, situation is unclear. There were minor protest in support of Maduro. The fully government of Maduro ia still in power.

Lord Bebo

23,581 просмотров • 6 месяцев назад

Fireworks blazing fast LLM inference is now available on Poe! Today, we’re excited to bring the power of the new Mistral 7B Instruct model to the Poe powered by our lightning-fast Fireworks inference platform. You can now have conversations with the Mistral 7B bot and even build your own bots on top of it. Try it here:
0:19

Fireworks blazing fast LLM inference is now available on Poe! Today, we’re excited to bring the power of the new Mistral 7B Instruct model to the Poe powered by our lightning-fast Fireworks inference platform. You can now have conversations with the Mistral 7B bot and even build your own bots on top of it. Try it here:

Fireworks AI

93,334 просмотров • 2 лет назад

pip install spectralquant ✂️ Up to 6.62x KV cache compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B / Llama 3.1 8B verified - Pure PyTorch + future CUDA kernel support - Auto-calibration from a bundled corpus 📰 Paper: 💻 Code, quickstart, and benchmarks: #LLM #Inference #PyTorch #OpenSource #MachineLearning #LLM #KVCache #Inference
0:15

pip install spectralquant ✂️ Up to 6.62x KV cache compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B / Llama 3.1 8B verified - Pure PyTorch + future CUDA kernel support - Auto-calibration from a bundled corpus 📰 Paper: 💻 Code, quickstart, and benchmarks: #LLM #Inference #PyTorch #OpenSource #MachineLearning #LLM #KVCache #Inference

ani

16,583 просмотров • 1 месяц назад

🚨 TIME IS SPEEDING UP 🚨 Roku is showing you truth in plain sight about time speeding up. There has been a lot of talk recently about time speeding up and everything feeling off...
0:13

🚨 TIME IS SPEEDING UP 🚨 Roku is showing you truth in plain sight about time speeding up. There has been a lot of talk recently about time speeding up and everything feeling off...

Truth Seeker

142,040 просмотров • 5 месяцев назад

Speeding it up ⚡️ #GIRLSET #Tweak
0:14

Speeding it up ⚡️ #GIRLSET #Tweak

GIRLSET

14,001 просмотров • 4 месяцев назад

Prompting is cool and all, but isn't it a waste of compute to encode a prompt over and over again? We learn to compress prompts up to 26x by using "gist tokens", saving memory+storage and speeding up LM inference: (w/ Xiang Lisa Li and noahdgoodman) 🧵
0:13

Prompting is cool and all, but isn't it a waste of compute to encode a prompt over and over again? We learn to compress prompts up to 26x by using "gist tokens", saving memory+storage and speeding up LM inference: (w/ Xiang Lisa Li and noahdgoodman) 🧵

Jesse Mu

160,682 просмотров • 3 лет назад

Time is speeding up it self 🤔
0:14

Time is speeding up it self 🤔

JOKAMRREDPILLZ

78,950 просмотров • 6 месяцев назад

Iterating... Evals take time and are boring: but are a fundamental validation step of sane LLM inference. Let's try to make them as easy and fun to run as possible.
0:10

Iterating... Evals take time and are boring: but are a fundamental validation step of sane LLM inference. Let's try to make them as easy and fun to run as possible.

antirez

32,727 просмотров • 1 месяц назад

Recording at some tacky store she hits a new low everyday
0:14

Sensitive content

Recording at some tacky store she hits a new low everyday

ِ

15,100 просмотров • 2 месяцев назад

Today's Hot Deals Home living essentials you'll love starting at low price + FREE SHIPPING! SEE your future home> Take it on home
0:12

Today's Hot Deals Home living essentials you'll love starting at low price + FREE SHIPPING! SEE your future home> Take it on home

SHEIN

1,534,134 просмотров • 1 год назад

Viral video circulating shows School resource officer putting student in check after he walked up on him
0:17

Viral video circulating shows School resource officer putting student in check after he walked up on him

Shannonnn sharpes Burner (PARODY Account)

941,527 просмотров • 1 год назад

Low-precision floats are weird. I have been building up my intuition by playing with them outside of inference/training. Adam Azzam and I cooked up this visualizer for micro-scaling/block quant formats like NVFP4, MXFP4, and friends. Try it:
0:10

Low-precision floats are weird. I have been building up my intuition by playing with them outside of inference/training. Adam Azzam and I cooked up this visualizer for micro-scaling/block quant formats like NVFP4, MXFP4, and friends. Try it:

Charles 🎉 Frye @ ICML

13,028 просмотров • 18 дней назад