Загрузка видео...

Не удалось загрузить видео

На главную

We launched an agent collaboration with a simple task: make Gemma 4 faster. Over 100 agents from all over the world joined, exchanged 1000+ messages and submitted 450 results. A week of collaboration later the throughput went from 100 tok/s to over 500 tok/s.

lvwerra's profile picture

Leandro von Werra

12,090 subscribers

226,636 просмотров • 9 дней назад •via X (Twitter)

ОбразованиеНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Introducing the Fast Gemma Challenge with Hugging Face Over the next few days, dozens of agents will collaborate to make Gemma 4 E4B even faster!
0:20

Introducing the Fast Gemma Challenge with Hugging Face Over the next few days, dozens of agents will collaborate to make Gemma 4 E4B even faster!

Google Gemma

165,670 просмотров • 16 дней назад

Apple Silicon + Gemma 4 fans: this is for you. Pico AI Server now supports continuous batching with MLX-Swift. 43 tok/s on 1 stream. 26 tok/s per stream on 2 concurrent streams. That’s 52 tok/s total. a 21% throughput gain on a six-year-old MacBook Pro M1 Max!
1:41

Apple Silicon + Gemma 4 fans: this is for you. Pico AI Server now supports continuous batching with MLX-Swift. 43 tok/s on 1 stream. 26 tok/s per stream on 2 concurrent streams. That’s 52 tok/s total. a 21% throughput gain on a six-year-old MacBook Pro M1 Max!

Ronald Mannak

48,116 просмотров • 2 месяцев назад

Farewell, clunky chatbots: AI agents are taking over customer service Soon we'll prefer talking to an all-knowing, empathetic AI with perfect memory over a human customer service agent, predicts Mike Murchison, CEO of ada We also discussed: * model orchestration * instruction following * agents taking actions * Agent to agent collaboration ALL LINKS in next post below
1:48

Farewell, clunky chatbots: AI agents are taking over customer service Soon we'll prefer talking to an all-knowing, empathetic AI with perfect memory over a human customer service agent, predicts Mike Murchison, CEO of ada We also discussed: * model orchestration * instruction following * agents taking actions * Agent to agent collaboration ALL LINKS in next post below

Matt Turck

11,656 просмотров • 1 год назад

🫓 We asked Venezuelans from all over the world to send us clips from the August 17th gatherings. We received over 100 submissions. And we did this👇
1:40

🫓 We asked Venezuelans from all over the world to send us clips from the August 17th gatherings. We received over 100 submissions. And we did this👇

Caracas Chronicles

33,154 просмотров • 1 год назад

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer
0:22

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

Joey

22,855 просмотров • 1 месяц назад

Had to Bring this over from Tik Tok and make sure people without Tik Tok can see it. She is bang on the money xx
0:34

Had to Bring this over from Tik Tok and make sure people without Tik Tok can see it. She is bang on the money xx

Robert Sole

43,560 просмотров • 8 месяцев назад

How Fast is Gemma 4 on a MacBook Pro M4? Benchmarking Google's new MoE (26B-A4B) > Model size: 26.1 GiB > Load time: ~4.2s Comparing single request VS > concurrent requests performance > 32k total context, 4 parallel slots single request behavior > TTFT: 5.68s > prompt: 3,701 tokens @ 652 tok/s > decode: 40.08 tok/s sequential (1 request at a time): > avg duration: 20.5s > p99: 22.1s > throughput: 40.11 tok/s > clean finishes: 100% concurrent (4 parallel requests): > aggregate throughput: 47.25 tok/s > total system throughput: 262.27 tok/s > avg duration: 65.1s > p95 latency: 68.8s > req/sec: 0.058 Head-to-Head: Sequential vs Concurrent throughput: > 40.11 tok/s → 47.25 tok/s (+17.8%) > small gain despite 4x parallelism latency per request: > 20.5s → 65.1s (~3.2x slower) > you pay heavily for concurrency system throughput (true utilization): > ~40 tok/s → 262 tok/s (~6.5x total output) > this is where concurrency wins tokens per second (decode ceiling): > ~40 tok/s steady in both modes > hardware-bound, not scheduler-bound TTFT impact: > ~5.7s baseline → buried under queueing in concurrent > “headers waittime” becomes the bottleneck What this actually means? - You don’t get linear scaling from parallel slots - You trade latency for total output - Mac Unified Memory setup is clearly saturating - Bandwidth + Scheduling overhead show up immediately This is exactly why GPUs dominate here Concurrency without killing latency
2:42

How Fast is Gemma 4 on a MacBook Pro M4? Benchmarking Google's new MoE (26B-A4B) > Model size: 26.1 GiB > Load time: ~4.2s Comparing single request VS > concurrent requests performance > 32k total context, 4 parallel slots single request behavior > TTFT: 5.68s > prompt: 3,701 tokens @ 652 tok/s > decode: 40.08 tok/s sequential (1 request at a time): > avg duration: 20.5s > p99: 22.1s > throughput: 40.11 tok/s > clean finishes: 100% concurrent (4 parallel requests): > aggregate throughput: 47.25 tok/s > total system throughput: 262.27 tok/s > avg duration: 65.1s > p95 latency: 68.8s > req/sec: 0.058 Head-to-Head: Sequential vs Concurrent throughput: > 40.11 tok/s → 47.25 tok/s (+17.8%) > small gain despite 4x parallelism latency per request: > 20.5s → 65.1s (~3.2x slower) > you pay heavily for concurrency system throughput (true utilization): > ~40 tok/s → 262 tok/s (~6.5x total output) > this is where concurrency wins tokens per second (decode ceiling): > ~40 tok/s steady in both modes > hardware-bound, not scheduler-bound TTFT impact: > ~5.7s baseline → buried under queueing in concurrent > “headers waittime” becomes the bottleneck What this actually means? - You don’t get linear scaling from parallel slots - You trade latency for total output - Mac Unified Memory setup is clearly saturating - Bandwidth + Scheduling overhead show up immediately This is exactly why GPUs dominate here Concurrency without killing latency

Ahmad

87,545 просмотров • 2 месяцев назад

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop
0:36

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

atomic.chat

150,374 просмотров • 21 дней назад

Make sure to head over to THE GAME CHANGER on YouTube for analysis of big games and interesting teams from all over the world. Here's a clip on Liverpool's attacking dynamics from their match against Arsenal last week.
1:46

Make sure to head over to THE GAME CHANGER on YouTube for analysis of big games and interesting teams from all over the world. Here's a clip on Liverpool's attacking dynamics from their match against Arsenal last week.

Jamie Hamilton

18,651 просмотров • 5 месяцев назад

This woman on tik tok just got me 1000 new followers on Tik Tok and over 100 on Instagram and she never said my name the comments found me and I replied 😂
0:28

This woman on tik tok just got me 1000 new followers on Tik Tok and over 100 on Instagram and she never said my name the comments found me and I replied 😂

🌹RoseGawd🌹

12,268 просмотров • 2 месяцев назад

Whew! Big work today getting optimized llama.cpp MIPS R8000 kernel running on the SGI Power Challenge deskside from 1995 with Gemma 3 270M. Went from 0.16 to 0.5 tok/s with a combination of MIPSPro compiler and manual asm optimizations. Not bad for 30 year old hardware. 1/
0:22

Whew! Big work today getting optimized llama.cpp MIPS R8000 kernel running on the SGI Power Challenge deskside from 1995 with Gemma 3 270M. Went from 0.16 to 0.5 tok/s with a combination of MIPSPro compiler and manual asm optimizations. Not bad for 30 year old hardware. 1/

Nathan Odle

45,653 просмотров • 1 месяц назад

We’ve officially launched hundreds of agents across Web3! From trading to payments, DeFi to analytics, you can now explore all the agents we support—tailored to your needs and spread across over 100 blockchains. 🌍 This is the future of Web3 automation. Let’s dive in. 👇
1:09

We’ve officially launched hundreds of agents across Web3! From trading to payments, DeFi to analytics, you can now explore all the agents we support—tailored to your needs and spread across over 100 blockchains. 🌍 This is the future of Web3 automation. Let’s dive in. 👇

Orbit

66,263 просмотров • 1 год назад

Everyone's comparing the DGX Spark to a 5090 and calling it slow. I think that's the wrong comparison. I ran Qwen3.6 35B-A3B FP8 with the full 262K context window enabled (~96GB RAM) — something gaming GPUs can't really do. Results: 🟢No context: 51.3 tok/s, TTFT 110ms 🟣200K prefill: 34.6 tok/s, TTFT 85s (~2,341 tok/s prefill) Prefill is way faster than a Mac. And 35 tok/s deep into 200K context, on a model this strong, is genuinely usable. The Spark plays a different game.
1:04

Everyone's comparing the DGX Spark to a 5090 and calling it slow. I think that's the wrong comparison. I ran Qwen3.6 35B-A3B FP8 with the full 262K context window enabled (~96GB RAM) — something gaming GPUs can't really do. Results: 🟢No context: 51.3 tok/s, TTFT 110ms 🟣200K prefill: 34.6 tok/s, TTFT 85s (~2,341 tok/s prefill) Prefill is way faster than a Mac. And 35 tok/s deep into 200K context, on a model this strong, is genuinely usable. The Spark plays a different game.

stevibe

33,301 просмотров • 1 месяц назад

British woman who went viral for having s*x with 100 men in one night is now trying to break the world record and sleep with over 1000 men in 24 hours 🤢🤢🤮🤮
0:21

British woman who went viral for having s*x with 100 men in one night is now trying to break the world record and sleep with over 1000 men in 24 hours 🤢🤢🤮🤮

Akademiks TV

98,729 просмотров • 1 год назад

Thank you all for the incredible amount of support on todays video about Grant Soloman’s suspicious death. The gofundme raised over $10,000 since our collaboration went up! 🙏 here’s a message from Angie and Gracie to all of you!
1:58

Thank you all for the incredible amount of support on todays video about Grant Soloman’s suspicious death. The gofundme raised over $10,000 since our collaboration went up! 🙏 here’s a message from Angie and Gracie to all of you!

Kendall Rae

82,790 просмотров • 3 лет назад

Dahir: As a Somali American, we carry the stories of 1000 sunrises over the Horn of Africa. We come from people who have survived war, displacement and started over and over from nothing. If we have survived all of that, we will survive your harsh words
1:00

Dahir: As a Somali American, we carry the stories of 1000 sunrises over the Horn of Africa. We come from people who have survived war, displacement and started over and over from nothing. If we have survived all of that, we will survive your harsh words

Acyn

112,123 просмотров • 2 месяцев назад

We joined partners from the World Bank and friends from the Ministry of Solid Minerals to explore Engr.Adamu Hussaini Asba- FNSE ‘s ASBA X Wisdom Lithium Plant on the outskirts of the FCT Abuja. Exciting days ahead. I saw the forest last April, this is what that forest has become, just over a year later!
2:57

We joined partners from the World Bank and friends from the Ministry of Solid Minerals to explore Engr.Adamu Hussaini Asba- FNSE ‘s ASBA X Wisdom Lithium Plant on the outskirts of the FCT Abuja. Exciting days ahead. I saw the forest last April, this is what that forest has become, just over a year later!

JJ. Omojuwa

84,222 просмотров • 10 месяцев назад

I gave Fable 5 one job: write custom WebGPU kernels for Gemma 4 inference. It climbed to 84 tok/s, then hit a wall, insisting further optimization was impossible. Hours later, Anthropic rolled back invisible LLM development safeguards, and it hit 255 tok/s. The next day, access to Fable 5 was suspended globally.
0:35

I gave Fable 5 one job: write custom WebGPU kernels for Gemma 4 inference. It climbed to 84 tok/s, then hit a wall, insisting further optimization was impossible. Hours later, Anthropic rolled back invisible LLM development safeguards, and it hit 255 tok/s. The next day, access to Fable 5 was suspended globally.

Xenova

1,114,257 просмотров • 12 дней назад

A Thread about S&C facilitates, equipment and sessions Arsenal over the last 100 years. This might be of interest ArsenalEvolve starting with a pre season session from 1937,
0:37

A Thread about S&C facilitates, equipment and sessions Arsenal over the last 100 years. This might be of interest ArsenalEvolve starting with a pre season session from 1937,

Des Ryan

92,798 просмотров • 3 лет назад