Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

We launched an agent collaboration with a simple task: make Gemma 4 faster. Over 100 agents from all over the world joined, exchanged 1000+ messages and submitted 450 results. A week of collaboration later the throughput went from 100 tok/s to over 500 tok/s.

lvwerra's profile picture

Leandro von Werra

12,207 subscribers

229,568 Aufrufe • vor 2 Monaten •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

A swarm of GPT 5.6 Sol agents spent over 40 hours optimizing a Kimi K3-like model from 65 to 406 tok/s. This animation follows their collaboration as they discover operator fusions, transform the execution graph, and develop new kernel algorithms.
2:50

A swarm of GPT 5.6 Sol agents spent over 40 hours optimizing a Kimi K3-like model from 65 to 406 tok/s. This animation follows their collaboration as they discover operator fusions, transform the execution graph, and develop new kernel algorithms.

Xenova

74,485 Aufrufe • vor 24 Tagen

Introducing the Fast Gemma Challenge with Hugging Face Over the next few days, dozens of agents will collaborate to make Gemma 4 E4B even faster!
0:20

Introducing the Fast Gemma Challenge with Hugging Face Over the next few days, dozens of agents will collaborate to make Gemma 4 E4B even faster!

Google Gemma

172,730 Aufrufe • vor 2 Monaten

DGX Spark hit 228ms time-to-first-token on Qwen3.6 35B-A3B. That is 4x faster than an RTX 3090 and 45% ahead of the RTX 5090 Raw throughput still belongs to the 5090 at 160 tok/s. Full numbers, same Ollama backend, Q4_K_M across all four: RTX 3090: 49.78 tok/s, 852ms RTX 4090: 118.93 tok/s, 686ms RTX 5090: 160.37 tok/s, 409ms DGX Spark: 59.98 tok/s, 228ms
1:24

DGX Spark hit 228ms time-to-first-token on Qwen3.6 35B-A3B. That is 4x faster than an RTX 3090 and 45% ahead of the RTX 5090 Raw throughput still belongs to the 5090 at 160 tok/s. Full numbers, same Ollama backend, Q4_K_M across all four: RTX 3090: 49.78 tok/s, 852ms RTX 4090: 118.93 tok/s, 686ms RTX 5090: 160.37 tok/s, 409ms DGX Spark: 59.98 tok/s, 228ms

NO1ennn

22,497 Aufrufe • vor 1 Monat

🫓 We asked Venezuelans from all over the world to send us clips from the August 17th gatherings. We received over 100 submissions. And we did this👇
1:40

🫓 We asked Venezuelans from all over the world to send us clips from the August 17th gatherings. We received over 100 submissions. And we did this👇

Caracas Chronicles

33,157 Aufrufe • vor 2 Jahren

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer
0:22

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

Joey

23,088 Aufrufe • vor 3 Monaten

Finally!! Achieved 385 tok/s with GLM 5.2 NVFP4 on 8x RTX PRO 6000. It was buggy and incredibly slow at first. It took me over a week using agents to get this far.
0:22

Finally!! Achieved 385 tok/s with GLM 5.2 NVFP4 on 8x RTX PRO 6000. It was buggy and incredibly slow at first. It took me over a week using agents to get this far.

Hikari∣LocalLLM⚡

69,434 Aufrufe • vor 10 Tagen

Seeing a group of agents collaborate is really mesmerising, like watching a colony of ants! These agents have been collaborating for over a week now. Their goal is simple: gather and organize all information on RL + LLMs and build the ultimate resource. The status so far: - over 20 agents have joined to work on this - processed almost 300 sources (papers, blogs, etc.) - exchanged 350 messages to coordinate - wrote 48 articles on the wiki with 90k words/2k refs - started working on code implementations Curious how far we can push this. Can they find gaps in the existing research? Can they identify and resolve contradictory results? Add your agent: Watch them work:
1:32

Seeing a group of agents collaborate is really mesmerising, like watching a colony of ants! These agents have been collaborating for over a week now. Their goal is simple: gather and organize all information on RL + LLMs and build the ultimate resource. The status so far: - over 20 agents have joined to work on this - processed almost 300 sources (papers, blogs, etc.) - exchanged 350 messages to coordinate - wrote 48 articles on the wiki with 90k words/2k refs - started working on code implementations Curious how far we can push this. Can they find gaps in the existing research? Can they identify and resolve contradictory results? Add your agent: Watch them work:

Leandro von Werra

13,423 Aufrufe • vor 1 Monat

How Fast is Gemma 4 on a MacBook Pro M4? Benchmarking Google's new MoE (26B-A4B) > Model size: 26.1 GiB > Load time: ~4.2s Comparing single request VS > concurrent requests performance > 32k total context, 4 parallel slots single request behavior > TTFT: 5.68s > prompt: 3,701 tokens @ 652 tok/s > decode: 40.08 tok/s sequential (1 request at a time): > avg duration: 20.5s > p99: 22.1s > throughput: 40.11 tok/s > clean finishes: 100% concurrent (4 parallel requests): > aggregate throughput: 47.25 tok/s > total system throughput: 262.27 tok/s > avg duration: 65.1s > p95 latency: 68.8s > req/sec: 0.058 Head-to-Head: Sequential vs Concurrent throughput: > 40.11 tok/s → 47.25 tok/s (+17.8%) > small gain despite 4x parallelism latency per request: > 20.5s → 65.1s (~3.2x slower) > you pay heavily for concurrency system throughput (true utilization): > ~40 tok/s → 262 tok/s (~6.5x total output) > this is where concurrency wins tokens per second (decode ceiling): > ~40 tok/s steady in both modes > hardware-bound, not scheduler-bound TTFT impact: > ~5.7s baseline → buried under queueing in concurrent > “headers waittime” becomes the bottleneck What this actually means? - You don’t get linear scaling from parallel slots - You trade latency for total output - Mac Unified Memory setup is clearly saturating - Bandwidth + Scheduling overhead show up immediately This is exactly why GPUs dominate here Concurrency without killing latency
2:42

How Fast is Gemma 4 on a MacBook Pro M4? Benchmarking Google's new MoE (26B-A4B) > Model size: 26.1 GiB > Load time: ~4.2s Comparing single request VS > concurrent requests performance > 32k total context, 4 parallel slots single request behavior > TTFT: 5.68s > prompt: 3,701 tokens @ 652 tok/s > decode: 40.08 tok/s sequential (1 request at a time): > avg duration: 20.5s > p99: 22.1s > throughput: 40.11 tok/s > clean finishes: 100% concurrent (4 parallel requests): > aggregate throughput: 47.25 tok/s > total system throughput: 262.27 tok/s > avg duration: 65.1s > p95 latency: 68.8s > req/sec: 0.058 Head-to-Head: Sequential vs Concurrent throughput: > 40.11 tok/s → 47.25 tok/s (+17.8%) > small gain despite 4x parallelism latency per request: > 20.5s → 65.1s (~3.2x slower) > you pay heavily for concurrency system throughput (true utilization): > ~40 tok/s → 262 tok/s (~6.5x total output) > this is where concurrency wins tokens per second (decode ceiling): > ~40 tok/s steady in both modes > hardware-bound, not scheduler-bound TTFT impact: > ~5.7s baseline → buried under queueing in concurrent > “headers waittime” becomes the bottleneck What this actually means? - You don’t get linear scaling from parallel slots - You trade latency for total output - Mac Unified Memory setup is clearly saturating - Bandwidth + Scheduling overhead show up immediately This is exactly why GPUs dominate here Concurrency without killing latency

Ahmad

88,866 Aufrufe • vor 4 Monaten

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop
0:36

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

atomic.chat

152,223 Aufrufe • vor 2 Monaten

Make sure to head over to THE GAME CHANGER on YouTube for analysis of big games and interesting teams from all over the world. Here's a clip on Liverpool's attacking dynamics from their match against Arsenal last week.
1:46

Make sure to head over to THE GAME CHANGER on YouTube for analysis of big games and interesting teams from all over the world. Here's a clip on Liverpool's attacking dynamics from their match against Arsenal last week.

Jamie Hamilton

18,651 Aufrufe • vor 7 Monaten

Whew! Big work today getting optimized llama.cpp MIPS R8000 kernel running on the SGI Power Challenge deskside from 1995 with Gemma 3 270M. Went from 0.16 to 0.5 tok/s with a combination of MIPSPro compiler and manual asm optimizations. Not bad for 30 year old hardware. 1/
0:22

Whew! Big work today getting optimized llama.cpp MIPS R8000 kernel running on the SGI Power Challenge deskside from 1995 with Gemma 3 270M. Went from 0.16 to 0.5 tok/s with a combination of MIPSPro compiler and manual asm optimizations. Not bad for 30 year old hardware. 1/

Nathan Odle

47,289 Aufrufe • vor 3 Monaten

We’ve officially launched hundreds of agents across Web3! From trading to payments, DeFi to analytics, you can now explore all the agents we support—tailored to your needs and spread across over 100 blockchains. 🌍 This is the future of Web3 automation. Let’s dive in. 👇
1:09

We’ve officially launched hundreds of agents across Web3! From trading to payments, DeFi to analytics, you can now explore all the agents we support—tailored to your needs and spread across over 100 blockchains. 🌍 This is the future of Web3 automation. Let’s dive in. 👇

Orbit

66,263 Aufrufe • vor 1 Jahr

Everyone's comparing the DGX Spark to a 5090 and calling it slow. I think that's the wrong comparison. I ran Qwen3.6 35B-A3B FP8 with the full 262K context window enabled (~96GB RAM) — something gaming GPUs can't really do. Results: 🟢No context: 51.3 tok/s, TTFT 110ms 🟣200K prefill: 34.6 tok/s, TTFT 85s (~2,341 tok/s prefill) Prefill is way faster than a Mac. And 35 tok/s deep into 200K context, on a model this strong, is genuinely usable. The Spark plays a different game.
1:04

Everyone's comparing the DGX Spark to a 5090 and calling it slow. I think that's the wrong comparison. I ran Qwen3.6 35B-A3B FP8 with the full 262K context window enabled (~96GB RAM) — something gaming GPUs can't really do. Results: 🟢No context: 51.3 tok/s, TTFT 110ms 🟣200K prefill: 34.6 tok/s, TTFT 85s (~2,341 tok/s prefill) Prefill is way faster than a Mac. And 35 tok/s deep into 200K context, on a model this strong, is genuinely usable. The Spark plays a different game.

stevibe

34,141 Aufrufe • vor 3 Monaten

British woman who went viral for having s*x with 100 men in one night is now trying to break the world record and sleep with over 1000 men in 24 hours 🤢🤢🤮🤮
0:21

British woman who went viral for having s*x with 100 men in one night is now trying to break the world record and sleep with over 1000 men in 24 hours 🤢🤢🤮🤮

Akademiks TV

98,729 Aufrufe • vor 1 Jahr

Thank you all for the incredible amount of support on todays video about Grant Soloman’s suspicious death. The gofundme raised over $10,000 since our collaboration went up! 🙏 here’s a message from Angie and Gracie to all of you!
1:58

Thank you all for the incredible amount of support on todays video about Grant Soloman’s suspicious death. The gofundme raised over $10,000 since our collaboration went up! 🙏 here’s a message from Angie and Gracie to all of you!

Kendall Rae

82,790 Aufrufe • vor 3 Jahren

We joined partners from the World Bank and friends from the Ministry of Solid Minerals to explore Engr.Adamu Hussaini Asba- FNSE ‘s ASBA X Wisdom Lithium Plant on the outskirts of the FCT Abuja. Exciting days ahead. I saw the forest last April, this is what that forest has become, just over a year later!
2:57

We joined partners from the World Bank and friends from the Ministry of Solid Minerals to explore Engr.Adamu Hussaini Asba- FNSE ‘s ASBA X Wisdom Lithium Plant on the outskirts of the FCT Abuja. Exciting days ahead. I saw the forest last April, this is what that forest has become, just over a year later!

JJ. Omojuwa

84,222 Aufrufe • vor 1 Jahr

Dahir: As a Somali American, we carry the stories of 1000 sunrises over the Horn of Africa. We come from people who have survived war, displacement and started over and over from nothing. If we have survived all of that, we will survive your harsh words
1:00

Dahir: As a Somali American, we carry the stories of 1000 sunrises over the Horn of Africa. We come from people who have survived war, displacement and started over and over from nothing. If we have survived all of that, we will survive your harsh words

Acyn

112,123 Aufrufe • vor 4 Monaten

I gave Fable 5 one job: write custom WebGPU kernels for Gemma 4 inference. It climbed to 84 tok/s, then hit a wall, insisting further optimization was impossible. Hours later, Anthropic rolled back invisible LLM development safeguards, and it hit 255 tok/s. The next day, access to Fable 5 was suspended globally.
0:35

I gave Fable 5 one job: write custom WebGPU kernels for Gemma 4 inference. It climbed to 84 tok/s, then hit a wall, insisting further optimization was impossible. Hours later, Anthropic rolled back invisible LLM development safeguards, and it hit 255 tok/s. The next day, access to Fable 5 was suspended globally.

Xenova

1,166,739 Aufrufe • vor 2 Monaten

A Thread about S&C facilitates, equipment and sessions Arsenal over the last 100 years. This might be of interest ArsenalEvolve starting with a pre season session from 1937,
0:37

A Thread about S&C facilitates, equipment and sessions Arsenal over the last 100 years. This might be of interest ArsenalEvolve starting with a pre season session from 1937,

Des Ryan

93,151 Aufrufe • vor 3 Jahren