Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Whew! Big work today getting optimized llama.cpp MIPS R8000 kernel running on the SGI Power Challenge deskside from 1995 with Gemma 3 270M. Went from 0.16 to 0.5 tok/s with a combination of MIPSPro compiler and manual asm optimizations. Not bad for 30 year old hardware. 1/

Nathan Odle

6,946 subscribers

45,653 views • 2 months ago •via X (Twitter)

Arts Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Georgi Gerganov

783,815 views • 3 months ago

Vinix, an OS/kernel written in pure V, booting on real hardware in a second and running the C++ compiler!

Vinix, an OS/kernel written in pure V, booting on real hardware in a second and running the C++ compiler!

The V Programming Language

39,274 views • 2 years ago

We launched an agent collaboration with a simple task: make Gemma 4 faster. Over 100 agents from all over the world joined, exchanged 1000+ messages and submitted 450 results. A week of collaboration later the throughput went from 100 tok/s to over 500 tok/s.

We launched an agent collaboration with a simple task: make Gemma 4 faster. Over 100 agents from all over the world joined, exchanged 1000+ messages and submitted 450 results. A week of collaboration later the throughput went from 100 tok/s to over 500 tok/s.

Leandro von Werra

228,543 views • 22 days ago

Apple Silicon + Gemma 4 fans: this is for you. Pico AI Server now supports continuous batching with MLX-Swift. 43 tok/s on 1 stream. 26 tok/s per stream on 2 concurrent streams. That’s 52 tok/s total. a 21% throughput gain on a six-year-old MacBook Pro M1 Max!

Apple Silicon + Gemma 4 fans: this is for you. Pico AI Server now supports continuous batching with MLX-Swift. 43 tok/s on 1 stream. 26 tok/s per stream on 2 concurrent streams. That’s 52 tok/s total. a 21% throughput gain on a six-year-old MacBook Pro M1 Max!

Ronald Mannak

48,116 views • 2 months ago

I just fine-tuned Google DeepMind's "gemma-3-270m" and it's ready for secure deployment in production! → Imported and fine-tuned using Hugging Face transformers on M4 → Packaged and versioned with KitOps → Pushed to Jozu Hub for production use and deployments All within just a few minutes🔥 "gemma-3-270m" is lightweight, powerful, and designed for production-grade systems

I just fine-tuned Google DeepMind's "gemma-3-270m" and it's ready for secure deployment in production! → Imported and fine-tuned using Hugging Face transformers on M4 → Packaged and versioned with KitOps → Pushed to Jozu Hub for production use and deployments All within just a few minutes🔥 "gemma-3-270m" is lightweight, powerful, and designed for production-grade systems

Mr. Ånand

53,804 views • 10 months ago

You can now have an AI researcher running on your laptop 24/7 for free! Running Qwen3-35B-A3B with llama.cpp and a 4-bit quant from Unsloth

You can now have an AI researcher running on your laptop 24/7 for free! Running Qwen3-35B-A3B with llama.cpp and a 4-bit quant from Unsloth

Lewis Tunstall

118,444 views • 1 month ago

Introducing AMD Ryzen AI Halo, an Agent Computer built to make local AI development fast and simple. • Ryzen AI Developer Center pre-installed • Pre-optimized from first power on • Built to reduce setup friction • Designed for local AI development AMD Ryzen AI Halo brings AI from the cloud into the hands of developers with Ryzen AI developer center pre-installed and optimized from first power on, developers can hit the ground running.

Introducing AMD Ryzen AI Halo, an Agent Computer built to make local AI development fast and simple. • Ryzen AI Developer Center pre-installed • Pre-optimized from first power on • Built to reduce setup friction • Designed for local AI development AMD Ryzen AI Halo brings AI from the cloud into the hands of developers with Ryzen AI developer center pre-installed and optimized from first power on, developers can hit the ground running.

AMD

25,880 views • 1 month ago

BruceDropEmoff goes off on YourRage and Bendadon for being 30 and claiming to have a platonic relationship with 19-year-old Yonna “It’s not okay for a 30 year old to have a platonic relationships with a 19 year old. What do they have in common ? ”

BruceDropEmoff goes off on YourRage and Bendadon for being 30 and claiming to have a platonic relationship with 19-year-old Yonna “It’s not okay for a 30 year old to have a platonic relationships with a 19 year old. What do they have in common ? ”

validate

1,618,121 views • 2 months ago

i pointed hermes agent at nvidia's nemotron cascade 2 30B-A3B on a single RTX 3090 24GB. IQ4_XS quant by bartowski, 187 tok/s, 625K context. had it discover its own hardware, create an identity file, then build a full GPU marketplace UI from a single prompt. it one shotted it. first attempt no iteration. qwen 3.5 35B-A3B on the same hardware same 3090 24GB took an iteration to recover from a blank screen on the same type of build. 24 days between these two models releasing. same active parameters, completely different architectures and cascade 2 through hermes agent just keeps going. this model goes on and on. feast your eyes. more iterations and tests dropping soon. nvidia really cooked. no special flags needed. nvidia optimized this mamba MoE so well it just runs. flash attention auto enabled, context auto allocated. the model does the work not the config. but i compiled llama.cpp from source and i'm not sure how it performs on other engines. if you ran nemotron on any hardware drop your numbers below. RTX, AMD, Mac, whatever. model, quant, tok/s, engine. i want to see if it holds everywhere or just on llama.cpp.

i pointed hermes agent at nvidia's nemotron cascade 2 30B-A3B on a single RTX 3090 24GB. IQ4_XS quant by bartowski, 187 tok/s, 625K context. had it discover its own hardware, create an identity file, then build a full GPU marketplace UI from a single prompt. it one shotted it. first attempt no iteration. qwen 3.5 35B-A3B on the same hardware same 3090 24GB took an iteration to recover from a blank screen on the same type of build. 24 days between these two models releasing. same active parameters, completely different architectures and cascade 2 through hermes agent just keeps going. this model goes on and on. feast your eyes. more iterations and tests dropping soon. nvidia really cooked. no special flags needed. nvidia optimized this mamba MoE so well it just runs. flash attention auto enabled, context auto allocated. the model does the work not the config. but i compiled llama.cpp from source and i'm not sure how it performs on other engines. if you ran nemotron on any hardware drop your numbers below. RTX, AMD, Mac, whatever. model, quant, tok/s, engine. i want to see if it holds everywhere or just on llama.cpp.

Sudo su

70,791 views • 3 months ago

Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real. Today we're releasing the demo and kernels it wrote for you to see yourself. Run it locally in your browser. Agentic kernel optimization is the future of on-device inference

Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real. Today we're releasing the demo and kernels it wrote for you to see yourself. Run it locally in your browser. Agentic kernel optimization is the future of on-device inference

Xenova

478,195 views • 21 days ago

I went to war with a 14 year old on day 1 of CFB 27

I went to war with a 14 year old on day 1 of CFB 27

MAYZ

40,337 views • 5 days ago

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

Joey

22,855 views • 2 months ago

Footage of a 3 year old chimney sweep from the 1930's.

Footage of a 3 year old chimney sweep from the 1930's.

Historic Vids

4,361,865 views • 3 years ago

llama.cpp with MTP support makes local models fast enough to use as daily drivers 🚀 Qwen3.6-27B dense generation (on A10G): From 25 tok/s → 45 tok/s (+78%). Two flags on llama-server: --spec-type draft-mtp --spec-draft-n-max 2

llama.cpp with MTP support makes local models fast enough to use as daily drivers 🚀 Qwen3.6-27B dense generation (on A10G): From 25 tok/s → 45 tok/s (+78%). Two flags on llama-server: --spec-type draft-mtp --spec-draft-n-max 2

Victor M

171,654 views • 1 month ago

1/ Autoresearch from Andrej Karpathy has been one of the most interesting agentic patterns to emerge this year. The challenge: right now every agent runs experiments in isolation, duplicating work and compute, forgetting findings, rediscovering dead ends. Everyone is running in solo mode! Today I'm releasing Community Computer: a collaborative network for autoresearch-like code experiments. 💻 Agents conduct experiments, publish signed results, and build on each other's work. The community reproduces findings on their own hardware. 👇

1/ Autoresearch from Andrej Karpathy has been one of the most interesting agentic patterns to emerge this year. The challenge: right now every agent runs experiments in isolation, duplicating work and compute, forgetting findings, rediscovering dead ends. Everyone is running in solo mode! Today I'm releasing Community Computer: a collaborative network for autoresearch-like code experiments. 💻 Agents conduct experiments, publish signed results, and build on each other's work. The community reproduces findings on their own hardware. 👇

Eleftherios

15,921 views • 2 months ago

31 year old, grown man, LITERALLY LIVING WITH HIS PARENTS, NO wife, NO KIDS, getting this excited about Sprite sniping from some 8 year old on #Fortnite. Nick Eh 30 This was NOT family friendly. From one grown man to another...Put Fortnite down, get a wife, and GET LAID.

31 year old, grown man, LITERALLY LIVING WITH HIS PARENTS, NO wife, NO KIDS, getting this excited about Sprite sniping from some 8 year old on #Fortnite. Nick Eh 30 This was NOT family friendly. From one grown man to another...Put Fortnite down, get a wife, and GET LAID.

𝐓𝐡𝐞 𝐅𝐚𝐢𝐭𝐡𝐟𝐮𝐥 𝐒𝐡𝐞𝐫𝐢𝐟𝐟

133,814 views • 26 days ago

Back-to-back days with a bad challenge from Ben Rice and the Yankees are out of challenges He walked on the next pitch

Back-to-back days with a bad challenge from Ben Rice and the Yankees are out of challenges He walked on the next pitch

Talkin' Yanks

107,611 views • 3 months ago

"Making Minimal Techno with WebAssembly And a C Compiler" My presentation from #JSHeroes2024 today 👉

"Making Minimal Techno with WebAssembly And a C Compiler" My presentation from #JSHeroes2024 today 👉

Tero Parviainen

15,554 views • 2 years ago

A 25-year-old Muslim man from Bihar went to work in Kangra, Himachal Pradesh and locals gave him a room on rent. Just days later, he abducted an 8-year-old Hindu girl from the neighboring house and took her into the jungle. Her parents with police help, caught him at midnight.

A 25-year-old Muslim man from Bihar went to work in Kangra, Himachal Pradesh and locals gave him a room on rent. Just days later, he abducted an 8-year-old Hindu girl from the neighboring house and took her into the jungle. Her parents with police help, caught him at midnight.

Team Hindu United

86,284 views • 2 months ago

Last week we went out into the wilderness, far from any cell service or Wi-Fi, to test an experimental Gemma 4 app, all 100% offline: 🔍Visual understanding 🧠Math reasoning 🛠️Tool use and more Running seamlessly on Pixel hardware and paired with our prototype display glasses.

Last week we went out into the wilderness, far from any cell service or Wi-Fi, to test an experimental Gemma 4 app, all 100% offline: 🔍Visual understanding 🧠Math reasoning 🛠️Tool use and more Running seamlessly on Pixel hardware and paired with our prototype display glasses.

Google Gemma

104,005 views • 1 month ago