正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Whew! Big work today getting optimized llama.cpp MIPS R8000 kernel running on the SGI Power Challenge deskside from 1995 with Gemma 3 270M. Went from 0.16 to 0.5 tok/s with a combination of MIPSPro compiler and manual asm optimizations. Not bad for 30 year old hardware. 1/

Nathan Odle

6,946 subscribers

45,653 次观看 • 2 个月前 •via X (Twitter)

艺术科学技术

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Let me demonstrate the true power of llama.cpp: - Running on Mac Studio M2 Ultra (3 years old) - Gemma 4 26B A4B Q8_0 (full quality) - Built-in WebUI (ships with llama.cpp) - MCP support out of the box (web-search, HF, github, etc.) - Prompt speculative decoding The result: 300t/s (realtime video)

Georgi Gerganov

783,815 次观看 • 3 个月前

Vinix, an OS/kernel written in pure V, booting on real hardware in a second and running the C++ compiler!

Vinix, an OS/kernel written in pure V, booting on real hardware in a second and running the C++ compiler!

The V Programming Language

39,274 次观看 • 2 年前

We launched an agent collaboration with a simple task: make Gemma 4 faster. Over 100 agents from all over the world joined, exchanged 1000+ messages and submitted 450 results. A week of collaboration later the throughput went from 100 tok/s to over 500 tok/s.

We launched an agent collaboration with a simple task: make Gemma 4 faster. Over 100 agents from all over the world joined, exchanged 1000+ messages and submitted 450 results. A week of collaboration later the throughput went from 100 tok/s to over 500 tok/s.

Leandro von Werra

228,663 次观看 • 24 天前

Apple Silicon + Gemma 4 fans: this is for you. Pico AI Server now supports continuous batching with MLX-Swift. 43 tok/s on 1 stream. 26 tok/s per stream on 2 concurrent streams. That’s 52 tok/s total. a 21% throughput gain on a six-year-old MacBook Pro M1 Max!

Apple Silicon + Gemma 4 fans: this is for you. Pico AI Server now supports continuous batching with MLX-Swift. 43 tok/s on 1 stream. 26 tok/s per stream on 2 concurrent streams. That’s 52 tok/s total. a 21% throughput gain on a six-year-old MacBook Pro M1 Max!

Ronald Mannak

48,116 次观看 • 2 个月前

I just fine-tuned Google DeepMind's "gemma-3-270m" and it's ready for secure deployment in production! → Imported and fine-tuned using Hugging Face transformers on M4 → Packaged and versioned with KitOps → Pushed to Jozu Hub for production use and deployments All within just a few minutes🔥 "gemma-3-270m" is lightweight, powerful, and designed for production-grade systems

I just fine-tuned Google DeepMind's "gemma-3-270m" and it's ready for secure deployment in production! → Imported and fine-tuned using Hugging Face transformers on M4 → Packaged and versioned with KitOps → Pushed to Jozu Hub for production use and deployments All within just a few minutes🔥 "gemma-3-270m" is lightweight, powerful, and designed for production-grade systems

Mr. Ånand

53,804 次观看 • 10 个月前

You can now have an AI researcher running on your laptop 24/7 for free! Running Qwen3-35B-A3B with llama.cpp and a 4-bit quant from Unsloth

You can now have an AI researcher running on your laptop 24/7 for free! Running Qwen3-35B-A3B with llama.cpp and a 4-bit quant from Unsloth

Lewis Tunstall

118,444 次观看 • 1 个月前

Introducing AMD Ryzen AI Halo, an Agent Computer built to make local AI development fast and simple. • Ryzen AI Developer Center pre-installed • Pre-optimized from first power on • Built to reduce setup friction • Designed for local AI development AMD Ryzen AI Halo brings AI from the cloud into the hands of developers with Ryzen AI developer center pre-installed and optimized from first power on, developers can hit the ground running.

Introducing AMD Ryzen AI Halo, an Agent Computer built to make local AI development fast and simple. • Ryzen AI Developer Center pre-installed • Pre-optimized from first power on • Built to reduce setup friction • Designed for local AI development AMD Ryzen AI Halo brings AI from the cloud into the hands of developers with Ryzen AI developer center pre-installed and optimized from first power on, developers can hit the ground running.

AMD

25,880 次观看 • 1 个月前

BruceDropEmoff goes off on YourRage and Bendadon for being 30 and claiming to have a platonic relationship with 19-year-old Yonna “It’s not okay for a 30 year old to have a platonic relationships with a 19 year old. What do they have in common ? ”

BruceDropEmoff goes off on YourRage and Bendadon for being 30 and claiming to have a platonic relationship with 19-year-old Yonna “It’s not okay for a 30 year old to have a platonic relationships with a 19 year old. What do they have in common ? ”

validate

1,618,121 次观看 • 2 个月前

i pointed hermes agent at nvidia's nemotron cascade 2 30B-A3B on a single RTX 3090 24GB. IQ4_XS quant by bartowski, 187 tok/s, 625K context. had it discover its own hardware, create an identity file, then build a full GPU marketplace UI from a single prompt. it one shotted it. first attempt no iteration. qwen 3.5 35B-A3B on the same hardware same 3090 24GB took an iteration to recover from a blank screen on the same type of build. 24 days between these two models releasing. same active parameters, completely different architectures and cascade 2 through hermes agent just keeps going. this model goes on and on. feast your eyes. more iterations and tests dropping soon. nvidia really cooked. no special flags needed. nvidia optimized this mamba MoE so well it just runs. flash attention auto enabled, context auto allocated. the model does the work not the config. but i compiled llama.cpp from source and i'm not sure how it performs on other engines. if you ran nemotron on any hardware drop your numbers below. RTX, AMD, Mac, whatever. model, quant, tok/s, engine. i want to see if it holds everywhere or just on llama.cpp.

i pointed hermes agent at nvidia's nemotron cascade 2 30B-A3B on a single RTX 3090 24GB. IQ4_XS quant by bartowski, 187 tok/s, 625K context. had it discover its own hardware, create an identity file, then build a full GPU marketplace UI from a single prompt. it one shotted it. first attempt no iteration. qwen 3.5 35B-A3B on the same hardware same 3090 24GB took an iteration to recover from a blank screen on the same type of build. 24 days between these two models releasing. same active parameters, completely different architectures and cascade 2 through hermes agent just keeps going. this model goes on and on. feast your eyes. more iterations and tests dropping soon. nvidia really cooked. no special flags needed. nvidia optimized this mamba MoE so well it just runs. flash attention auto enabled, context auto allocated. the model does the work not the config. but i compiled llama.cpp from source and i'm not sure how it performs on other engines. if you ran nemotron on any hardware drop your numbers below. RTX, AMD, Mac, whatever. model, quant, tok/s, engine. i want to see if it holds everywhere or just on llama.cpp.

Sudo su

70,791 次观看 • 3 个月前

I went to war with a 14 year old on day 1 of CFB 27

I went to war with a 14 year old on day 1 of CFB 27

MAYZ

41,964 次观看 • 7 天前

Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real. Today we're releasing the demo and kernels it wrote for you to see yourself. Run it locally in your browser. Agentic kernel optimization is the future of on-device inference

Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real. Today we're releasing the demo and kernels it wrote for you to see yourself. Run it locally in your browser. Agentic kernel optimization is the future of on-device inference

Xenova

479,071 次观看 • 23 天前

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

got gemma 4 31B with MTP running on my DGX Spark. Hermes Agent did most of the legwork. baseline vs MTP on GB10: • c=1: 3.65 → 6.37 tok/s (1.74x) • c=4: 14.34 → 23.59 tok/s (1.65x) • c=8: 14.37 → 24.18 tok/s (1.68x) google says "up to 2x" — we're not quite there but it's real, not vapor. stack: DGX Spark / GB10 + gemma-4-31b-it + gemma-4-31b-it-assistant (MTP drafter) + vLLM built from PR 41745 MTP is basically a lightweight draft model that predicts multiple tokens while the big model verifies them all at once. smaller model does the busywork, bigger model just says yes/no. simple idea, weird to implement. next: tune the draft block size and see if we can push past 2x. also want to try it with Hermes Agent feeding prompts end to end. p.s: this was all done from telegram. Google DeepMind NVIDIA AI Developer

Joey

22,855 次观看 • 2 个月前

Footage of a 3 year old chimney sweep from the 1930's.

Footage of a 3 year old chimney sweep from the 1930's.

Historic Vids

4,361,865 次观看 • 3 年前

llama.cpp with MTP support makes local models fast enough to use as daily drivers 🚀 Qwen3.6-27B dense generation (on A10G): From 25 tok/s → 45 tok/s (+78%). Two flags on llama-server: --spec-type draft-mtp --spec-draft-n-max 2

llama.cpp with MTP support makes local models fast enough to use as daily drivers 🚀 Qwen3.6-27B dense generation (on A10G): From 25 tok/s → 45 tok/s (+78%). Two flags on llama-server: --spec-type draft-mtp --spec-draft-n-max 2

Victor M

171,654 次观看 • 1 个月前

1/ Autoresearch from Andrej Karpathy has been one of the most interesting agentic patterns to emerge this year. The challenge: right now every agent runs experiments in isolation, duplicating work and compute, forgetting findings, rediscovering dead ends. Everyone is running in solo mode! Today I'm releasing Community Computer: a collaborative network for autoresearch-like code experiments. 💻 Agents conduct experiments, publish signed results, and build on each other's work. The community reproduces findings on their own hardware. 👇

1/ Autoresearch from Andrej Karpathy has been one of the most interesting agentic patterns to emerge this year. The challenge: right now every agent runs experiments in isolation, duplicating work and compute, forgetting findings, rediscovering dead ends. Everyone is running in solo mode! Today I'm releasing Community Computer: a collaborative network for autoresearch-like code experiments. 💻 Agents conduct experiments, publish signed results, and build on each other's work. The community reproduces findings on their own hardware. 👇

Eleftherios

15,921 次观看 • 2 个月前

31 year old, grown man, LITERALLY LIVING WITH HIS PARENTS, NO wife, NO KIDS, getting this excited about Sprite sniping from some 8 year old on #Fortnite. Nick Eh 30 This was NOT family friendly. From one grown man to another...Put Fortnite down, get a wife, and GET LAID.

31 year old, grown man, LITERALLY LIVING WITH HIS PARENTS, NO wife, NO KIDS, getting this excited about Sprite sniping from some 8 year old on #Fortnite. Nick Eh 30 This was NOT family friendly. From one grown man to another...Put Fortnite down, get a wife, and GET LAID.

𝐓𝐡𝐞 𝐅𝐚𝐢𝐭𝐡𝐟𝐮𝐥 𝐒𝐡𝐞𝐫𝐢𝐟𝐟

133,814 次观看 • 28 天前

Back-to-back days with a bad challenge from Ben Rice and the Yankees are out of challenges He walked on the next pitch

Back-to-back days with a bad challenge from Ben Rice and the Yankees are out of challenges He walked on the next pitch

Talkin' Yanks

107,611 次观看 • 3 个月前

"Making Minimal Techno with WebAssembly And a C Compiler" My presentation from #JSHeroes2024 today 👉

"Making Minimal Techno with WebAssembly And a C Compiler" My presentation from #JSHeroes2024 today 👉

Tero Parviainen

15,554 次观看 • 2 年前

* You are filled with the power of learning not to snap on beat 1 and 3.

Sensitive content

* You are filled with the power of learning not to snap on beat 1 and 3.

SQUIFF

144,469 次观看 • 1 年前

A 25-year-old Muslim man from Bihar went to work in Kangra, Himachal Pradesh and locals gave him a room on rent. Just days later, he abducted an 8-year-old Hindu girl from the neighboring house and took her into the jungle. Her parents with police help, caught him at midnight.

A 25-year-old Muslim man from Bihar went to work in Kangra, Himachal Pradesh and locals gave him a room on rent. Just days later, he abducted an 8-year-old Hindu girl from the neighboring house and took her into the jungle. Her parents with police help, caught him at midnight.

Team Hindu United

86,317 次观看 • 2 个月前