Ahmad

@TheAhmadOsman • 59,655 subscribers

ai, chips, systems engineering, infra & hardware · on a mission to build a frontier, infra-first AI Lab in the West · i mod GPUs on r/LocalLLaMA

Shorts

Feels like I am just gonna end paying the California taxes

Feels like I am just gonna end paying the California taxes

12,521 次观看

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

INCREDIBLE SPEED running Claude Code w/ local models on my own GPUs at home > SGLang serving MiniMax-M2.1 > on 8x RTX 3090s > nvtop showing live GPU load > Claude Code generating code + docs > end-2-end on my AI cluster MiniMax-M2.1 is my favorite model to run locally nowadays

INCREDIBLE SPEED running Claude Code w/ local models on my own GPUs at home > SGLang serving MiniMax-M2.1 > on 8x RTX 3090s > nvtop showing live GPU load > Claude Code generating code + docs > end-2-end on my AI cluster MiniMax-M2.1 is my favorite model to run locally nowadays

592,866 次观看 • 4 个月前

running Qwen3.5 397B MoE (17B active/token) on 4x DGX Sparks in FP8 (~400GB) > OpenCode driving > agent exploring its own config > probing all 4 Sparks (via ssh) + reporting thermals > inspecting how vLLM is serving it > collecting + analyzing its own stats local AI is awesome

running Qwen3.5 397B MoE (17B active/token) on 4x DGX Sparks in FP8 (~400GB) > OpenCode driving > agent exploring its own config > probing all 4 Sparks (via ssh) + reporting thermals > inspecting how vLLM is serving it > collecting + analyzing its own stats local AI is awesome

121,691 次观看 • 2 个月前

running Claude Code w/ local models on my own GPUs at home > vLLM serving GLM-4.5 Air > on 4x RTX 3090s > nvtop showing live GPU load > Claude Code generating code + docs > end-to-end on my AI cluster this is what local AI actually looks like Buy a GPU

running Claude Code w/ local models on my own GPUs at home > vLLM serving GLM-4.5 Air > on 4x RTX 3090s > nvtop showing live GPU load > Claude Code generating code + docs > end-to-end on my AI cluster this is what local AI actually looks like Buy a GPU

261,640 次观看 • 5 个月前

MiniMax M2.7 at home running on 4x DGX Sparks vLLM serving full BF16 weights, 200k context OpenCode having the model monitor its own hardware and report thermals, tokens/sec, TTFT, and other runtime stats in real time What benchmarks / workflows / things do you wanna see next?

MiniMax M2.7 at home running on 4x DGX Sparks vLLM serving full BF16 weights, 200k context OpenCode having the model monitor its own hardware and report thermals, tokens/sec, TTFT, and other runtime stats in real time What benchmarks / workflows / things do you wanna see next?

91,625 次观看 • 2 个月前

How Fast is Gemma 4 on a MacBook Pro M4? Benchmarking Google's new MoE (26B-A4B) > Model size: 26.1 GiB > Load time: ~4.2s Comparing single request VS > concurrent requests performance > 32k total context, 4 parallel slots single request behavior > TTFT: 5.68s > prompt: 3,701 tokens @ 652 tok/s > decode: 40.08 tok/s sequential (1 request at a time): > avg duration: 20.5s > p99: 22.1s > throughput: 40.11 tok/s > clean finishes: 100% concurrent (4 parallel requests): > aggregate throughput: 47.25 tok/s > total system throughput: 262.27 tok/s > avg duration: 65.1s > p95 latency: 68.8s > req/sec: 0.058 Head-to-Head: Sequential vs Concurrent throughput: > 40.11 tok/s → 47.25 tok/s (+17.8%) > small gain despite 4x parallelism latency per request: > 20.5s → 65.1s (~3.2x slower) > you pay heavily for concurrency system throughput (true utilization): > ~40 tok/s → 262 tok/s (~6.5x total output) > this is where concurrency wins tokens per second (decode ceiling): > ~40 tok/s steady in both modes > hardware-bound, not scheduler-bound TTFT impact: > ~5.7s baseline → buried under queueing in concurrent > “headers waittime” becomes the bottleneck What this actually means? - You don’t get linear scaling from parallel slots - You trade latency for total output - Mac Unified Memory setup is clearly saturating - Bandwidth + Scheduling overhead show up immediately This is exactly why GPUs dominate here Concurrency without killing latency

How Fast is Gemma 4 on a MacBook Pro M4? Benchmarking Google's new MoE (26B-A4B) > Model size: 26.1 GiB > Load time: ~4.2s Comparing single request VS > concurrent requests performance > 32k total context, 4 parallel slots single request behavior > TTFT: 5.68s > prompt: 3,701 tokens @ 652 tok/s > decode: 40.08 tok/s sequential (1 request at a time): > avg duration: 20.5s > p99: 22.1s > throughput: 40.11 tok/s > clean finishes: 100% concurrent (4 parallel requests): > aggregate throughput: 47.25 tok/s > total system throughput: 262.27 tok/s > avg duration: 65.1s > p95 latency: 68.8s > req/sec: 0.058 Head-to-Head: Sequential vs Concurrent throughput: > 40.11 tok/s → 47.25 tok/s (+17.8%) > small gain despite 4x parallelism latency per request: > 20.5s → 65.1s (~3.2x slower) > you pay heavily for concurrency system throughput (true utilization): > ~40 tok/s → 262 tok/s (~6.5x total output) > this is where concurrency wins tokens per second (decode ceiling): > ~40 tok/s steady in both modes > hardware-bound, not scheduler-bound TTFT impact: > ~5.7s baseline → buried under queueing in concurrent > “headers waittime” becomes the bottleneck What this actually means? - You don’t get linear scaling from parallel slots - You trade latency for total output - Mac Unified Memory setup is clearly saturating - Bandwidth + Scheduling overhead show up immediately This is exactly why GPUs dominate here Concurrency without killing latency

87,545 次观看 • 2 个月前

Running Qwen 3.5 4B on my iPhone 17 Pro Max Very smart & capable model for how small it is Very fast as well

Running Qwen 3.5 4B on my iPhone 17 Pro Max Very smart & capable model for how small it is Very fast as well

107,405 次观看 • 3 个月前

Impressive demo running on a single box The future of AI is local btw

Impressive demo running on a single box The future of AI is local btw

40,078 次观看 • 1 个月前

I asked Jensen whether we will see more Nemotron models or if the recent releases were just to prove NVFP4 training works

I asked Jensen whether we will see more Nemotron models or if the recent releases were just to prove NVFP4 training works

59,656 次观看 • 2 个月前

MASSIVE llama cpp now now ships with a built-in web UI stop using ollama, there are no more excuses

MASSIVE llama cpp now now ships with a built-in web UI stop using ollama, there are no more excuses

94,975 次观看 • 7 个月前

GPU GIVEAWAY Buy a GPU × GTC 2026 = Give a GPU > RTX PRO 6000 Blackwell > 96GB VRAM • NVFP4 > ~$15K value > Brand new If this does well I’ll ask NVIDIA for more GPUs next time… maybe even DGX Sparks How to enter? Short clip here Full clip in the replies GO GO GO

36,697 次观看 • 3 个月前

NVIDIA AI pulled me in for an interview at GTC this week

NVIDIA AI pulled me in for an interview at GTC this week

27,664 次观看 • 2 个月前

TEN HOURS LEFT TO WIN DGX Sparks & Mac Minis > ur submission can be as simple as > sending hello world msg while showing ur laptop > or as complicated as a room full of hardware > u have to use ur office chair as a laptop stand reply with ur submissions so i don’t miss them

TEN HOURS LEFT TO WIN DGX Sparks & Mac Minis > ur submission can be as simple as > sending hello world msg while showing ur laptop > or as complicated as a room full of hardware > u have to use ur office chair as a laptop stand reply with ur submissions so i don’t miss them

26,838 次观看 • 6 个月前

没有更多内容可加载