Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Opus at home, at 200+ tok/s I love ZAI

0xSero

65,642 subscribers

201,119 views • 7 days ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Nemotron-3-Ultra running on 4x 6000s edits my latest demo video.. - 75 tok/s decode - 8x concurrency - 256k context - 899 tok/s prefill - 20k tok/s prefill cache - NVFP4 Setting it up to be my Hermes driver. It's good enough at most things and doesn't talk like a moron.

Nemotron-3-Ultra running on 4x 6000s edits my latest demo video.. - 75 tok/s decode - 8x concurrency - 256k context - 899 tok/s prefill - 20k tok/s prefill cache - NVFP4 Setting it up to be my Hermes driver. It's good enough at most things and doesn't talk like a moron.

0xSero

15,674 views • 2 months ago

Finally, full precision MiniMax-M2.7 running at home. 100 tokens/s decode 5050 tokens/s prefill

Finally, full precision MiniMax-M2.7 running at home. 100 tokens/s decode 5050 tokens/s prefill

0xSero

72,465 views • 4 months ago

GPT 5.6 Sol at 750 tok/s on Cerebras release is going to be crazy As a reminder, what Opus at <100 tok/s feels like Wake me up when it's done

GPT 5.6 Sol at 750 tok/s on Cerebras release is going to be crazy As a reminder, what Opus at <100 tok/s feels like Wake me up when it's done

Sarah Chieng

88,618 views • 1 month ago

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 at 38.5 tok/s This is a breakthrough.

0xSero

157,562 views • 1 month ago

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (:

0xSero

74,772 views • 4 months ago

Minimax-M3 running on 4x RTX Pro 6000s - 800k context - 4x concurrency at 250k - 70-120 tok/s - 2000 tok/s prefill no cache - 376gb vram - mxfp4 It's working on improving the audio on one of my videos, it's actually doing a good job in researching solutions. Good model

Minimax-M3 running on 4x RTX Pro 6000s - 800k context - 4x concurrency at 250k - 70-120 tok/s - 2000 tok/s prefill no cache - 376gb vram - mxfp4 It's working on improving the audio on one of my videos, it's actually doing a good job in researching solutions. Good model

0xSero

24,634 views • 2 months ago

I am blown away by MiMo-V2.5 It's so steerable, I didn't like MiMo-V2 This is self hosted btw, 125 tok/s across 2 sessions 4k prefill tok/s It's updating my website + editing a longer video I will be posting in a few minutes.

I am blown away by MiMo-V2.5 It's so steerable, I didn't like MiMo-V2 This is self hosted btw, 125 tok/s across 2 sessions 4k prefill tok/s It's updating my website + editing a longer video I will be posting in a few minutes.

0xSero

34,494 views • 4 months ago

Kimi-K2.6 running locally 110+ tok/s

Kimi-K2.6 running locally 110+ tok/s

0xSero

24,640 views • 3 months ago

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio

Gemma-4-31B builds a website from a wireframe I sketched out in Excalidraw Not perfect but really really fast with Spec decoding 150 tok/s+ and it's good at following instructions Free workhorse. Use speculative decoding, it's only 0.7GB of vram for a 2x speed up! vllm-studio

0xSero

21,768 views • 3 months ago

Guess who got GLM-5.2 at home? You can't deny I'm elite at inference.

Guess who got GLM-5.2 at home? You can't deny I'm elite at inference.

0xSero

27,083 views • 2 months ago

MiniMax-M3 running locally 100+ tok/s Good questions not often that models ask good questions.

MiniMax-M3 running locally 100+ tok/s Good questions not often that models ask good questions.

0xSero

32,728 views • 2 months ago

The RTX 3090 is a 5-year-old GPU and it still runs a 27B model at 20 tok/s I tested Qwen3.5:27b across 3 generations of NVIDIA: 5090 → ~60 tok/s 4090 → ~40 tok/s 3090 → ~20 tok/s Perfectly linear scaling. Double the generation, double the speed.

The RTX 3090 is a 5-year-old GPU and it still runs a 27B model at 20 tok/s I tested Qwen3.5:27b across 3 generations of NVIDIA: 5090 → ~60 tok/s 4090 → ~40 tok/s 3090 → ~20 tok/s Perfectly linear scaling. Double the generation, double the speed.

stevibe

142,493 views • 6 months ago

Carne asada at home I love you

Carne asada at home I love you

Jackie

71,626 views • 2 months ago

I love my stay-at-home robot

I love my stay-at-home robot

Michael Kandel

33,477 views • 10 months ago

I use Opus 5 with Clairvoyance to make Claude of Duty using Matt Shumer 's prompt. Single shot. 120fps at 4k.

I use Opus 5 with Clairvoyance to make Claude of Duty using Matt Shumer 's prompt. Single shot. 120fps at 4k.

Brad Wardell

26,446 views • 1 month ago

Almost ready, I guess I will finally work on the onboarding experience. I have been using vllm-studio for work all day, this is so good. I legit haven't noticed a single issue with the model, the outputs or even the app's usability. You will have your coding agent at home.

Almost ready, I guess I will finally work on the onboarding experience. I have been using vllm-studio for work all day, this is so good. I legit haven't noticed a single issue with the model, the outputs or even the app's usability. You will have your coding agent at home.

0xSero

14,301 views • 3 months ago

DFlash v0.1.4 : custom Metal verify kernels for quantized Qwen3 hybrid models, plus significant peak memory reduction at long context. M5 Max 40-core GPU, 64GB, stock mlx_lm baseline: Qwen3.6-35B-A3B-4bit: ► @ 1024 · 138.3 → 300.3 tok/s (2.20x) ► @ 2048 · 135.6 → 246.4 tok/s (1.81x) ► @ 4096 · 134.5 → 208.4 tok/s (1.56x) ► @ 8192 · 133.2 → 177.4 tok/s (1.33x) Qwen3.5-27B-4bit: ► @ 1024 · 33.5 → 79.0 tok/s (2.37x) ► @ 2048 · 33.1 → 70.2 tok/s (2.12x) ► @ 4096 · 31.5 → 55.7 tok/s (1.77x) ► @ 8192 · 33.9 → 45.3 tok/s (1.34x) Working on making this usable for agentic workloads goal is to never drop below baseline at any context depth. LLM decode is memory-bandwidth bound. M5 Max runs at 614 GB/s, that's 1.5x more than M1-M4 Max (400-410 GB/s). Results will vary on lower bandwidth chips.

DFlash v0.1.4 : custom Metal verify kernels for quantized Qwen3 hybrid models, plus significant peak memory reduction at long context. M5 Max 40-core GPU, 64GB, stock mlx_lm baseline: Qwen3.6-35B-A3B-4bit: ► @ 1024 · 138.3 → 300.3 tok/s (2.20x) ► @ 2048 · 135.6 → 246.4 tok/s (1.81x) ► @ 4096 · 134.5 → 208.4 tok/s (1.56x) ► @ 8192 · 133.2 → 177.4 tok/s (1.33x) Qwen3.5-27B-4bit: ► @ 1024 · 33.5 → 79.0 tok/s (2.37x) ► @ 2048 · 33.1 → 70.2 tok/s (2.12x) ► @ 4096 · 31.5 → 55.7 tok/s (1.77x) ► @ 8192 · 33.9 → 45.3 tok/s (1.34x) Working on making this usable for agentic workloads goal is to never drop below baseline at any context depth. LLM decode is memory-bandwidth bound. M5 Max runs at 614 GB/s, that's 1.5x more than M1-M4 Max (400-410 GB/s). Results will vary on lower bandwidth chips.

bstn 👁️

23,120 views • 4 months ago

For those of you wondering what a scrappy home AI rig looks like 192gb VRAM 512gb DDR4 Epyc 7443P Pulling a Max 1600w at peak. This can fit: - MiniMax-M2.5 in 4 bits - 60% of GLM-4.7 at 4 bits - Qwen3-coder-next at full precision - 2x GLM-4.7- Flash Bought for 12k now 18k

For those of you wondering what a scrappy home AI rig looks like 192gb VRAM 512gb DDR4 Epyc 7443P Pulling a Max 1600w at peak. This can fit: - MiniMax-M2.5 in 4 bits - 60% of GLM-4.7 at 4 bits - Qwen3-coder-next at full precision - 2x GLM-4.7- Flash Bought for 12k now 18k

0xSero

47,340 views • 6 months ago

Stay At Home GF S

Stay At Home GF S

The Desirable Truth

33,925 views • 2 months ago

Kimi-K2.5 is the best model at browser use, it's been able to handle complex setups, purchasing stuff, working on Stripe. Scraping, QA, etc.. I love it. I am making it the default model and making a paid experience for Parchi but will keep it OSS.

Kimi-K2.5 is the best model at browser use, it's been able to handle complex setups, purchasing stuff, working on Stripe. Scraping, QA, etc.. I love it. I am making it the default model and making a paid experience for Parchi but will keep it OSS.

0xSero

24,291 views • 6 months ago