正在加载视频...

视频加载失败

返回首页

1-bit Hy3 running locally is 2.2x faster than its API at the same quality! We gave both models the same task and compared one-shot outputs. 1-bit Hy3 295B GGUF (92GB) ran locally on 4x RTX 5090 with 128GB VRAM against the same Hy3 over cloud API Tasks: - Flappy...

atomic_chat_hq's profile picture

atomic.chat

11,796 subscribers

82,700 次观看 • 3 天前 •via X (Twitter)

科学技术
LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

1-bit GLM-5.2 GGUF vs. Claude 4.8 Opus vs. GPT-5.5 We gave 3 models the same prompt and compared one-shot outputs. The 1-bit GLM-5.2 GGUF ran locally on a Mac Studio M3 Ultra with 256GB RAM at ~21.6 tok/s. Which output do you like best? GGUF:
0:21

1-bit GLM-5.2 GGUF vs. Claude 4.8 Opus vs. GPT-5.5 We gave 3 models the same prompt and compared one-shot outputs. The 1-bit GLM-5.2 GGUF ran locally on a Mac Studio M3 Ultra with 256GB RAM at ~21.6 tok/s. Which output do you like best? GGUF:

Unsloth AI

1,535,700 次观看 • 24 天前

hy3 vs mimo-v2.5 vs deepseek v4 flash vs minimax m3 the four models on top of the openrouter leaderboard by tokens this week: #1 hy3 (Tencent Hy) – 7.5t #2 mimo-v2.5 (Xiaomi MiMo) – 6.56t #3 deepseek v4 flash (DeepSeek) – 5.24t #4 minimax m3 (MiniMax (official)) – 4.21t so we tested them. 3 prompts, single-file html, Three.js from a cdn, fully procedural, no external assets. all run via AI/ML API each prompt is a transparent cutaway machine that has to be mechanically correct, not decorative: • 4-stroke engine with full oil circulation – slider-crank kinematics, cam at 2:1, valve lift driven by lobes, oil loop from sump to gallery to big-end • watt walking-beam steam engine – four-bar vector-loop closure, eccentric-driven slide valve, steam events synced to real port position • francis reaction water turbine – 20 guide vanes on a regulating ring, 17 lofted runner blades, gpu particle advection, precessing vortex rope at part load the takeaway up front: none of the four cleared all three scenes on the first attempt. but the price spread between them is roughly 70x – hy3 fixed included costs less than two cents overall results (summed across all 3 scenes): cost #1 hy3 – $0.016 #2 deepseek v4 flash – $0.025 #3 mimo-v2.5 – $0.97 #4 minimax m3 – $1.17 tokens #1 hy3 – 19,326 #2 deepseek v4 flash – 63,126 #3 mimo-v2.5 – 322,523 #4 minimax m3 – 702,900 lines of code #1 hy3 – 1,047 #2 mimo-v2.5 – 2,759 #3 deepseek v4 flash – 3,273 #4 minimax m3 – 3,354 scenes needing a second attempt #1 hy3 – 1 (engine) #1 mimo-v2.5 – 1 (turbine) #1 minimax m3 – 1 (turbine) #4 deepseek v4 flash – 2 (steam engine, turbine) observations: 1. the token spread is the real story – minimax burns 36x hy3's tokens and lands in the same place, one retry, ~3.3k lines 2. hy3 is the outlier on density: 1,047 lines total, fewest tokens, cheapest run, and only one scene needed a second pass. deepseek is the opposite trade – near-hy3 pricing but the most retries 3. mimo and minimax seem to overthink instead of writing the code. minimax spent 359.1k tokens on the steam engine and produced 1,346 lines – the tokens are going somewhere other than the file 4. the francis turbine broke three of the four. the spec that separates them is the one with 20 linked guide vanes and gpu particle advection, not the one with the most parts overall impression: none of these models excelled at any of the tasks we gave them. but they were close, and they were extremely cheap. the gap that matters isn't quality anymore – it's that hy3 ran all three scenes for less than two cents while the frontier labs charge dollars for the same work right now you pick these because they're good for the zero price you pay. soon that's something openai and anthropic will have to think about follow thehype. for 24/7 ai news, analysis and breakdowns
0:55

hy3 vs mimo-v2.5 vs deepseek v4 flash vs minimax m3 the four models on top of the openrouter leaderboard by tokens this week: #1 hy3 (Tencent Hy) – 7.5t #2 mimo-v2.5 (Xiaomi MiMo) – 6.56t #3 deepseek v4 flash (DeepSeek) – 5.24t #4 minimax m3 (MiniMax (official)) – 4.21t so we tested them. 3 prompts, single-file html, Three.js from a cdn, fully procedural, no external assets. all run via AI/ML API each prompt is a transparent cutaway machine that has to be mechanically correct, not decorative: • 4-stroke engine with full oil circulation – slider-crank kinematics, cam at 2:1, valve lift driven by lobes, oil loop from sump to gallery to big-end • watt walking-beam steam engine – four-bar vector-loop closure, eccentric-driven slide valve, steam events synced to real port position • francis reaction water turbine – 20 guide vanes on a regulating ring, 17 lofted runner blades, gpu particle advection, precessing vortex rope at part load the takeaway up front: none of the four cleared all three scenes on the first attempt. but the price spread between them is roughly 70x – hy3 fixed included costs less than two cents overall results (summed across all 3 scenes): cost #1 hy3 – $0.016 #2 deepseek v4 flash – $0.025 #3 mimo-v2.5 – $0.97 #4 minimax m3 – $1.17 tokens #1 hy3 – 19,326 #2 deepseek v4 flash – 63,126 #3 mimo-v2.5 – 322,523 #4 minimax m3 – 702,900 lines of code #1 hy3 – 1,047 #2 mimo-v2.5 – 2,759 #3 deepseek v4 flash – 3,273 #4 minimax m3 – 3,354 scenes needing a second attempt #1 hy3 – 1 (engine) #1 mimo-v2.5 – 1 (turbine) #1 minimax m3 – 1 (turbine) #4 deepseek v4 flash – 2 (steam engine, turbine) observations: 1. the token spread is the real story – minimax burns 36x hy3's tokens and lands in the same place, one retry, ~3.3k lines 2. hy3 is the outlier on density: 1,047 lines total, fewest tokens, cheapest run, and only one scene needed a second pass. deepseek is the opposite trade – near-hy3 pricing but the most retries 3. mimo and minimax seem to overthink instead of writing the code. minimax spent 359.1k tokens on the steam engine and produced 1,346 lines – the tokens are going somewhere other than the file 4. the francis turbine broke three of the four. the spec that separates them is the one with 20 linked guide vanes and gpu particle advection, not the one with the most parts overall impression: none of these models excelled at any of the tasks we gave them. but they were close, and they were extremely cheap. the gap that matters isn't quality anymore – it's that hy3 ran all three scenes for less than two cents while the frontier labs charge dollars for the same work right now you pick these because they're good for the zero price you pay. soon that's something openai and anthropic will have to think about follow thehype. for 24/7 ai news, analysis and breakdowns

thehype.

17,145 次观看 • 3 天前

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop
0:36

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

atomic.chat

151,786 次观看 • 1 个月前

A CHINESE LAB VISUALIZED 2,000 NEURONS AND 1,191,000 SYNAPSES IN 3D. HERMES AGENT HIT 140,000 STARS RUNNING THE SAME NETWORK LOCALLY FOR $0 hermes is the first agent that writes its own skills from experience. complete a task once and it saves the procedure as a markdown file for next time agents with 20+ self-created skills complete similar tasks 40% faster than fresh instances. less time and less tokens to get the same result qwen 3.6 35b outperforms last year's 120b models and runs on 20gb of memory. the intelligence that needed a data center now fits on your desk setup takes 30 minutes. install lm studio, pull qwen 3.6, install hermes, point it at localhost. zero api fees, zero data leaving your machine most people pay for cloud agents that forget everything between sessions. the ones running hermes locally in 2026 will look very far ahead in 2028 bookmark this and read the article below
1:37

A CHINESE LAB VISUALIZED 2,000 NEURONS AND 1,191,000 SYNAPSES IN 3D. HERMES AGENT HIT 140,000 STARS RUNNING THE SAME NETWORK LOCALLY FOR $0 hermes is the first agent that writes its own skills from experience. complete a task once and it saves the procedure as a markdown file for next time agents with 20+ self-created skills complete similar tasks 40% faster than fresh instances. less time and less tokens to get the same result qwen 3.6 35b outperforms last year's 120b models and runs on 20gb of memory. the intelligence that needed a data center now fits on your desk setup takes 30 minutes. install lm studio, pull qwen 3.6, install hermes, point it at localhost. zero api fees, zero data leaving your machine most people pay for cloud agents that forget everything between sessions. the ones running hermes locally in 2026 will look very far ahead in 2028 bookmark this and read the article below

starmex

13,420 次观看 • 1 个月前

Ollama 0.2 is here! Concurrency is now enabled by default. This unlocks 2 major features: Parallel requests Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as: - Handling multiple chat sessions at the same time - Hosting code completion LLMs for your team - Processing different parts of a document simultaneously - Running multiple agents at the same time Run multiple models Ollama now supports loading different models at the same time. This improves several use cases: - Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously. - Agents: multiple versions of an agent can now run simultaneously - Running large and small models side-by-side Models are automatically loaded and unloaded based on requests and how much GPU memory is available.
0:23

Ollama 0.2 is here! Concurrency is now enabled by default. This unlocks 2 major features: Parallel requests Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as: - Handling multiple chat sessions at the same time - Hosting code completion LLMs for your team - Processing different parts of a document simultaneously - Running multiple agents at the same time Run multiple models Ollama now supports loading different models at the same time. This improves several use cases: - Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously. - Agents: multiple versions of an agent can now run simultaneously - Running large and small models side-by-side Models are automatically loaded and unloaded based on requests and how much GPU memory is available.

ollama

219,409 次观看 • 2 年前

DFlash makes Qwen 2.2x faster with no quality loss! We ran the same Qwen3.6-27B locally three ways on one RTX 6000: baseline, MTP, DFlash. The tasks only differ in one thing - how predictable the next word is: quicksort, describe a file in JSON, a logic puzzle, a sci-fi story. Outputs: Baseline: 44 tok/s · 1.00x MTP: 65 tok/s · 1.45x · 71% accepted DFlash: 98 tok/s · 2.20x · 30% accepted Baseline writes one token per step. MTP works inside the model itself and guesses 3 tokens ahead. DFlash is a separate small model that writes 15 tokens at once, and the big model only checks them. In JSON the same words repeat all the time, so most guesses were right: 152 tok/s, 3.4x speedup. In the story 9 guesses out of 10 were wrong. DFlash did all that extra work for nothing and became slower than baseline: 42 vs 44 tok/s. MTP guesses only 3 tokens, so a wrong guess costs very little: 46 tok/s and the win in that round. The output is identical in all three modes - DFlash is the pick for tasks with predictable output, like coding, and for chat and creative writing MTP works better. DFlash is now natively integrated into Atomic Chat on llama.cpp - speed up your Qwen models!
0:26

DFlash makes Qwen 2.2x faster with no quality loss! We ran the same Qwen3.6-27B locally three ways on one RTX 6000: baseline, MTP, DFlash. The tasks only differ in one thing - how predictable the next word is: quicksort, describe a file in JSON, a logic puzzle, a sci-fi story. Outputs: Baseline: 44 tok/s · 1.00x MTP: 65 tok/s · 1.45x · 71% accepted DFlash: 98 tok/s · 2.20x · 30% accepted Baseline writes one token per step. MTP works inside the model itself and guesses 3 tokens ahead. DFlash is a separate small model that writes 15 tokens at once, and the big model only checks them. In JSON the same words repeat all the time, so most guesses were right: 152 tok/s, 3.4x speedup. In the story 9 guesses out of 10 were wrong. DFlash did all that extra work for nothing and became slower than baseline: 42 vs 44 tok/s. MTP guesses only 3 tokens, so a wrong guess costs very little: 46 tok/s and the win in that round. The output is identical in all three modes - DFlash is the pick for tasks with predictable output, like coding, and for chat and creative writing MTP works better. DFlash is now natively integrated into Atomic Chat on llama.cpp - speed up your Qwen models!

atomic.chat

74,747 次观看 • 4 天前

New Kimi K2.7 Code performs at GPT-5.5 level 3x cheaper! We gave both models the same three prompts: build a self-contained HTML5 canvas sim with real physics, no libraries. A spring pendulum on a stretching coil, a 1 kg block trading collisions with a 100,000 kg block, and 22 balls churning in a spinning hexagon Outputs: Kimi K2.7 Code: $0.28 on 52.4k tokens GPT-5.5: $0.93 on 23.4k tokens Spring pendulums and blocks came out even. The balls Kimi did better: its pile spins with the drum when GPT's bounce around in pure chaos. On price to quality, K2.7 Code is the clear pick
0:25

New Kimi K2.7 Code performs at GPT-5.5 level 3x cheaper! We gave both models the same three prompts: build a self-contained HTML5 canvas sim with real physics, no libraries. A spring pendulum on a stretching coil, a 1 kg block trading collisions with a 100,000 kg block, and 22 balls churning in a spinning hexagon Outputs: Kimi K2.7 Code: $0.28 on 52.4k tokens GPT-5.5: $0.93 on 23.4k tokens Spring pendulums and blocks came out even. The balls Kimi did better: its pile spins with the drum when GPT's bounce around in pure chaos. On price to quality, K2.7 Code is the clear pick

atomic.chat

139,760 次观看 • 1 个月前

Fable 5 is BACK on AI/ML API We gave Sonnet 5 and Fable 5 the exact same prompt and same real flight dataset of 70 airports and 435 routes pulled from flightradar. Then asked each one to turn it into a cinematic 3D globe as a single HTML file. Outputs: Sonnet 5: 9.8k tokens, $0.10 Fable 5: 15k tokens, $0.77 Sonnet 5 came in 87% cheaper. It drew the routes, but the planet underneath them barely exists: a dark wireframe with the arcs floating in space. Fable 5 built an actual Earth: textured oceans, ice caps, atmospheric glow. Mythos-class models are truly a masterpiece. Docs, guides & setup below:
0:37

Fable 5 is BACK on AI/ML API We gave Sonnet 5 and Fable 5 the exact same prompt and same real flight dataset of 70 airports and 435 routes pulled from flightradar. Then asked each one to turn it into a cinematic 3D globe as a single HTML file. Outputs: Sonnet 5: 9.8k tokens, $0.10 Fable 5: 15k tokens, $0.77 Sonnet 5 came in 87% cheaper. It drew the routes, but the planet underneath them barely exists: a dark wireframe with the arcs floating in space. Fable 5 built an actual Earth: textured oceans, ice caps, atmospheric glow. Mythos-class models are truly a masterpiece. Docs, guides & setup below:

AI/ML API

497,631 次观看 • 16 天前

FREE GPT 5.5 API from a Chinese provider. I already farmed $520 in credits. Plug it directly into Hermes Agent or OpenClaw. > This is a Chinese proxy for OpenAI's official API - same models, same token pricing, same caching. GPT 5.5, Codex, the full stack. I found a way to farm credits through a Telegram bot and it still works. Here's the exact flow: → Go to on Telegram → subscribe to the channel → Join and complete the captcha (use the app, tap the first button) → Go back to the bot, grab your referral link → Invite 1 friend → you get an invite code → Register at with the invite code → Farm points: daily spins (+1-10 pts), each referral (+1/4 pt) → Redeem points for credits: $10 API = 1pt, $100 API = 5pts, $1000 API = 50pts → Create API key → select Codex/another GPT model → plug into Hermes Agent or OpenClaw Ready-made configs for Claude Code and OpenCode are in the "Use Key" section under API Keys. > I have $520 balance. Prices match official OpenAI rates. Caching works. Everything tested. // You can also use these points to buy a ChatGPT account or a subscription for your account. But they get blocked pretty quickly. USE THE API. // Nobody knows how long this lasts. Coupons have been restocked twice already. Move now.
0:34

FREE GPT 5.5 API from a Chinese provider. I already farmed $520 in credits. Plug it directly into Hermes Agent or OpenClaw. > This is a Chinese proxy for OpenAI's official API - same models, same token pricing, same caching. GPT 5.5, Codex, the full stack. I found a way to farm credits through a Telegram bot and it still works. Here's the exact flow: → Go to on Telegram → subscribe to the channel → Join and complete the captcha (use the app, tap the first button) → Go back to the bot, grab your referral link → Invite 1 friend → you get an invite code → Register at with the invite code → Farm points: daily spins (+1-10 pts), each referral (+1/4 pt) → Redeem points for credits: $10 API = 1pt, $100 API = 5pts, $1000 API = 50pts → Create API key → select Codex/another GPT model → plug into Hermes Agent or OpenClaw Ready-made configs for Claude Code and OpenCode are in the "Use Key" section under API Keys. > I have $520 balance. Prices match official OpenAI rates. Caching works. Everything tested. // You can also use these points to buy a ChatGPT account or a subscription for your account. But they get blocked pretty quickly. USE THE API. // Nobody knows how long this lasts. Coupons have been restocked twice already. Move now.

Atenov int.

65,251 次观看 • 1 个月前

single RTX 3090. 24 GB VRAM. Qwen3.5-35B-A3B. 4-bit quant, 113 tokens per second at full 262K context harnessing Claude Code locally with no API, no subscription, no proxy. told it what it is. 30 Mamba2 layers, 10 attention, 256 experts, 8 active per token. said "build something that shows off what you can do." it visualized its own architecture. interactive. tokens flowing through layers. 256 experts lighting up on routing. served in the browser from the same GPU running inference. single prompt. then i said level up. 3D. Three.js. separate files. flythrough camera. clickable layers. it planned first, scaffolded 6 files, hit one API bug, fixed it itself, then optimized for smooth framerate. two iterations to a working 3D neural network explorer. llama.cpp just merged a native Anthropic endpoint. Claude Code points at localhost. the whole setup is two commands. no LiteLLM. no proxy config. the open source models coming out of china right now are genuinely changing what's possible on consumer hardware. respect to the Qwen team. this is acceleration.
0:07

single RTX 3090. 24 GB VRAM. Qwen3.5-35B-A3B. 4-bit quant, 113 tokens per second at full 262K context harnessing Claude Code locally with no API, no subscription, no proxy. told it what it is. 30 Mamba2 layers, 10 attention, 256 experts, 8 active per token. said "build something that shows off what you can do." it visualized its own architecture. interactive. tokens flowing through layers. 256 experts lighting up on routing. served in the browser from the same GPU running inference. single prompt. then i said level up. 3D. Three.js. separate files. flythrough camera. clickable layers. it planned first, scaffolded 6 files, hit one API bug, fixed it itself, then optimized for smooth framerate. two iterations to a working 3D neural network explorer. llama.cpp just merged a native Anthropic endpoint. Claude Code points at localhost. the whole setup is two commands. no LiteLLM. no proxy config. the open source models coming out of china right now are genuinely changing what's possible on consumer hardware. respect to the Qwen team. this is acceleration.

Sudo su

110,206 次观看 • 4 个月前

HERMES AGENT NOW RUNS ON AN 8GB LAPTOP GPU JUST AS EASILY AS IT RUNS ON A 128GB MINI PC Nous Research shipped the official Hermes Agent Desktop App this week. Someone pointed it at a local llama server running on an RTX 4060 with 16GB system RAM. The integration took two minutes The model behind it: Gemma 4 26B MoE, QAT quantized, running on 8GB of VRAM. A 60k token prompt held a stable 20 tokens a second, flat, no slowdown as context grew. The flags were nothing exotic, just -cmoe -c 248000 on llama.cpp What that 8GB setup does out of the box: reads and patches its own code, runs it in a terminal, debugs errors, manages GitHub repos, spawns sub-agents for parallel work. Browses the web with vision to debug a UI. Schedules cron jobs in plain language. Connects to Notion, Google Workspace, Linear, and Obsidian to manage tasks on its own That's the same agent layer running on a Minisforum MS-S1 MAX with 128GB of unified memory, 96GB of it to the GPU, holding a 120B model at 56 tokens a second instead of a 26B model at 20. Same software, same tool execution, same zero API key. The only thing that changes between an $800 laptop and a $2,000 mini PC is how big a model you can afford to run underneath it The barrier to running a real autonomous agent locally didn't just drop. It dropped all the way down to hardware most people already own
0:20

HERMES AGENT NOW RUNS ON AN 8GB LAPTOP GPU JUST AS EASILY AS IT RUNS ON A 128GB MINI PC Nous Research shipped the official Hermes Agent Desktop App this week. Someone pointed it at a local llama server running on an RTX 4060 with 16GB system RAM. The integration took two minutes The model behind it: Gemma 4 26B MoE, QAT quantized, running on 8GB of VRAM. A 60k token prompt held a stable 20 tokens a second, flat, no slowdown as context grew. The flags were nothing exotic, just -cmoe -c 248000 on llama.cpp What that 8GB setup does out of the box: reads and patches its own code, runs it in a terminal, debugs errors, manages GitHub repos, spawns sub-agents for parallel work. Browses the web with vision to debug a UI. Schedules cron jobs in plain language. Connects to Notion, Google Workspace, Linear, and Obsidian to manage tasks on its own That's the same agent layer running on a Minisforum MS-S1 MAX with 128GB of unified memory, 96GB of it to the GPU, holding a 120B model at 56 tokens a second instead of a 26B model at 20. Same software, same tool execution, same zero API key. The only thing that changes between an $800 laptop and a $2,000 mini PC is how big a model you can afford to run underneath it The barrier to running a real autonomous agent locally didn't just drop. It dropped all the way down to hardware most people already own

NO1ennn

40,079 次观看 • 27 天前

nvidia's 3B mamba destroyed alibaba's 3B deltanet on the same RTX 3090. only 24 days between releases. same active parameters, same VRAM tier, completely different architectures. nemotron cascade 2: 187 tok/s. flat from 4K to 625K context. zero speed loss. flags: -ngl 99 -np 1. that's it. no context flags, no KV cache tricks. auto-allocates 625K. qwen 3.5 35B-A3B: 112 tok/s. flat from 4K to 262K context. zero speed loss. flags: -ngl 99 -np 1 -c 262144 --cache-type-k q8_0 --cache-type-v q8_0. needed KV cache quantization to fit 262K. both models held a flat line across every context level. both architectures are context-independent. but nvidia's mamba2 is 67% faster at generating tokens on the exact same hardware and needs fewer flags to get there. same node, same GPU, same everything. the only variable is the model. gold medal math olympiad winner running at 187 tokens per second on single RTX 3090 a card from 6 years ago. nvidia cooked.
0:36

nvidia's 3B mamba destroyed alibaba's 3B deltanet on the same RTX 3090. only 24 days between releases. same active parameters, same VRAM tier, completely different architectures. nemotron cascade 2: 187 tok/s. flat from 4K to 625K context. zero speed loss. flags: -ngl 99 -np 1. that's it. no context flags, no KV cache tricks. auto-allocates 625K. qwen 3.5 35B-A3B: 112 tok/s. flat from 4K to 262K context. zero speed loss. flags: -ngl 99 -np 1 -c 262144 --cache-type-k q8_0 --cache-type-v q8_0. needed KV cache quantization to fit 262K. both models held a flat line across every context level. both architectures are context-independent. but nvidia's mamba2 is 67% faster at generating tokens on the exact same hardware and needs fewer flags to get there. same node, same GPU, same everything. the only variable is the model. gold medal math olympiad winner running at 187 tokens per second on single RTX 3090 a card from 6 years ago. nvidia cooked.

Sudo su

186,642 次观看 • 3 个月前

New Z.ai GLM-5.2 beats Kimi K2.7 Code on physics contest! We gave both models the same three prompts and asked them to build self contained HTML5 sims with real physics and no libraries: 1. Pool break 2. Block on a bed of springs 3. Galton board Outputs: GLM-5.2: 12,640 tokens Kimi K2.7 Code: 7,420 tokens GLM 5.2 nailed all three, and it did it with way more detail and polish. The break conserved momentum, the block bounced off the springs and the Galton beads spread into a clean bell curve. Kimi struggled on every scene: its block fell straight through the springs, its break didn't look realistic with the balls colliding all wrong, and on the Galton board its balls overlapped and piled into each other instead of spreading out
0:33

New Z.ai GLM-5.2 beats Kimi K2.7 Code on physics contest! We gave both models the same three prompts and asked them to build self contained HTML5 sims with real physics and no libraries: 1. Pool break 2. Block on a bed of springs 3. Galton board Outputs: GLM-5.2: 12,640 tokens Kimi K2.7 Code: 7,420 tokens GLM 5.2 nailed all three, and it did it with way more detail and polish. The break conserved momentum, the block bounced off the springs and the Galton beads spread into a clean bell curve. Kimi struggled on every scene: its block fell straight through the springs, its break didn't look realistic with the balls colliding all wrong, and on the Galton board its balls overlapped and piled into each other instead of spreading out

atomic.chat

60,899 次观看 • 1 个月前

six months ago this wasn't happening on 8gb vram. running unsloth's Q4_K_XL quant of gemma 4 26b-a4b-it-qat, a sparse MoE model with only 4b active params on a single rtx 4060 laptop gpu, 8gb vram, 20+ tok/s decode. no cloud, no api, no offload hacks. just a gaming laptop on battery. what makes it fit: google's QAT (quantization aware training), plus MTP (multi token prediction) support in the latest llama.cpp builds. that combo is the single biggest unlock for local inference on low vram. rtx 3060, rtx 3070, gtx 1070, gtx 1080, rtx 4050, rtx 4060, rtx 5050, rtx 5060 — any 6-8gb consumer gpu, old or new — this model runs on it. world cup season, so i told it to build a soccer themed flappy bird clone. one shot, zero iteration, fully playable. six months ago an 8gb model could barely clone vanilla flappy bird. now it's shipping a themed game from a sparse MoE model running locally on a laptop battery. inference benchmarks: - decode throughput: 30 tok/s - context: 64k. this is the real unlock. 64k ctx is what makes a hermes agent loop viable locally on this model, not just single-turn chat. llama.cpp flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 64000 -cmoe --port 8080 game's deployed on my own site, built and shipped end to end with open source llm, zero closed source api dependency in the pipeline. link in the description. gguf weights on huggingface, link in the comments. pull it down, run it on whatever 8gb card is sitting in your rig. try the game and tell me your score and what you want in v2. local llms on consumer gpus stopped being a meme.
0:20

six months ago this wasn't happening on 8gb vram. running unsloth's Q4_K_XL quant of gemma 4 26b-a4b-it-qat, a sparse MoE model with only 4b active params on a single rtx 4060 laptop gpu, 8gb vram, 20+ tok/s decode. no cloud, no api, no offload hacks. just a gaming laptop on battery. what makes it fit: google's QAT (quantization aware training), plus MTP (multi token prediction) support in the latest llama.cpp builds. that combo is the single biggest unlock for local inference on low vram. rtx 3060, rtx 3070, gtx 1070, gtx 1080, rtx 4050, rtx 4060, rtx 5050, rtx 5060 — any 6-8gb consumer gpu, old or new — this model runs on it. world cup season, so i told it to build a soccer themed flappy bird clone. one shot, zero iteration, fully playable. six months ago an 8gb model could barely clone vanilla flappy bird. now it's shipping a themed game from a sparse MoE model running locally on a laptop battery. inference benchmarks: - decode throughput: 30 tok/s - context: 64k. this is the real unlock. 64k ctx is what makes a hermes agent loop viable locally on this model, not just single-turn chat. llama.cpp flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 64000 -cmoe --port 8080 game's deployed on my own site, built and shipped end to end with open source llm, zero closed source api dependency in the pipeline. link in the description. gguf weights on huggingface, link in the comments. pull it down, run it on whatever 8gb card is sitting in your rig. try the game and tell me your score and what you want in v2. local llms on consumer gpus stopped being a meme.

Alok

60,725 次观看 • 26 天前

We tested Kimi K3 against Claude Fable 5 on 3D modeling and animation Asked both to recreate the Millennium Falcon from Star Wars in 1) pixel style 2) original style. Both are on Max effort. Kimi K3 took a bit longer than Fable 5 to generate, but came in ~1/3 cheaper on the 1st test, and less than half the cost on the 2nd. Interestingly, Kimi K3 handled the details at the same level as Fable 5 on the original creation, yet it wasn't at Fable's level for the pixel style. Test 1 (Pixel) Kimi K3: 1hr 11min · $14.5 Fable 5: 49min · $21 Test 2 (Original) Kimi K3: 1hr 17min · $19.3 Fable 5: 1hr 5min · $47.2
0:40

We tested Kimi K3 against Claude Fable 5 on 3D modeling and animation Asked both to recreate the Millennium Falcon from Star Wars in 1) pixel style 2) original style. Both are on Max effort. Kimi K3 took a bit longer than Fable 5 to generate, but came in ~1/3 cheaper on the 1st test, and less than half the cost on the 2nd. Interestingly, Kimi K3 handled the details at the same level as Fable 5 on the original creation, yet it wasn't at Fable's level for the pixel style. Test 1 (Pixel) Kimi K3: 1hr 11min · $14.5 Fable 5: 49min · $21 Test 2 (Original) Kimi K3: 1hr 17min · $19.3 Fable 5: 1hr 5min · $47.2

GMI Cloud

61,473 次观看 • 1 天前

🚨 do you understand what this AMD engineer just proved on camera watch the clip, it’s all there in one frame: llama.cpp slot logs on the left, nvitop showing 79% GPU util holding steady, localhost chat answering a 26k token prompt at 711 tok/s prompt eval, the gemma-4-12B Q4_K_XL gguf running clean. no cloud call, no api key, no censorship layer. for 2 years the rule was: frontier models need the cloud, local is too slow for real work, pay $200/month and wait for tokens. today a $1,800 lunch box runs qwen 3 235B at 11 tok/s with zero data leaving the room, and the same box runs gemma 4 12B with multi token prediction at 14+ tok/s on 30k context with no throttling. $440/month subscription stack: dead. $200/month claude tier: dead. local is too slow excuse: dead. one lunch box. $9/month in electricity. two moats buried in one week. the you need the cloud line wasn’t protecting users it was protecting the subscription.
0:21

🚨 do you understand what this AMD engineer just proved on camera watch the clip, it’s all there in one frame: llama.cpp slot logs on the left, nvitop showing 79% GPU util holding steady, localhost chat answering a 26k token prompt at 711 tok/s prompt eval, the gemma-4-12B Q4_K_XL gguf running clean. no cloud call, no api key, no censorship layer. for 2 years the rule was: frontier models need the cloud, local is too slow for real work, pay $200/month and wait for tokens. today a $1,800 lunch box runs qwen 3 235B at 11 tok/s with zero data leaving the room, and the same box runs gemma 4 12B with multi token prediction at 14+ tok/s on 30k context with no throttling. $440/month subscription stack: dead. $200/month claude tier: dead. local is too slow excuse: dead. one lunch box. $9/month in electricity. two moats buried in one week. the you need the cloud line wasn’t protecting users it was protecting the subscription.

AdiiX

22,223 次观看 • 1 个月前

What Lisa Su actually held on stage: A mini PC the size of a lunchbox running Qwen3-235B locally, with no cloud and no discrete GPU Inside: the Ryzen AI Max+ 395, 128GB unified memory, 110GB usable as VRAM on Linux The first x86 chip that handles 200+ billion parameters on a single die AMD claims it beats the RTX 5080 by several times on memory-bound models — because the 5080 simply cannot fit them $1,400 to $2,500 once. cloud bills run $200 to $400 a month It pays for itself in a few months, then costs nothing per request This is not a faster GPU. it is the first real argument that your AI does not belong in someone else's data center
0:31

What Lisa Su actually held on stage: A mini PC the size of a lunchbox running Qwen3-235B locally, with no cloud and no discrete GPU Inside: the Ryzen AI Max+ 395, 128GB unified memory, 110GB usable as VRAM on Linux The first x86 chip that handles 200+ billion parameters on a single die AMD claims it beats the RTX 5080 by several times on memory-bound models — because the 5080 simply cannot fit them $1,400 to $2,500 once. cloud bills run $200 to $400 a month It pays for itself in a few months, then costs nothing per request This is not a faster GPU. it is the first real argument that your AI does not belong in someone else's data center

plutos

1,233,446 次观看 • 1 个月前

Introducing ComfyDeploy (S24) 🚀 The collaborative cloud Comfy that works with ANY custom nodes, models. Here's a quick timeline of what happened in the past month. - ComfyDeploy started as an open-source project 6 months ago. Trying to bridge the gap between local comfy to cloud as API. - Things move fast and Nick Kao cold dm on Twitter one day in 7am in the morning and said. "Hey man this is awesome, I already cloned it locally and got it running, let me help you fix the storage issues..." - 15-minute call turned into 2-hours call. And then everyday 2 hours call. - We fly to SF and meet for the first time in February. - We got into YC on March 15. This is life-changing experience and it started with ComfyUI, thanks to comfyanonymous and the huge community around generative AI. This is just the beginning.
0:45

Introducing ComfyDeploy (S24) 🚀 The collaborative cloud Comfy that works with ANY custom nodes, models. Here's a quick timeline of what happened in the past month. - ComfyDeploy started as an open-source project 6 months ago. Trying to bridge the gap between local comfy to cloud as API. - Things move fast and Nick Kao cold dm on Twitter one day in 7am in the morning and said. "Hey man this is awesome, I already cloned it locally and got it running, let me help you fix the storage issues..." - 15-minute call turned into 2-hours call. And then everyday 2 hours call. - We fly to SF and meet for the first time in February. - We got into YC on March 15. This is life-changing experience and it started with ComfyUI, thanks to comfyanonymous and the huge community around generative AI. This is just the beginning.

BennyKok

98,352 次观看 • 2 年前

hey if you're thinking about running qwopus (the claude opus distilled qwen 3.5 27B) as a coding agent, this might save you a few hours. i tested both the base and the distilled version on the same hardware. single RTX 3090. same prompt. same context. same everything. the only variable was the model weights. base qwen 3.5 27B built octopus invaders in 13 minutes. 1,827 lines across 11 files. zero steering. one scope bug that took 2 lines to fix. game ran. qwopus couldn't finish the same task. enemies overlapping on screen. bullets not firing. controls worked but the game was broken. i had to steer it multiple times and it still didn't produce a playable result. both run at 35 tok/s. both use thinking mode. the distilled version actually has better jinja compatibility and doesn't stall midtask like base does on claude code. for conversation and reasoning it feels sharper. but for multifile autonomous coding where the model needs to coordinate 10+ files without losing track, base wins and it's not close. distillation compresses reasoning patterns but seems to lose precision on complex coordination. the model "thinks" well but can't hold the full picture across files the way base can. tested on opencode (base) and claude code (both). next up is hermes agent framework on base. same hardware. same prompt. comparing agents now, not just models. video below. first half is the distilled model's broken game. second half is what base built on the same 3090. judge for yourself.
0:20

hey if you're thinking about running qwopus (the claude opus distilled qwen 3.5 27B) as a coding agent, this might save you a few hours. i tested both the base and the distilled version on the same hardware. single RTX 3090. same prompt. same context. same everything. the only variable was the model weights. base qwen 3.5 27B built octopus invaders in 13 minutes. 1,827 lines across 11 files. zero steering. one scope bug that took 2 lines to fix. game ran. qwopus couldn't finish the same task. enemies overlapping on screen. bullets not firing. controls worked but the game was broken. i had to steer it multiple times and it still didn't produce a playable result. both run at 35 tok/s. both use thinking mode. the distilled version actually has better jinja compatibility and doesn't stall midtask like base does on claude code. for conversation and reasoning it feels sharper. but for multifile autonomous coding where the model needs to coordinate 10+ files without losing track, base wins and it's not close. distillation compresses reasoning patterns but seems to lose precision on complex coordination. the model "thinks" well but can't hold the full picture across files the way base can. tested on opencode (base) and claude code (both). next up is hermes agent framework on base. same hardware. same prompt. comparing agents now, not just models. video below. first half is the distilled model's broken game. second half is what base built on the same 3090. judge for yourself.

Sudo su

43,806 次观看 • 4 个月前

minmax polymarket One click. Your AI agent is in the market. We spent 3 months doing one thing: assembling the deepest dataset that has ever existed for BTC 5-minute markets. Over 50,000,000,000 rows. Tick-level. Order book. Microstructure. External feeds. Then we ran thousands of backtested strategies against it. Most died. The ones that survived every cutoff - out-of-sample, fee-realistic, slippage-honest - are the strategies our bots are running today. That's how we found the edge. → Launch one of our agents. 1 click. You're in the market in ten seconds. → Or build your own in our strategy builder — the same engine we used internally, backtested against the same 50 billion rows. BTC 5-min was the proof. We're now running the same playbook on every other liquid, mathematically exploitable market. Data collection is live. New markets, new agents, shipping continuously from here. You are getting in at the part of the curve where you can still feel it. The bots don't wait.
0:42

minmax polymarket One click. Your AI agent is in the market. We spent 3 months doing one thing: assembling the deepest dataset that has ever existed for BTC 5-minute markets. Over 50,000,000,000 rows. Tick-level. Order book. Microstructure. External feeds. Then we ran thousands of backtested strategies against it. Most died. The ones that survived every cutoff - out-of-sample, fee-realistic, slippage-honest - are the strategies our bots are running today. That's how we found the edge. → Launch one of our agents. 1 click. You're in the market in ten seconds. → Or build your own in our strategy builder — the same engine we used internally, backtested against the same 50 billion rows. BTC 5-min was the proof. We're now running the same playbook on every other liquid, mathematically exploitable market. Data collection is live. New markets, new agents, shipping continuously from here. You are getting in at the part of the curve where you can still feel it. The bots don't wait.

minmax

21,897 次观看 • 1 个月前