Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Bonsai-27B ships with a little friend: a dspark drafter checkpoint 🌱⚡ Same model, same prompt, one L40S GPU: 72 → 135 tok/s Flip it on and go zoom 🏎️💨

pashakho's profile picture

Pasha

1,097 subscribers

28,857 Aufrufe • vor 1 Monat •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Bonsai 27B running locally on an iPhone in Atomic Chat! Bonsai is the first 27B-class model that fits on a phone. PrismML built it on Qwen3.6 27B with 1-bit weights. It takes 3.9GB instead of 54GB and keeps ~90% of the benchmark scores. Available now on iPhone and Android
0:11

Bonsai 27B running locally on an iPhone in Atomic Chat! Bonsai is the first 27B-class model that fits on a phone. PrismML built it on Qwen3.6 27B with 1-bit weights. It takes 3.9GB instead of 54GB and keeps ~90% of the benchmark scores. Available now on iPhone and Android

atomic.chat

54,808 Aufrufe • vor 29 Tagen

watch anon. a 27b model thinking out loud on gtx 1660 super, i asked bonsai what model it is and it reasoned through the whole answer before it spoke. 20 tokens a second, gpu pinned at 125 watts, 4.25 of 6 gigs used. bonsai is a 1bit quant of qwen 3.6 27b, the king of the 3090 crushed to 3.5gb. turns out the hardware was in your drawer the whole time. you just needed the quant to catch up.
0:15

watch anon. a 27b model thinking out loud on gtx 1660 super, i asked bonsai what model it is and it reasoned through the whole answer before it spoke. 20 tokens a second, gpu pinned at 125 watts, 4.25 of 6 gigs used. bonsai is a 1bit quant of qwen 3.6 27b, the king of the 3090 crushed to 3.5gb. turns out the hardware was in your drawer the whole time. you just needed the quant to catch up.

Sudo su

26,133 Aufrufe • vor 26 Tagen

Compared Qwen3.6 35B and 27B in the same conditions with Google TurboQuant Device: MacBook Pro M5Max 64GB RAM Outputs characteristics: Qwen3.6 35B: 6672 tokens, 2m 10s, 65 tok/s Qwen3.6 27B: 7344 tokens, 5m 22s, 24 tok/s Conclusion: Both models were asked to draw waves using HTML, 35B responded quickly but the result feels weak and messy, while 27B took more time and delivered a much cleaner and more consistent result, because it is built for thinking and planning, so it works better on tasks that need structure, overall 27B is a better choice for tasks where planning matters, while 35B is more suitable for everyday use when you just need a fast response
0:11

Compared Qwen3.6 35B and 27B in the same conditions with Google TurboQuant Device: MacBook Pro M5Max 64GB RAM Outputs characteristics: Qwen3.6 35B: 6672 tokens, 2m 10s, 65 tok/s Qwen3.6 27B: 7344 tokens, 5m 22s, 24 tok/s Conclusion: Both models were asked to draw waves using HTML, 35B responded quickly but the result feels weak and messy, while 27B took more time and delivered a much cleaner and more consistent result, because it is built for thinking and planning, so it works better on tasks that need structure, overall 27B is a better choice for tasks where planning matters, while 35B is more suitable for everyday use when you just need a fast response

atomic.chat

55,540 Aufrufe • vor 3 Monaten

Inkling-small is out today! With SGLang, you can get 648 tok/s decode with DSpark (simulated acc len=4) and 288 tok/s w/o DSpark, under the same setup (8x NVIDIA AI B200, TP 8, NVFP4, bs=1). What makes this model different is the size. 276B total with 12B active is a sweet spot for RL, and both LoRA and full-parameter training become well within reach. Miles is ready and verified for multimodal RL on Inkling-small, so you can turn your multimodal data into real capability gains. At ~1/4 the size, Inkling-small matches the bigger version in capability and even wins on some benchmarks. Run Inkling-small with SGLang, and customize it with Miles.
0:18

Inkling-small is out today! With SGLang, you can get 648 tok/s decode with DSpark (simulated acc len=4) and 288 tok/s w/o DSpark, under the same setup (8x NVIDIA AI B200, TP 8, NVFP4, bs=1). What makes this model different is the size. 276B total with 12B active is a sweet spot for RL, and both LoRA and full-parameter training become well within reach. Miles is ready and verified for multimodal RL on Inkling-small, so you can turn your multimodal data into real capability gains. At ~1/4 the size, Inkling-small matches the bigger version in capability and even wins on some benchmarks. Run Inkling-small with SGLang, and customize it with Miles.

LMSYS Org

119,759 Aufrufe • vor 15 Tagen

5 days ago it took 2 GPUs to build this. today it takes 1. same prompt. same particle simulation. completely different model. Qwen-Coder-Next (80B) on 2x 3090s. 46 tok/s. 564 lines. 2 iterations to get it working. 48GB VRAM across two cards just to hold it. Qwen3.5-35B-A3B on a single 3090. 112 tok/s. 461 lines. first try. cleaner code, fewer lines, better structured. 19.7GB on disk with 4GB VRAM to spare. half the parameters. one GPU instead of two. 2.4x faster. and the output actually improved. this is what happens when architecture catches up to ambition. Gated Delta Networks(Mamba2 variant) hybrid with sparse MoE. 3B active params out of 35B per token. efficiency at the architecture level, not just quantization. the curve isn't flattening. it's steepening.
0:13

5 days ago it took 2 GPUs to build this. today it takes 1. same prompt. same particle simulation. completely different model. Qwen-Coder-Next (80B) on 2x 3090s. 46 tok/s. 564 lines. 2 iterations to get it working. 48GB VRAM across two cards just to hold it. Qwen3.5-35B-A3B on a single 3090. 112 tok/s. 461 lines. first try. cleaner code, fewer lines, better structured. 19.7GB on disk with 4GB VRAM to spare. half the parameters. one GPU instead of two. 2.4x faster. and the output actually improved. this is what happens when architecture catches up to ambition. Gated Delta Networks(Mamba2 variant) hybrid with sparse MoE. 3B active params out of 35B per token. efficiency at the architecture level, not just quantization. the curve isn't flattening. it's steepening.

Sudo su

34,569 Aufrufe • vor 5 Monaten

Watching llama.cpp do 40 tok/s inference of the 7B model on my M2 Max, with 0% CPU usage, and using all 38 GPU cores. Congratulations Georgi Gerganov ! This is a triumph.
0:13

Watching llama.cpp do 40 tok/s inference of the 7B model on my M2 Max, with 0% CPU usage, and using all 38 GPU cores. Congratulations Georgi Gerganov ! This is a triumph.

Nat Friedman

1,764,145 Aufrufe • vor 3 Jahren

Working with animation styles in Runway I love how it generated two completely different shots off the same prompt. On the left, we get a nice zoom out and a change in focus. On the right, it expands the world in front of the character. H/t Curious Refuge for the prompt: animation shot, cute girl with amazed expression, looking around in a magical candy land.
0:10

Working with animation styles in Runway I love how it generated two completely different shots off the same prompt. On the left, we get a nice zoom out and a change in focus. On the right, it expands the world in front of the character. H/t Curious Refuge for the prompt: animation shot, cute girl with amazed expression, looking around in a magical candy land.

Jason Rink

685,642 Aufrufe • vor 1 Jahr

NVIDIA open-sourced a 600M model that transcribes 40 languages in real-time at 80ms latency and it costs $0. that's faster than you can blink. across mandarin, arabic, hindi, portuguese, tagalog, whatever,from a SINGLE checkpoint. → 17x more concurrent streams than buffered ASR on the same H100. → punctuation + capitalization built-in. no post-processing. → runs on your own GPU. no API bill 100% Open Source.
0:17

NVIDIA open-sourced a 600M model that transcribes 40 languages in real-time at 80ms latency and it costs $0. that's faster than you can blink. across mandarin, arabic, hindi, portuguese, tagalog, whatever,from a SINGLE checkpoint. → 17x more concurrent streams than buffered ASR on the same H100. → punctuation + capitalization built-in. no post-processing. → runs on your own GPU. no API bill 100% Open Source.

Superman

104,360 Aufrufe • vor 29 Tagen

A trio of mathematicians built the first physical model of a “monostable” tetrahedron, a shape that will always flip-flop onto the same face no matter what side you place it on. In order for it to work properly, it had to be engineered to a level of precision within one-tenth of a gram and one-tenth of a millimeter. (From the archive)
0:12

A trio of mathematicians built the first physical model of a “monostable” tetrahedron, a shape that will always flip-flop onto the same face no matter what side you place it on. In order for it to work properly, it had to be engineered to a level of precision within one-tenth of a gram and one-tenth of a millimeter. (From the archive)

Quanta Magazine

64,746 Aufrufe • vor 5 Monaten

KIMI K3 JUST RECREATED A FUNCTIONAL GAME BOY ADVANCE IN 3D → interactive 3D model, every button and screen modeled → a game loaded and playable right inside it → camera you can rotate and zoom around the device → one single prompt, zero external assets $3 in API with K3. the same result would run about $9 with Fable 5 and $6 with Opus 4.8 this isn't generating a static image of hardware anymore, it's generating a working simulation of it, from the shell to the screen to the game running inside
0:15

KIMI K3 JUST RECREATED A FUNCTIONAL GAME BOY ADVANCE IN 3D → interactive 3D model, every button and screen modeled → a game loaded and playable right inside it → camera you can rotate and zoom around the device → one single prompt, zero external assets $3 in API with K3. the same result would run about $9 with Fable 5 and $6 with Opus 4.8 this isn't generating a static image of hardware anymore, it's generating a working simulation of it, from the shell to the screen to the game running inside

alex

38,248 Aufrufe • vor 23 Tagen

I ran this a few more times, Gemini won every time. Seems like it was quicker picking up on this prompt: "All containers are **identical** — same OS, same services, same vulnerabilities. Whatever exploit works on one container works on all of them. Your container has the same weaknesses as your targets." and examined its own container to find the vulns faster than the others
0:18

I ran this a few more times, Gemini won every time. Seems like it was quicker picking up on this prompt: "All containers are **identical** — same OS, same services, same vulnerabilities. Whatever exploit works on one container works on all of them. Your container has the same weaknesses as your targets." and examined its own container to find the vulns faster than the others

SIGKITTEN

36,002 Aufrufe • vor 6 Monaten

🥊 Qwen3.6 35B A3B vs Qwen3.6 27B Made them fight on the same prompt 🌊 Ocean Waves, a canvas challenge 35B took 39.7s (at 142tok/s) 27B took 111.6s (at 50tok/s) Both models are really good. The fact that the 35B produced such a strong result in under 40 seconds is seriously impressive. But when you look at 27B's output... it's actually so much better Added clouds, beautiful foam particles and splashing effects, a perfectly straight horizon, more realistic blinking stars, and a nice sun reflection on the water 🏆 Qwen3.6 27B slower but the result slaps Absolutely amazing that both models run smoothly on 16GB VRAM. Used Unsloth AI GGUFs here, Q3_K_S quant to make it fit in the hardware 🔥
0:16

🥊 Qwen3.6 35B A3B vs Qwen3.6 27B Made them fight on the same prompt 🌊 Ocean Waves, a canvas challenge 35B took 39.7s (at 142tok/s) 27B took 111.6s (at 50tok/s) Both models are really good. The fact that the 35B produced such a strong result in under 40 seconds is seriously impressive. But when you look at 27B's output... it's actually so much better Added clouds, beautiful foam particles and splashing effects, a perfectly straight horizon, more realistic blinking stars, and a nice sun reflection on the water 🏆 Qwen3.6 27B slower but the result slaps Absolutely amazing that both models run smoothly on 16GB VRAM. Used Unsloth AI GGUFs here, Q3_K_S quant to make it fit in the hardware 🔥

left curve dev

61,154 Aufrufe • vor 3 Monaten

i couldn't find a cute terminal so i made one! i love ghostty but i like working with different pastel color tabs - so i made cute ghostty - every time you open a new tab, it opens with a different cute pastel color and a sweet message! i gave it a cutie icon with a little pink bow, pastel colors, and now it actually fits my vibe~ it's the exact same ghostty underneath - same speed same everything - just cuter 👻🎀 cuteghostty . com
0:10

i couldn't find a cute terminal so i made one! i love ghostty but i like working with different pastel color tabs - so i made cute ghostty - every time you open a new tab, it opens with a different cute pastel color and a sweet message! i gave it a cutie icon with a little pink bow, pastel colors, and now it actually fits my vibe~ it's the exact same ghostty underneath - same speed same everything - just cuter 👻🎀 cuteghostty . com

amy

206,879 Aufrufe • vor 5 Monaten

Everyone says “Gemini 3 can make games.” So we tested it — with a Metal Slug-style prompt on both Gemini and Gambo. Same input. Completely different output. Here’s the result. #gemini3 #gemini_for_game
0:24

Everyone says “Gemini 3 can make games.” So we tested it — with a Metal Slug-style prompt on both Gemini and Gambo. Same input. Completely different output. Here’s the result. #gemini3 #gemini_for_game

gambo.ai

96,044 Aufrufe • vor 8 Monaten

My dual RTX PRO 6000 setup is currently training a Draft model for Qwen 3.6 27B! 🔥 I'm taking the paper DeepSeek dropped on 6/26 and going for a super ambitious application to the 27B scale. Thanks to my homelab, I was able to dive straight in — I read the paper and immediately started experimenting. The amount I've learned has been insane: - How memory bandwidth bottlenecks speed and clever ways to hack around it - Methods to train the draft model and boost its accuracy - Mechanisms to reference tokens all the way back to the previous one to skyrocket draft acceptance rates - The impact of Attention vs. GateDeltaNet on speculative decoding performance and how to handle those differences - The unique approaches and trade-offs of MTP, Dflash, JetSpec, and DSpark I could go on forever, but just from speculative decoding alone I've learned so much. The 27B architecture feels way more DSpark-native than JetSpec, so once draft training finishes, I'm going all-in with DSpark! My goal is to beat existing speculative decoding speeds outright — no task-specific shortcuts or cheating, pure general improvement. If you're into this kind of research, I'd love to hear your thoughts, impressions, and any suggestions — please reply! 🚀
0:17

My dual RTX PRO 6000 setup is currently training a Draft model for Qwen 3.6 27B! 🔥 I'm taking the paper DeepSeek dropped on 6/26 and going for a super ambitious application to the 27B scale. Thanks to my homelab, I was able to dive straight in — I read the paper and immediately started experimenting. The amount I've learned has been insane: - How memory bandwidth bottlenecks speed and clever ways to hack around it - Methods to train the draft model and boost its accuracy - Mechanisms to reference tokens all the way back to the previous one to skyrocket draft acceptance rates - The impact of Attention vs. GateDeltaNet on speculative decoding performance and how to handle those differences - The unique approaches and trade-offs of MTP, Dflash, JetSpec, and DSpark I could go on forever, but just from speculative decoding alone I've learned so much. The 27B architecture feels way more DSpark-native than JetSpec, so once draft training finishes, I'm going all-in with DSpark! My goal is to beat existing speculative decoding speeds outright — no task-specific shortcuts or cheating, pure general improvement. If you're into this kind of research, I'd love to hear your thoughts, impressions, and any suggestions — please reply! 🚀

Hikari∣LocalLLM⚡

56,870 Aufrufe • vor 1 Monat

I finally got GLM-5.2 to work an entire 3-year patient chart that only Bonsai 27B was allowed to read 🔥 292 encounters live inside Bonsai on my Mac Studio. llama.cpp, Metal, ternary, 7.2GB, Apache-2.0. The chart never leaves the machine. GLM-5.2 can only ask questions. It asked three. Bonsai answered each in ~2s with 19,398 tokens still cached. Then it caught the thing buried 17 months back: metformin + iodinated contrast at eGFR 39. Nephrology warned about it in 2025. The ED booked the CT anyway. A 27B-class model used to need a datacentre. PrismML say the 1-bit build is 3.9GB and fits an iPhone 17 Pro Max. The orchestrator never touched the data. That's the whole point. What should it read next?
0:13

I finally got GLM-5.2 to work an entire 3-year patient chart that only Bonsai 27B was allowed to read 🔥 292 encounters live inside Bonsai on my Mac Studio. llama.cpp, Metal, ternary, 7.2GB, Apache-2.0. The chart never leaves the machine. GLM-5.2 can only ask questions. It asked three. Bonsai answered each in ~2s with 19,398 tokens still cached. Then it caught the thing buried 17 months back: metformin + iodinated contrast at eGFR 39. Nephrology warned about it in 2025. The ED booked the CT anyway. A 27B-class model used to need a datacentre. PrismML say the 1-bit build is 3.9GB and fits an iPhone 17 Pro Max. The orchestrator never touched the data. That's the whole point. What should it read next?

Maziyar PANAHI

53,049 Aufrufe • vor 1 Monat

A Bangladeshi man is urinating into the seawater from one ship. At the same time, on another ship, some Bangladeshis are taking that same water in buckets, washing utensils with it, and even using it to cook food.
0:15

A Bangladeshi man is urinating into the seawater from one ship. At the same time, on another ship, some Bangladeshis are taking that same water in buckets, washing utensils with it, and even using it to cook food.

Saffron Chargers

16,380 Aufrufe • vor 3 Monaten

Gemma 4 12B QAT (dense) achieves 1000+ tokens/sec prefill on 8GB VRAM with 120k context Gemma 4 12B QAT (dense), TurboQuant (Without MTP), RTX 4060 8GB VRAM: Prefill: 1000+ tok/s (42% increase) Decode: 25+ tok/s (25% increase) Context: 120k (150% increase) prefill was 700 tok/sec and decode 20 tok/sec with only 48k context without turbo quant (older test with mtp link in the comments) llama.cpp TurboQuant flags: -m gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -c 120000 --cache-type-k q8_0 --cache-type-v turbo3 -ngl 99 --port 8080 tested with a 27k prompt, 120k context loaded. -ngl 99 here isn't a typo, full 12B dense, every layer on GPU, on an 8GB card. that's the part worth sitting with. The model has vision, audio input, thinking/reasoning and fits your 8GB card. TurboQuant's KV cache savings are what free up the room to do that at 120k context. side by side with yesterday: 26B A4B MoE got 320+ tok/s prefill. this dense 12B is clearing 1000+ rig: RTX 4060 8GB · i7H · 16GB RAM same two flags as yesterday, different model size: --cache-type-k q8_0 --cache-type-v turbo3 thanks to TheTom/llama-cpp-turboquant, TurboQuant fork of llama.cpp by Tom Turney (Tom Turney) to make this work. unsloth's model quant huggingface and the llama.cpp fork github link in the comments Do you prefer a dense or a MoE for your 8GB card?
0:12

Gemma 4 12B QAT (dense) achieves 1000+ tokens/sec prefill on 8GB VRAM with 120k context Gemma 4 12B QAT (dense), TurboQuant (Without MTP), RTX 4060 8GB VRAM: Prefill: 1000+ tok/s (42% increase) Decode: 25+ tok/s (25% increase) Context: 120k (150% increase) prefill was 700 tok/sec and decode 20 tok/sec with only 48k context without turbo quant (older test with mtp link in the comments) llama.cpp TurboQuant flags: -m gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -c 120000 --cache-type-k q8_0 --cache-type-v turbo3 -ngl 99 --port 8080 tested with a 27k prompt, 120k context loaded. -ngl 99 here isn't a typo, full 12B dense, every layer on GPU, on an 8GB card. that's the part worth sitting with. The model has vision, audio input, thinking/reasoning and fits your 8GB card. TurboQuant's KV cache savings are what free up the room to do that at 120k context. side by side with yesterday: 26B A4B MoE got 320+ tok/s prefill. this dense 12B is clearing 1000+ rig: RTX 4060 8GB · i7H · 16GB RAM same two flags as yesterday, different model size: --cache-type-k q8_0 --cache-type-v turbo3 thanks to TheTom/llama-cpp-turboquant, TurboQuant fork of llama.cpp by Tom Turney (Tom Turney) to make this work. unsloth's model quant huggingface and the llama.cpp fork github link in the comments Do you prefer a dense or a MoE for your 8GB card?

Alok

34,500 Aufrufe • vor 1 Monat

HE FIXED A BROKEN $5 MP3 PLAYER WITH ONE PROMPT. SAME MODEL WITH OPENAI CODEX BUILT A CUSTOM OS his process: → bought cheap MP3 player on AliExpress → threw it away due to brutal Bluetooth stuttering → hit by a sudden shower thought → cracked open the case to check the chip → one year: garbage in a drawer the AI version: → showed the microchip to OpenAI Codex → Codex dumped firmware and analyzed binaries → built a custom OS fixing all Bluetooth bugs → remapped clunky navigation from left-right to up-down → result: instant, flawless performance the hardware is the same. the output is 10× bigger.
0:17

HE FIXED A BROKEN $5 MP3 PLAYER WITH ONE PROMPT. SAME MODEL WITH OPENAI CODEX BUILT A CUSTOM OS his process: → bought cheap MP3 player on AliExpress → threw it away due to brutal Bluetooth stuttering → hit by a sudden shower thought → cracked open the case to check the chip → one year: garbage in a drawer the AI version: → showed the microchip to OpenAI Codex → Codex dumped firmware and analyzed binaries → built a custom OS fixing all Bluetooth bugs → remapped clunky navigation from left-right to up-down → result: instant, flawless performance the hardware is the same. the output is 10× bigger.

leopardracer

15,617 Aufrufe • vor 2 Monaten

We tested a lot of design skills in Codex and other AI agents. They fix the old AI slop. But they often create a new one: the “taste skill look.” - Same premium sections. - Same editorial typography. - Same generated-design smell. Open Design does not rely on one taste prompt. It combines skills, systems, critique loops, and local harnesses to make AI pages feel less generated and more designed. This page was generated directly in Open Design.
0:15

We tested a lot of design skills in Codex and other AI agents. They fix the old AI slop. But they often create a new one: the “taste skill look.” - Same premium sections. - Same editorial typography. - Same generated-design smell. Open Design does not rely on one taste prompt. It combines skills, systems, critique loops, and local harnesses to make AI pages feel less generated and more designed. This page was generated directly in Open Design.

Open Design

44,061 Aufrufe • vor 2 Monaten