Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Here is Ternary Bonsai 27B running an end-to-end agentic workflow locally with Hermes on an NVIDIA GeForce RTX 5090 GPU. The model reasons, calls tools, reads outputs, modifies files, and surfaces insights - all on consumer hardware, while all private files, intermediate states, and iterations remain local.

PrismML

16,462 subscribers

154,473 views • 1 month ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Try Ternary Bonsai 27B directly on Hugging Face , powered by Together AI

Try Ternary Bonsai 27B directly on Hugging Face , powered by Together AI

PrismML

12,439 views • 1 month ago

It's been an amazing week for open models running on NVIDIA Local AI systems! 👀 Here's a few you’ll want to know about, all ready to run locally on NVIDIA RTX, DGX Spark, DGX Station, and Jetson.

It's been an amazing week for open models running on NVIDIA Local AI systems! 👀 Here's a few you’ll want to know about, all ready to run locally on NVIDIA RTX, DGX Spark, DGX Station, and Jetson.

NVIDIA RTX Spark

59,911 views • 12 days ago

Nvidia CEO Jensen Huang on Saudi Arabia’s MBS: His Royal Highness is an avid gamer, and his home is populated with Nvidia GeForce. He has an esports room with tens of PCs — the highest-end gaming PCs — and underneath the desk is all GeForce. Source: TIME

Nvidia CEO Jensen Huang on Saudi Arabia’s MBS: His Royal Highness is an avid gamer, and his home is populated with Nvidia GeForce. He has an esports room with tens of PCs — the highest-end gaming PCs — and underneath the desk is all GeForce. Source: TIME

Clash Report

737,671 views • 7 months ago

1-bit Bonsai 8B running locally on an M4 Pro (MLX) alongside a standard 16-bit 8B model. Same class of model, very different deployment profile: far lower memory use and substantially higher throughput.

PrismML

93,021 views • 4 months ago

The phone threshold is even harder than the storage number suggests. A phone exposes only part of its memory to an application, and the model must share that budget with its KV cache, activations, runtime, and the rest of the product. At 3.9 GB, 1-bit Bonsai 27B clears that threshold with room to work, making 27B-class local AI possible on a phone for the first time. (Demo Mode: cached & prefilled image context)

The phone threshold is even harder than the storage number suggests. A phone exposes only part of its memory to an application, and the model must share that budget with its KV cache, activations, runtime, and the rest of the product. At 3.9 GB, 1-bit Bonsai 27B clears that threshold with room to work, making 27B-class local AI possible on a phone for the first time. (Demo Mode: cached & prefilled image context)

PrismML

33,358 views • 1 month ago

Unsloth AI and NVIDIA are Revolutionizing Local LLM Fine-Tuning: From RTX Desktops to DGX Spark Fine-tune popular AI models faster with Unsloth on NVIDIA RTX AI PCs such as GeForce RTX desktops and laptops to RTX PRO workstations and the new DGX Spark to build personalized assistants for coding, creative work, and complex agentic workflows. The landscape of modern AI is shifting. We are moving away from a total reliance on massive, generalized cloud models and entering the era of local, agentic AI. Whether it is tuning a chatbot to handle hyper-specific product support or building a personal assistant that manages intricate schedules, the potential for generative AI on local hardware is boundless. However, developers face a persistent bottleneck: How do you get a Small Language Model (SLM) to punch above its weight class and respond with high accuracy for specialized tasks? The answer is Fine-Tuning, and the tool of choice is Unsloth. Unsloth provides an easy and high-speed method to customize models. Optimized for efficient, low-memory training on NVIDIA GPUs, Unsloth scales effortlessly from GeForce RTX desktops and laptop all the way to the DGX Spark, the world’s smallest AI supercomputer...... Full analysis: NVIDIA NVIDIA AI NVIDIA AIDev NVIDIAnewsroom Unsloth AI Unsloth

Unsloth AI and NVIDIA are Revolutionizing Local LLM Fine-Tuning: From RTX Desktops to DGX Spark Fine-tune popular AI models faster with Unsloth on NVIDIA RTX AI PCs such as GeForce RTX desktops and laptops to RTX PRO workstations and the new DGX Spark to build personalized assistants for coding, creative work, and complex agentic workflows. The landscape of modern AI is shifting. We are moving away from a total reliance on massive, generalized cloud models and entering the era of local, agentic AI. Whether it is tuning a chatbot to handle hyper-specific product support or building a personal assistant that manages intricate schedules, the potential for generative AI on local hardware is boundless. However, developers face a persistent bottleneck: How do you get a Small Language Model (SLM) to punch above its weight class and respond with high accuracy for specialized tasks? The answer is Fine-Tuning, and the tool of choice is Unsloth. Unsloth provides an easy and high-speed method to customize models. Optimized for efficient, low-memory training on NVIDIA GPUs, Unsloth scales effortlessly from GeForce RTX desktops and laptop all the way to the DGX Spark, the world’s smallest AI supercomputer...... Full analysis: NVIDIA NVIDIA AI NVIDIA AIDev NVIDIAnewsroom Unsloth AI Unsloth

Marktechpost AI Dev News ⚡

31,551 views • 8 months ago

You can now run Qwen 3.6 27B on <10GB of memory and retain 90% of the intelligence. Today, PrismML dropped Bonsai 27B - which takes their unique compression approach they have already successfully applied to Qwen3.6 8B and Flux2-Klein and applied it to a properly smart model. Here is it using an entire computer for deep research. Long horizon agentic calls already are strong with Qwen3.6 27B and they still are here - just on 10x less memory.

You can now run Qwen 3.6 27B on <10GB of memory and retain 90% of the intelligence. Today, PrismML dropped Bonsai 27B - which takes their unique compression approach they have already successfully applied to Qwen3.6 8B and Flux2-Klein and applied it to a properly smart model. Here is it using an entire computer for deep research. Long horizon agentic calls already are strong with Qwen3.6 27B and they still are here - just on 10x less memory.

Tim Carambat

62,879 views • 1 month ago

#Battlefield6 NVIDIA DLSS 4 Reveal Trailer 📽️ The game includes support for DLSS 4 with Multi Frame Generation, DLSS Frame Generation, DLSS Super Resolution, DLAA, and NVIDIA Reflex. Desktop GPUs Performance At 4K, Ultra settings, DLSS 4 with Multi Frame Generation and DLSS Super Resolution multiply Battlefield 6's GeForce RTX 50 Series frame rates by an average of 3.8X. ▪️ GeForce RTX 5090 performance rockets to over 470 FPS. ▪️ GeForce RTX 5080 exceeds 330 FPS. ▪️ GeForce RTX 5070 Ti is in touching distance of 300 FPS. ▪️ GeForce RTX 5070 surpasses 230 FPS. At 2560x1440, Ultra settings, DLSS 4 with Multi Frame Generation and DLSS Super Resolution increase Battlefield 6 frame rates by an average of 3X. ▪️ GeForce RTX 5090 runs at almost 600 FPS. ▪️ GeForce RTX 5060 Ti over 240 FPS. At 1920x1080, Ultra settings, the combination of DLSS 4 with Multi Frame Generation and DLSS Super Resolution boost Battlefield 6’s GeForce RTX 50 Series frame rates by an average of 2.9X. ▪️ Allowing every GPU in NVIDIA's line-up to play at over 230 FPS, maxing out at over 740 FPS on the GeForce RTX 5090. Laptop GPUs Performance GeForce RTX 50 Series Laptop GPUs benefit similarly from DLSS 4 with Multi Frame Generation and DLSS Super Resolution, with Battlefield 6 frame rates multiplied by 3X on average at 2560x1600, with Ultra settings, enabling performance of up to 360 FPS. At 1920x1080, Ultra settings, GeForce RTX 50 Series Laptop GPU performance increases by 2.8X on average, enabling Battlefield 6 frame rates to surpass 480 FPS, and the range to run at over 200 frames per second.

#Battlefield6 NVIDIA DLSS 4 Reveal Trailer 📽️ The game includes support for DLSS 4 with Multi Frame Generation, DLSS Frame Generation, DLSS Super Resolution, DLAA, and NVIDIA Reflex. Desktop GPUs Performance At 4K, Ultra settings, DLSS 4 with Multi Frame Generation and DLSS Super Resolution multiply Battlefield 6's GeForce RTX 50 Series frame rates by an average of 3.8X. ▪️ GeForce RTX 5090 performance rockets to over 470 FPS. ▪️ GeForce RTX 5080 exceeds 330 FPS. ▪️ GeForce RTX 5070 Ti is in touching distance of 300 FPS. ▪️ GeForce RTX 5070 surpasses 230 FPS. At 2560x1440, Ultra settings, DLSS 4 with Multi Frame Generation and DLSS Super Resolution increase Battlefield 6 frame rates by an average of 3X. ▪️ GeForce RTX 5090 runs at almost 600 FPS. ▪️ GeForce RTX 5060 Ti over 240 FPS. At 1920x1080, Ultra settings, the combination of DLSS 4 with Multi Frame Generation and DLSS Super Resolution boost Battlefield 6’s GeForce RTX 50 Series frame rates by an average of 2.9X. ▪️ Allowing every GPU in NVIDIA's line-up to play at over 230 FPS, maxing out at over 740 FPS on the GeForce RTX 5090. Laptop GPUs Performance GeForce RTX 50 Series Laptop GPUs benefit similarly from DLSS 4 with Multi Frame Generation and DLSS Super Resolution, with Battlefield 6 frame rates multiplied by 3X on average at 2560x1600, with Ultra settings, enabling performance of up to 360 FPS. At 1920x1080, Ultra settings, GeForce RTX 50 Series Laptop GPU performance increases by 2.8X on average, enabling Battlefield 6 frame rates to surpass 480 FPS, and the range to run at over 200 frames per second.

Battlefield Bulletin

30,559 views • 10 months ago

🚨 BREAKING: Nvidia just released Chat with RTX, an AI chatbot that runs locally on your PC. It can summarize or search documents across your PC's files and even YouTube videos and playlists. The chatbot runs locally, meaning results are fast, you can use it without the internet, and the user's data stays private. New day, new chatbot. Let's go.

🚨 BREAKING: Nvidia just released Chat with RTX, an AI chatbot that runs locally on your PC. It can summarize or search documents across your PC's files and even YouTube videos and playlists. The chatbot runs locally, meaning results are fast, you can use it without the internet, and the user's data stays private. New day, new chatbot. Let's go.

Rowan Cheung

2,725,637 views • 2 years ago

(1/5) FP4 hardware is here, but 4-bit attention still kills model quality, blocking true end-to-end FP4 serving. To fix that, we propose Attn-QAT, the first systematic study of quantization-aware training for attention. The result: FP4 attention quality is comparable to BF16 attention with 1.1x–1.5x higher throughput than SageAttention3 on an RTX 5090 and 1.39x speedup over FlashAttention-4 on a B200. Blog: Code: Checkpoints:

(1/5) FP4 hardware is here, but 4-bit attention still kills model quality, blocking true end-to-end FP4 serving. To fix that, we propose Attn-QAT, the first systematic study of quantization-aware training for attention. The result: FP4 attention quality is comparable to BF16 attention with 1.1x–1.5x higher throughput than SageAttention3 on an RTX 5090 and 1.39x speedup over FlashAttention-4 on a B200. Blog: Code: Checkpoints:

Hao AI Lab

37,869 views • 4 months ago

Create a personalized chatbot with the Chat with RTX tech demo. #AIonRTX Quickly get tailored info from your own files and content by connecting your data to an LLM on RTX-Powered PCs for local, fast, generative AI. Download demo here:

Create a personalized chatbot with the Chat with RTX tech demo. #AIonRTX Quickly get tailored info from your own files and content by connecting your data to an LLM on RTX-Powered PCs for local, fast, generative AI. Download demo here:

NVIDIA Studio

95,340 views • 2 years ago

We’re also launching Bonsai Studio, an iOS app for trying Bonsai Image 4B directly on your iPhone. Run image generation on-device, with no subscription and no API requests.

We’re also launching Bonsai Studio, an iOS app for trying Bonsai Image 4B directly on your iPhone. Run image generation on-device, with no subscription and no API requests.

PrismML

14,499 views • 3 months ago

watch anon! 27 billion parameter model dancing at 42 tokens a second on a used rtx 3060ti, that costs 200 dollars. consuming 159 watts. temp 62 degrees. pushing almost 100% on the gpu. 6.8 of 8 gigs used with a 128k context window loaded. and every one of those spikes is the model reasoning and firing tool calls. the model is bonsai 27b at 1 bit. PrismML took qwen 3.6 27b, and crushed it to 3.9 gigabytes, about a single sign bit per weight. it drops onto an 8gb rtx 3060 ti, an ampere card from 2020, and it doesn't just load, it serves, it holds the full 128k window, it runs a hermes agent for half an hour straight, it hits its own failures and recovers, it never once drops the loop. the absurd part is the power meter. that same 27b, in its normal quant 4, wants a 24gb card and nearly 17 gigs of weights. this is it at 3.9, doing agentic work at 160 watts, on the tier people call budget. the numbers are sitting right there on nvtop, not in a slide, not a claim, the meter reading while it works. so i keep landing on one question. the model is open, apache 2.0. the card is old and cheap. the whole thing runs offline. why is almost nobody actually running this. is it the hardware, the setup, or did nobody tell you a 27b fits on the card already in your closet.

watch anon! 27 billion parameter model dancing at 42 tokens a second on a used rtx 3060ti, that costs 200 dollars. consuming 159 watts. temp 62 degrees. pushing almost 100% on the gpu. 6.8 of 8 gigs used with a 128k context window loaded. and every one of those spikes is the model reasoning and firing tool calls. the model is bonsai 27b at 1 bit. PrismML took qwen 3.6 27b, and crushed it to 3.9 gigabytes, about a single sign bit per weight. it drops onto an 8gb rtx 3060 ti, an ampere card from 2020, and it doesn't just load, it serves, it holds the full 128k window, it runs a hermes agent for half an hour straight, it hits its own failures and recovers, it never once drops the loop. the absurd part is the power meter. that same 27b, in its normal quant 4, wants a 24gb card and nearly 17 gigs of weights. this is it at 3.9, doing agentic work at 160 watts, on the tier people call budget. the numbers are sitting right there on nvtop, not in a slide, not a claim, the meter reading while it works. so i keep landing on one question. the model is open, apache 2.0. the card is old and cheap. the whole thing runs offline. why is almost nobody actually running this. is it the hardware, the setup, or did nobody tell you a 27b fits on the card already in your closet.

Sudo su

23,391 views • 1 month ago

this is what 12 gigs of VRAM built in 2026. a 9 billion parameter model running on a 5 year old RTX 3060 wrote a full space shooter from a single prompt. blank screen on first try. i came back with a bug list and the same model on the same card fixed every issue across 11 files without touching a single line myself. enemies still looked wrong so i pushed another iteration and now the game has pixel art octopi, particle effects, screen shake, projectile physics and a combo system. all running locally on a card that was designed to play fortnite. three iterations. zero cloud. zero API calls. every token generated on hardware sitting under my desk. the model reads its own code, finds what's broken, patches it, validates syntax and restarts the server. i just describe what's wrong and it handles the rest. people are paying monthly subscriptions to type into a browser tab and wait for a server farm to respond. meanwhile a GPU you can find used on ebay is running a full autonomous hermes agent framework with 31 tools, 128K context window and thinking mode generating at 29 tokens per second nonstop. the game still needs work. level upgrades don't trigger and boss fights need tuning. but the fact that i'm iterating on gameplay balance instead of debugging whether the code runs at all tells you where this is headed. every iteration the game gets better on the same hardware. same 12 gigs. same 9 billion parameters. same RTX 3060 from 5 years ago your GPU is not a gaming card anymore. it's a local AI lab that never sends your data anywhere.

this is what 12 gigs of VRAM built in 2026. a 9 billion parameter model running on a 5 year old RTX 3060 wrote a full space shooter from a single prompt. blank screen on first try. i came back with a bug list and the same model on the same card fixed every issue across 11 files without touching a single line myself. enemies still looked wrong so i pushed another iteration and now the game has pixel art octopi, particle effects, screen shake, projectile physics and a combo system. all running locally on a card that was designed to play fortnite. three iterations. zero cloud. zero API calls. every token generated on hardware sitting under my desk. the model reads its own code, finds what's broken, patches it, validates syntax and restarts the server. i just describe what's wrong and it handles the rest. people are paying monthly subscriptions to type into a browser tab and wait for a server farm to respond. meanwhile a GPU you can find used on ebay is running a full autonomous hermes agent framework with 31 tools, 128K context window and thinking mode generating at 29 tokens per second nonstop. the game still needs work. level upgrades don't trigger and boss fights need tuning. but the fact that i'm iterating on gameplay balance instead of debugging whether the code runs at all tells you where this is headed. every iteration the game gets better on the same hardware. same 12 gigs. same 9 billion parameters. same RTX 3060 from 5 years ago your GPU is not a gaming card anymore. it's a local AI lab that never sends your data anywhere.

Sudo su

170,848 views • 5 months ago

This is a world model running locally on an RTX 5090. It was built from scratch by a team in India, lucidml. Like modern video games, you can choose between performance and graphics by choosing how many times the neural network is run to generate each frame. Absolutely wild.

This is a world model running locally on an RTX 5090. It was built from scratch by a team in India, lucidml. Like modern video games, you can choose between performance and graphics by choosing how many times the neural network is run to generate each frame. Absolutely wild.

Caleb

44,816 views • 6 months ago

six months ago this wasn't happening on 8gb vram. running unsloth's Q4_K_XL quant of gemma 4 26b-a4b-it-qat, a sparse MoE model with only 4b active params on a single rtx 4060 laptop gpu, 8gb vram, 20+ tok/s decode. no cloud, no api, no offload hacks. just a gaming laptop on battery. what makes it fit: google's QAT (quantization aware training), plus MTP (multi token prediction) support in the latest llama.cpp builds. that combo is the single biggest unlock for local inference on low vram. rtx 3060, rtx 3070, gtx 1070, gtx 1080, rtx 4050, rtx 4060, rtx 5050, rtx 5060 — any 6-8gb consumer gpu, old or new — this model runs on it. world cup season, so i told it to build a soccer themed flappy bird clone. one shot, zero iteration, fully playable. six months ago an 8gb model could barely clone vanilla flappy bird. now it's shipping a themed game from a sparse MoE model running locally on a laptop battery. inference benchmarks: - decode throughput: 30 tok/s - context: 64k. this is the real unlock. 64k ctx is what makes a hermes agent loop viable locally on this model, not just single-turn chat. llama.cpp flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 64000 -cmoe --port 8080 game's deployed on my own site, built and shipped end to end with open source llm, zero closed source api dependency in the pipeline. link in the description. gguf weights on huggingface, link in the comments. pull it down, run it on whatever 8gb card is sitting in your rig. try the game and tell me your score and what you want in v2. local llms on consumer gpus stopped being a meme.

six months ago this wasn't happening on 8gb vram. running unsloth's Q4_K_XL quant of gemma 4 26b-a4b-it-qat, a sparse MoE model with only 4b active params on a single rtx 4060 laptop gpu, 8gb vram, 20+ tok/s decode. no cloud, no api, no offload hacks. just a gaming laptop on battery. what makes it fit: google's QAT (quantization aware training), plus MTP (multi token prediction) support in the latest llama.cpp builds. that combo is the single biggest unlock for local inference on low vram. rtx 3060, rtx 3070, gtx 1070, gtx 1080, rtx 4050, rtx 4060, rtx 5050, rtx 5060 — any 6-8gb consumer gpu, old or new — this model runs on it. world cup season, so i told it to build a soccer themed flappy bird clone. one shot, zero iteration, fully playable. six months ago an 8gb model could barely clone vanilla flappy bird. now it's shipping a themed game from a sparse MoE model running locally on a laptop battery. inference benchmarks: - decode throughput: 30 tok/s - context: 64k. this is the real unlock. 64k ctx is what makes a hermes agent loop viable locally on this model, not just single-turn chat. llama.cpp flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 64000 -cmoe --port 8080 game's deployed on my own site, built and shipped end to end with open source llm, zero closed source api dependency in the pipeline. link in the description. gguf weights on huggingface, link in the comments. pull it down, run it on whatever 8gb card is sitting in your rig. try the game and tell me your score and what you want in v2. local llms on consumer gpus stopped being a meme.

Alok

61,660 views • 2 months ago

Eigent × Kimi.ai K2.5 We ran a real-world sales performance evaluation on Eigent using Kimi K2.5. Given a production sales dataset, Eigent coordinated multiple agents end to end: - The document agent extracts information from the source files - The terminal agent analyzes the data and generates an HTML report All results are stored locally! Try Eigent with Kimi K2.5 today.

Eigent × Kimi.ai K2.5 We ran a real-world sales performance evaluation on Eigent using Kimi K2.5. Given a production sales dataset, Eigent coordinated multiple agents end to end: - The document agent extracts information from the source files - The terminal agent analyzes the data and generates an HTML report All results are stored locally! Try Eigent with Kimi K2.5 today.

Eigent AI

25,263 views • 7 months ago

CoreWeave CEO Michael Intrator on CNBC: * Says GPU depreciation debate is "put to bed" * Over $25B in new contracts in 5 weeks since Q2 close * A 2020-vintage GPU generation was contracted all the way out to 2029 at "full freight" * 3 reasons old chips hold value are, Nvidia GPU being the best hardware, CUDA, and CoreWeave's own software layer > Says non-Nvidia chips may not behave this way * Demand is broad based, from enterprise, foundation labs, AI natives, industrials, financials * Nvidia ~$500B financing platform is an "affirmation" of CoreWeave's business model and the demand signals they've been seeing for years

CoreWeave CEO Michael Intrator on CNBC: * Says GPU depreciation debate is "put to bed" * Over $25B in new contracts in 5 weeks since Q2 close * A 2020-vintage GPU generation was contracted all the way out to 2029 at "full freight" * 3 reasons old chips hold value are, Nvidia GPU being the best hardware, CUDA, and CoreWeave's own software layer > Says non-Nvidia chips may not behave this way * Demand is broad based, from enterprise, foundation labs, AI natives, industrials, financials * Nvidia ~$500B financing platform is an "affirmation" of CoreWeave's business model and the demand signals they've been seeing for years

Fireside Alpha

122,778 views • 14 days ago