Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Laguna XS.2 from Poolside is a 33B MoE built for agentic coding. Red Hat AI trained a DFlash speculator for it: 0.6B drafter, 8 tokens per pass, no quality loss. FP8, NVFP4, and INT4 checkpoints via LLM Compressor. Models in comments. Speedup with vLLM:

Red Hat AI

11,748 subscribers

21,590 views • 2 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Red Hat AI just shipped DFlash speculator checkpoints for two of NVIDIA AI's most powerful open models: → Nemotron Ultra 550B → Nemotron Super 120B On math and reasoning: ~5 out of 7 draft tokens accepted on average. On code (HumanEval): ~3.4 out of 7. Both checkpoints trained with the open source Speculators library from vLLM. Apache 2.0. Validated on NVIDIA B200. One flag to enable in vLLM: --spec-model RedHatAI/NVIDIA-Nemotron-3-Ultra-550B-A55B-speculator.dflash --spec-tokens 7 --spec-method dflash 🔗 Ultra 550B: 🔗 Super 120B:

Red Hat AI just shipped DFlash speculator checkpoints for two of NVIDIA AI's most powerful open models: → Nemotron Ultra 550B → Nemotron Super 120B On math and reasoning: ~5 out of 7 draft tokens accepted on average. On code (HumanEval): ~3.4 out of 7. Both checkpoints trained with the open source Speculators library from vLLM. Apache 2.0. Validated on NVIDIA B200. One flag to enable in vLLM: --spec-model RedHatAI/NVIDIA-Nemotron-3-Ultra-550B-A55B-speculator.dflash --spec-tokens 7 --spec-method dflash 🔗 Ultra 550B: 🔗 Super 120B:

Red Hat AI

14,923 views • 26 days ago

Laguna S 2.1 is built to run locally. We heard the ask for a serious open model in the 80–120B range. Laguna S 2.1 is our answer, and we want it to redefine what this size class can do. Weights ship in BF16, FP8, INT4 and NVFP4, with official GGUF and MLX conversions, plus official DFlash draft models for faster inference. Day-one support across llama.cpp, ollama, Valerie Lee Make Up, SGLang, NVIDIA TRT-LLM, Hugging Face transformers and ZML. Watch it run on an NVIDIA AI DGX Spark.

Laguna S 2.1 is built to run locally. We heard the ask for a serious open model in the 80–120B range. Laguna S 2.1 is our answer, and we want it to redefine what this size class can do. Weights ship in BF16, FP8, INT4 and NVFP4, with official GGUF and MLX conversions, plus official DFlash draft models for faster inference. Day-one support across llama.cpp, ollama, Valerie Lee Make Up, SGLang, NVIDIA TRT-LLM, Hugging Face transformers and ZML. Watch it run on an NVIDIA AI DGX Spark.

Poolside

31,242 views • 20 days ago

🤝 Arcee.ai, an American AI lab, built its frontier open model family on NVIDIA Blackwell Ultra. Its flagship MoE model, Trinity-Large-Thinking, was RL post-trained using NVIDIA NeMo open libraries. Arcee optimized agentic model inference with NVIDIA Dynamo and vLLM, along with NVIDIA accelerated networking, to deliver low token cost, achieving: ✅ 3T+ tokens served on OpenRouter in its first two months ✅ $0.90 per 1M output tokens ✅ 2nd on PinchBench for open agentic models Explore Arcee's open models, powered by NVIDIA ➡️

🤝 Arcee.ai, an American AI lab, built its frontier open model family on NVIDIA Blackwell Ultra. Its flagship MoE model, Trinity-Large-Thinking, was RL post-trained using NVIDIA NeMo open libraries. Arcee optimized agentic model inference with NVIDIA Dynamo and vLLM, along with NVIDIA accelerated networking, to deliver low token cost, achieving: ✅ 3T+ tokens served on OpenRouter in its first two months ✅ $0.90 per 1M output tokens ✅ 2nd on PinchBench for open agentic models Explore Arcee's open models, powered by NVIDIA ➡️

NVIDIA AI Infrastructure

27,377 views • 27 days ago

Researchers found a way to make LLMs 8.5x faster! (without compromising accuracy) Speculative decoding is quite an effective way to address the single-token bottleneck in traditional LLM inference. A small "draft" model first generates the next several tokens, then the large model verifies all of them at once in a single forward pass. If a token at any position is wrong, you keep everything before it and restart from there. This never does worse than normal decoding. But current drafters in Speculative decoding still guess one token at a time. That makes the drafting step itself a bottleneck, capping real-world speedups at 2-3x. DFlash is a new technique that swaps the autoregressive drafter with a lightweight block diffusion model that guesses all tokens in one parallel shot. Drafting cost stays flat no matter how many tokens you speculate. On top of that, the drafter is conditioned on hidden features pulled from multiple layers of the target model and injected into every draft layer, so it makes significantly better guesses than a drafter working from scratch. In the side-by-side demo below, vanilla decoding runs at 48.5 tokens/sec. DFlash hits 415 tokens/sec on the same model, with zero quality loss. It's already integrated with vLLM, SGLang, and Transformers, with draft models on HuggingFace for several models like Qwen3, Qwen3.5, Llama 3.1, Kimi-K2.5, gpt-oss, and many more. I have shared the GitHub repo in the replies! KV caching is another must-know technique to boost LLM inference. I recently wrote an article about it. Read it below. 👉 Over to you: What use case are you working on that can benefit from this new technique?

Researchers found a way to make LLMs 8.5x faster! (without compromising accuracy) Speculative decoding is quite an effective way to address the single-token bottleneck in traditional LLM inference. A small "draft" model first generates the next several tokens, then the large model verifies all of them at once in a single forward pass. If a token at any position is wrong, you keep everything before it and restart from there. This never does worse than normal decoding. But current drafters in Speculative decoding still guess one token at a time. That makes the drafting step itself a bottleneck, capping real-world speedups at 2-3x. DFlash is a new technique that swaps the autoregressive drafter with a lightweight block diffusion model that guesses all tokens in one parallel shot. Drafting cost stays flat no matter how many tokens you speculate. On top of that, the drafter is conditioned on hidden features pulled from multiple layers of the target model and injected into every draft layer, so it makes significantly better guesses than a drafter working from scratch. In the side-by-side demo below, vanilla decoding runs at 48.5 tokens/sec. DFlash hits 415 tokens/sec on the same model, with zero quality loss. It's already integrated with vLLM, SGLang, and Transformers, with draft models on HuggingFace for several models like Qwen3, Qwen3.5, Llama 3.1, Kimi-K2.5, gpt-oss, and many more. I have shared the GitHub repo in the replies! KV caching is another must-know technique to boost LLM inference. I recently wrote an article about it. Read it below. 👉 Over to you: What use case are you working on that can benefit from this new technique?

Avi Chawla

157,390 views • 3 months ago

Today, we're introducing not one but TWO new models, striking the balance between efficiency and quality to enable you to build production AI agents. — Gemini 3.6 Flash: Addresses efficiency feedback we received from Gemini 3.5 Flash with upgrades in coding, knowledge work, and multimodal tasks faster, more accurately, and with substantially fewer tokens per task — Gemini 3.5 Flash-Lite: Our fastest, most cost-effective 3.5-class model yet built for agentic workflows, hitting ~350 output tokens/sec with improved coding and overall quality Start building with these today via the Gemini API in Google AI Studio or try them out in the Google Gemini

Today, we're introducing not one but TWO new models, striking the balance between efficiency and quality to enable you to build production AI agents. — Gemini 3.6 Flash: Addresses efficiency feedback we received from Gemini 3.5 Flash with upgrades in coding, knowledge work, and multimodal tasks faster, more accurately, and with substantially fewer tokens per task — Gemini 3.5 Flash-Lite: Our fastest, most cost-effective 3.5-class model yet built for agentic workflows, hitting ~350 output tokens/sec with improved coding and overall quality Start building with these today via the Gemini API in Google AI Studio or try them out in the Google Gemini

Google AI

434,795 views • 20 days ago

DFlash makes Qwen 2.2x faster with no quality loss! We ran the same Qwen3.6-27B locally three ways on one RTX 6000: baseline, MTP, DFlash. The tasks only differ in one thing - how predictable the next word is: quicksort, describe a file in JSON, a logic puzzle, a sci-fi story. Outputs: Baseline: 44 tok/s · 1.00x MTP: 65 tok/s · 1.45x · 71% accepted DFlash: 98 tok/s · 2.20x · 30% accepted Baseline writes one token per step. MTP works inside the model itself and guesses 3 tokens ahead. DFlash is a separate small model that writes 15 tokens at once, and the big model only checks them. In JSON the same words repeat all the time, so most guesses were right: 152 tok/s, 3.4x speedup. In the story 9 guesses out of 10 were wrong. DFlash did all that extra work for nothing and became slower than baseline: 42 vs 44 tok/s. MTP guesses only 3 tokens, so a wrong guess costs very little: 46 tok/s and the win in that round. The output is identical in all three modes - DFlash is the pick for tasks with predictable output, like coding, and for chat and creative writing MTP works better. DFlash is now natively integrated into Atomic Chat on llama.cpp - speed up your Qwen models!

DFlash makes Qwen 2.2x faster with no quality loss! We ran the same Qwen3.6-27B locally three ways on one RTX 6000: baseline, MTP, DFlash. The tasks only differ in one thing - how predictable the next word is: quicksort, describe a file in JSON, a logic puzzle, a sci-fi story. Outputs: Baseline: 44 tok/s · 1.00x MTP: 65 tok/s · 1.45x · 71% accepted DFlash: 98 tok/s · 2.20x · 30% accepted Baseline writes one token per step. MTP works inside the model itself and guesses 3 tokens ahead. DFlash is a separate small model that writes 15 tokens at once, and the big model only checks them. In JSON the same words repeat all the time, so most guesses were right: 152 tok/s, 3.4x speedup. In the story 9 guesses out of 10 were wrong. DFlash did all that extra work for nothing and became slower than baseline: 42 vs 44 tok/s. MTP guesses only 3 tokens, so a wrong guess costs very little: 46 tok/s and the win in that round. The output is identical in all three modes - DFlash is the pick for tasks with predictable output, like coding, and for chat and creative writing MTP works better. DFlash is now natively integrated into Atomic Chat on llama.cpp - speed up your Qwen models!

atomic.chat

75,910 views • 28 days ago

Finally got a chance to play around with Andrej Karpathy's LLM Council. I built it as a plugin inside of Claude Code. Hooked it up with OpenRouter for models. The AskUserQuestion tool came in handy to select the council and chairman. This is my first test, but I agree with Karpathy that the concept of LLM ensembles can be used beyond models that offer perspectives on interesting questions. I feel like this could have really cool applications in agentic coding. More on that soon. I built this as a plugin, so next I will be exploring other user cases around agentic coding, like evaluation, tool building, designing, and research. If there is enough interest, I will clean it up and push it out as an open plugin.

Finally got a chance to play around with Andrej Karpathy's LLM Council. I built it as a plugin inside of Claude Code. Hooked it up with OpenRouter for models. The AskUserQuestion tool came in handy to select the council and chairman. This is my first test, but I agree with Karpathy that the concept of LLM ensembles can be used beyond models that offer perspectives on interesting questions. I feel like this could have really cool applications in agentic coding. More on that soon. I built this as a plugin, so next I will be exploring other user cases around agentic coding, like evaluation, tool building, designing, and research. If there is enough interest, I will clean it up and push it out as an open plugin.

elvis

79,766 views • 7 months ago

I used Grok 4.5 in Grok Build to create a game engine for agentic coding, then built an ARPG on it, with all game assets generated via Grok Imagine in Grok Build. AI agents can build worlds, combat, loot, progression, maps, tests, and playable games. ⚒️🎮

I used Grok 4.5 in Grok Build to create a game engine for agentic coding, then built an ARPG on it, with all game assets generated via Grok Imagine in Grok Build. AI agents can build worlds, combat, loot, progression, maps, tests, and playable games. ⚒️🎮

tetsuo

197,959 views • 26 days ago

Scale alone is not enough for AI data. Quality and complexity are equally critical. Excited to support all of these for LLM developers with Snorkel AI Data-as-a-Service, and to share our new leaderboard! — Our decade-plus of research and work in AI data has a simple point: scale alone is not enough. AI success is all about the quality, complexity, and distribution of data—in addition to volume. We’re excited to be powering leading LLM developers with Snorkel AI Expert Data-as-a-Service, our white glove service for custom, expert-level AI datasets—and to now preview some of what we’re building via our new Expert Data Leaderboard (🔗 in 🧵) + upcoming OSS dataset releases! Snorkel Expert Data-as-a-Service is built to meet the rapidly evolving data needs of the agentic AI world—where success is built on the quality, complexity, and distribution of datasets, in addition to size and scale. This kind of high-quality, frontier AI data can only come from a union of technology and human expertise. With Snorkel Expert Data-as-a-Service, we’re powering frontier LLM developers across agentic, expert knowledge, reasoning, coding, multi-modal, and other task types via the combination of these two key components: - (1) The Snorkel Expert Network: A global team of subject matter experts focused wholly on specialized knowledge–spanning thousands of topics in STEM/academic, vertical/professional, and consumer/lifestyle domains. - (2) Snorkel AI Data Development Platform: Our unique programmatic data curation and quality control platform, accelerating and improving expert authoring and review through principled techniques developed over the last decade of R&D. Now: we’re incredibly excited to showcase some of the power of Snorkel Expert Data-as-a-Service via the new Snorkel Leaderboard—putting frontier models to the test in complex, agentic, and reasoning settings inspired by real industry scenarios (not esoteric puzzles)! We’ll be releasing new leaderboards and accompanying expert-verified open source datasets (coming soon!) regularly. To start, we’re sharing three initial ones in preview: - SnorkelFinance: Q&A over financial documents requiring agentic tool-calling and reasoning - SnorkelUnderwrite: Agentic insurance tasks requiring industry-specific reasoning and tool use - SnorkelSequences: Mathematical tasks requiring compositional multi-step reasoning

Scale alone is not enough for AI data. Quality and complexity are equally critical. Excited to support all of these for LLM developers with Snorkel AI Data-as-a-Service, and to share our new leaderboard! — Our decade-plus of research and work in AI data has a simple point: scale alone is not enough. AI success is all about the quality, complexity, and distribution of data—in addition to volume. We’re excited to be powering leading LLM developers with Snorkel AI Expert Data-as-a-Service, our white glove service for custom, expert-level AI datasets—and to now preview some of what we’re building via our new Expert Data Leaderboard (🔗 in 🧵) + upcoming OSS dataset releases! Snorkel Expert Data-as-a-Service is built to meet the rapidly evolving data needs of the agentic AI world—where success is built on the quality, complexity, and distribution of datasets, in addition to size and scale. This kind of high-quality, frontier AI data can only come from a union of technology and human expertise. With Snorkel Expert Data-as-a-Service, we’re powering frontier LLM developers across agentic, expert knowledge, reasoning, coding, multi-modal, and other task types via the combination of these two key components: - (1) The Snorkel Expert Network: A global team of subject matter experts focused wholly on specialized knowledge–spanning thousands of topics in STEM/academic, vertical/professional, and consumer/lifestyle domains. - (2) Snorkel AI Data Development Platform: Our unique programmatic data curation and quality control platform, accelerating and improving expert authoring and review through principled techniques developed over the last decade of R&D. Now: we’re incredibly excited to showcase some of the power of Snorkel Expert Data-as-a-Service via the new Snorkel Leaderboard—putting frontier models to the test in complex, agentic, and reasoning settings inspired by real industry scenarios (not esoteric puzzles)! We’ll be releasing new leaderboards and accompanying expert-verified open source datasets (coming soon!) regularly. To start, we’re sharing three initial ones in preview: - SnorkelFinance: Q&A over financial documents requiring agentic tool-calling and reasoning - SnorkelUnderwrite: Agentic insurance tasks requiring industry-specific reasoning and tool use - SnorkelSequences: Mathematical tasks requiring compositional multi-step reasoning

Alex Ratner

495,851 views • 1 year ago

This is it for AI coding.. I made this recipe app with a built-in TikTok feed with just 2 lines of prompts. And anyone (yes, anyone) can do the same. I will show you how (Bookmark for later):

This is it for AI coding.. I made this recipe app with a built-in TikTok feed with just 2 lines of prompts. And anyone (yes, anyone) can do the same. I will show you how (Bookmark for later):

Rez Karim

31,365 views • 1 year ago

Generate image, 3D, and video from a single sketch in one click in ComfyUI with AI! I’m excited to share a cutting-edge workflow I’ve developed that combines inside ComfyUI: Runway with fal. ai api (yes there is a node for it :) ), Stable AI’s stable fast 3D , powerful Flux, ControlNet, IPAdapter, Gemini LLm in ComfyUI. This innovation allows you to generate high-quality visuals and 3D models, reposition objects, and create videos—completely automated from a simple sketch input. Would you link a tutorial for it ? #AI #ComfyUI #3DModeling #Automation #CreativeTech

Generate image, 3D, and video from a single sketch in one click in ComfyUI with AI! I’m excited to share a cutting-edge workflow I’ve developed that combines inside ComfyUI: Runway with fal. ai api (yes there is a node for it :) ), Stable AI’s stable fast 3D , powerful Flux, ControlNet, IPAdapter, Gemini LLm in ComfyUI. This innovation allows you to generate high-quality visuals and 3D models, reposition objects, and create videos—completely automated from a simple sketch input. Would you link a tutorial for it ? #AI #ComfyUI #3DModeling #Automation #CreativeTech

Lovis Odin

36,789 views • 1 year ago

KREA 2 is live on fal. KREA's first foundation image model, trained from scratch. Built around a real tradeoff in image generation: - most models force a choice between aesthetic quality and creative control - KREA 2 holds style, mood, and direction across a full workflow, not just one shot - stronger primitive for moodboard-driven, brand-consistent, art-directed pipelines

fal

15,613 views • 2 months ago

He is a 1st year AI engineering student at Stanford - and already launched an automated system making $2.1M a year: 00:20 - how Anthropic engineers make millions with LLM and Transformers 38:34 - In-Context Learning makes AI smarter for free 59:47 - 1st year student made $2.1M using LLM from scratch after watching I gave LLM the idea I had been holding in my head for 3 years - it built it in one evening. save & watch - the article below is a step by step guide to build your automated LLM system.

He is a 1st year AI engineering student at Stanford - and already launched an automated system making $2.1M a year: 00:20 - how Anthropic engineers make millions with LLM and Transformers 38:34 - In-Context Learning makes AI smarter for free 59:47 - 1st year student made $2.1M using LLM from scratch after watching I gave LLM the idea I had been holding in my head for 3 years - it built it in one evening. save & watch - the article below is a step by step guide to build your automated LLM system.

Noisy

42,621 views • 9 days ago

DUAL INTEL ARC PRO B70 GPUS AT $1,900 RUN QWEN 3 CODER 30B AT 268 TOKENS PER SECOND VIA VLLM ON 8 CONCURRENT REQUESTS, A NEW INTEL TIER UNDER YOUR MAP THAT BEATS RTX 3090 STACKS ON THROUGHPUT PER DOLLAR 02:22 the operator reads off his benchmark sheet, "vllm was at 268 tokens per second which is just wild to me, getting 250+ tokens per second out of these overall kind of slow Intel cards" dual Intel Arc Pro B70 ships at $950 per card, 32GB GDDR6 each, 64GB combined VRAM for $1,900 total, the same budget gets you two used RTX 3090s at 48GB combined and worse multi-user throughput Qwen 3 coder 30B on dual B70 with vLLM hits 268 t/s at 8 concurrent users, llama.cpp tops out at 84 t/s on the same hardware, the gap is not 2x or 3x, vLLM with Intel's tweaks pulls 3.2x on parallel inference prefill on a 16,000 word context took 26 seconds on llama.cpp and 3 seconds on vLLM, agents spawning subagents or households sharing one box turn that 8x speedup into 250 t/s of usable throughput your map skips Intel because the Arc cards looked weak in 2025, the B70 plus the vLLM stack flipped that, $1,900 in dual Intel silicon now serves a 4 person team at the speed openai serves enterprise tiers for $400 a month per seat bookmark this and read the article below

DUAL INTEL ARC PRO B70 GPUS AT $1,900 RUN QWEN 3 CODER 30B AT 268 TOKENS PER SECOND VIA VLLM ON 8 CONCURRENT REQUESTS, A NEW INTEL TIER UNDER YOUR MAP THAT BEATS RTX 3090 STACKS ON THROUGHPUT PER DOLLAR 02:22 the operator reads off his benchmark sheet, "vllm was at 268 tokens per second which is just wild to me, getting 250+ tokens per second out of these overall kind of slow Intel cards" dual Intel Arc Pro B70 ships at $950 per card, 32GB GDDR6 each, 64GB combined VRAM for $1,900 total, the same budget gets you two used RTX 3090s at 48GB combined and worse multi-user throughput Qwen 3 coder 30B on dual B70 with vLLM hits 268 t/s at 8 concurrent users, llama.cpp tops out at 84 t/s on the same hardware, the gap is not 2x or 3x, vLLM with Intel's tweaks pulls 3.2x on parallel inference prefill on a 16,000 word context took 26 seconds on llama.cpp and 3 seconds on vLLM, agents spawning subagents or households sharing one box turn that 8x speedup into 250 t/s of usable throughput your map skips Intel because the Arc cards looked weak in 2025, the B70 plus the vLLM stack flipped that, $1,900 in dual Intel silicon now serves a 4 person team at the speed openai serves enterprise tiers for $400 a month per seat bookmark this and read the article below

starmex

28,401 views • 1 month ago

AI will be a massive accelerant for fast-growing economies. GDP growth that might have otherwise taken 50 years can now happen in 10, unleashing trillions of dollars in value. Till now, India has been… - a net exporter of people (talent) & - a net importer of agents (models) …that era ends now. AI is making tools of acceleration widely available so the best models, agents and agentic companies can be built from India. Emergent is blazing a new trail and it is time for others to follow suit. Proud to partner with Emergent on Vibecon to accelerate the agentic era of India!

AI will be a massive accelerant for fast-growing economies. GDP growth that might have otherwise taken 50 years can now happen in 10, unleashing trillions of dollars in value. Till now, India has been… - a net exporter of people (talent) & - a net importer of agents (models) …that era ends now. AI is making tools of acceleration widely available so the best models, agents and agentic companies can be built from India. Emergent is blazing a new trail and it is time for others to follow suit. Proud to partner with Emergent on Vibecon to accelerate the agentic era of India!

Sreeram Kannan

46,600 views • 4 months ago

Google engineer explained how to fine-tune a tiny LLM from 46% to 90% accuracy on your phone in 21 minutes - better than $1500 on-device AI bootcamps. pick Gemma 270M -> generate synthetic task data -> fine-tune with LoRA -> quantize to int4 -> deploy to Pixel and hit 2000 tokens per second. That loop is how a 270M model beats a 70B one on your task, running fully offline in your pocket. Gemma 270M + synthetic data + LoRA + int4 quantization + on-device runtime - that's the stack. Watch and save it, then fine-tune your own tiny agent tonight.

Google engineer explained how to fine-tune a tiny LLM from 46% to 90% accuracy on your phone in 21 minutes - better than $1500 on-device AI bootcamps. pick Gemma 270M -> generate synthetic task data -> fine-tune with LoRA -> quantize to int4 -> deploy to Pixel and hit 2000 tokens per second. That loop is how a 270M model beats a 70B one on your task, running fully offline in your pocket. Gemma 270M + synthetic data + LoRA + int4 quantization + on-device runtime - that's the stack. Watch and save it, then fine-tune your own tiny agent tonight.

h100envy

701,421 views • 25 days ago

Tencent's Hy3 lands today with real coding and agent ability and a price that undercuts the models you'd put it against. It's a 295B MoE with only 21B active parameters, a big part of why it stays cheap to serve, plus a 256K context window and an Apache 2.0 license that even outperforms GLM-5.1 although GLM is 2.5x larger. On OpenRouter it runs at $0.14 per million input tokens and $0.58 per million output. For a model Tencent benchmarks against much larger flagships, that pricing is insane. Price-performance-ratio is China's real moat. I already wrote about Tencent Hy3 earlier and the significant leap in quality it represents. Therefore, I'm pleased to recommend this model from Tencent Hy. Open Router link down below: #Hy3 #Hunyuan #AICoding

Tencent's Hy3 lands today with real coding and agent ability and a price that undercuts the models you'd put it against. It's a 295B MoE with only 21B active parameters, a big part of why it stays cheap to serve, plus a 256K context window and an Apache 2.0 license that even outperforms GLM-5.1 although GLM is 2.5x larger. On OpenRouter it runs at $0.14 per million input tokens and $0.58 per million output. For a model Tencent benchmarks against much larger flagships, that pricing is insane. Price-performance-ratio is China's real moat. I already wrote about Tencent Hy3 earlier and the significant leap in quality it represents. Therefore, I'm pleased to recommend this model from Tencent Hy. Open Router link down below: #Hy3 #Hunyuan #AICoding

Chubby♨️

22,879 views • 1 month ago

Timelapse #156 (36 hrs) - Worked with the tiny corp on getting GLM 5.2 running on 8xMI300X (sglang won here) - Launched KernelBench-Mega and updated Kernelbench-Hard with h100 and b200 sweeps - Took care of boring business stuff - Did some training sweeps for specialized technical vocab audio model - Some bugs with putting kernelbench-mega and hard on cloud instances so had to do some reruns. Learned a lot though - Setting up my own local rl infra and profiling concurrency 128 rollouts with vllm. Became clear to me that I need to serve in nvfp4, use MoE only for throughput, reap so training doesn’t OOM, dig into the vllm kernel graph itself to not underutilize my hardware from poor flashinfer/cutlass selections for my rtx pro 6000 sm120 architecture - Might do online distillation from glm 5.2 but for now taking it one step at a time - Slept for a bit then woke up and showered - Fixed an issue with SGLang tensor parallel deadlock on GLM 5.2 architecture with MTP enabled - GLM 5.2 inference is 2-3x faster than coding plans and running on amd boxes - Spent time with family - Hung out with some friends - Recorded some yoctogpt lectures with the revamped notebook (high taste btw) - Setting up dflash training for GLM 5.2

Timelapse #156 (36 hrs) - Worked with the tiny corp on getting GLM 5.2 running on 8xMI300X (sglang won here) - Launched KernelBench-Mega and updated Kernelbench-Hard with h100 and b200 sweeps - Took care of boring business stuff - Did some training sweeps for specialized technical vocab audio model - Some bugs with putting kernelbench-mega and hard on cloud instances so had to do some reruns. Learned a lot though - Setting up my own local rl infra and profiling concurrency 128 rollouts with vllm. Became clear to me that I need to serve in nvfp4, use MoE only for throughput, reap so training doesn’t OOM, dig into the vllm kernel graph itself to not underutilize my hardware from poor flashinfer/cutlass selections for my rtx pro 6000 sm120 architecture - Might do online distillation from glm 5.2 but for now taking it one step at a time - Slept for a bit then woke up and showered - Fixed an issue with SGLang tensor parallel deadlock on GLM 5.2 architecture with MTP enabled - GLM 5.2 inference is 2-3x faster than coding plans and running on amd boxes - Spent time with family - Hung out with some friends - Recorded some yoctogpt lectures with the revamped notebook (high taste btw) - Setting up dflash training for GLM 5.2

Elliot Arledge

45,447 views • 1 month ago

MiniCPM-V 4.5 achieves an average score of 77.0 on OpenCompass, a comprehensive evaluation of 8 popular benchmarks. With only 8B parameters, it surpasses widely used proprietary models like GPT-4o-latest, Gemini-2.0 Pro, and strong open-source models like Qwen2.5-VL 72B powered by a new unified 3D-Resampler over images and videos, MiniCPM-V 4.5 can now achieve 96x compression rate for video tokens, where 6 448x448 video frames can be jointly compressed into 64 video tokens (normally 1,536 tokens for most MLLMs). vibe coding a Video Chat AI app with MiniCPM-V-4.5 in anycoder

MiniCPM-V 4.5 achieves an average score of 77.0 on OpenCompass, a comprehensive evaluation of 8 popular benchmarks. With only 8B parameters, it surpasses widely used proprietary models like GPT-4o-latest, Gemini-2.0 Pro, and strong open-source models like Qwen2.5-VL 72B powered by a new unified 3D-Resampler over images and videos, MiniCPM-V 4.5 can now achieve 96x compression rate for video tokens, where 6 448x448 video frames can be jointly compressed into 64 video tokens (normally 1,536 tokens for most MLLMs). vibe coding a Video Chat AI app with MiniCPM-V-4.5 in anycoder

AK

19,088 views • 11 months ago

Unsloth AI and NVIDIA are Revolutionizing Local LLM Fine-Tuning: From RTX Desktops to DGX Spark Fine-tune popular AI models faster with Unsloth on NVIDIA RTX AI PCs such as GeForce RTX desktops and laptops to RTX PRO workstations and the new DGX Spark to build personalized assistants for coding, creative work, and complex agentic workflows. The landscape of modern AI is shifting. We are moving away from a total reliance on massive, generalized cloud models and entering the era of local, agentic AI. Whether it is tuning a chatbot to handle hyper-specific product support or building a personal assistant that manages intricate schedules, the potential for generative AI on local hardware is boundless. However, developers face a persistent bottleneck: How do you get a Small Language Model (SLM) to punch above its weight class and respond with high accuracy for specialized tasks? The answer is Fine-Tuning, and the tool of choice is Unsloth. Unsloth provides an easy and high-speed method to customize models. Optimized for efficient, low-memory training on NVIDIA GPUs, Unsloth scales effortlessly from GeForce RTX desktops and laptop all the way to the DGX Spark, the world’s smallest AI supercomputer...... Full analysis: NVIDIA NVIDIA AI NVIDIA AIDev NVIDIAnewsroom Unsloth AI Unsloth

Unsloth AI and NVIDIA are Revolutionizing Local LLM Fine-Tuning: From RTX Desktops to DGX Spark Fine-tune popular AI models faster with Unsloth on NVIDIA RTX AI PCs such as GeForce RTX desktops and laptops to RTX PRO workstations and the new DGX Spark to build personalized assistants for coding, creative work, and complex agentic workflows. The landscape of modern AI is shifting. We are moving away from a total reliance on massive, generalized cloud models and entering the era of local, agentic AI. Whether it is tuning a chatbot to handle hyper-specific product support or building a personal assistant that manages intricate schedules, the potential for generative AI on local hardware is boundless. However, developers face a persistent bottleneck: How do you get a Small Language Model (SLM) to punch above its weight class and respond with high accuracy for specialized tasks? The answer is Fine-Tuning, and the tool of choice is Unsloth. Unsloth provides an easy and high-speed method to customize models. Optimized for efficient, low-memory training on NVIDIA GPUs, Unsloth scales effortlessly from GeForce RTX desktops and laptop all the way to the DGX Spark, the world’s smallest AI supercomputer...... Full analysis: NVIDIA NVIDIA AI NVIDIA AIDev NVIDIAnewsroom Unsloth AI Unsloth

Marktechpost AI Dev News ⚡

31,551 views • 7 months ago