Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

I tested MTPLX v2 with QWEN 3.6 27B and compared it with oMLX without cache on M5 Max and DGX Spark on vllm using nvfp4 model version. More details in 🧵 I've reached 82.8 tps of max decoding speed! 🔥 Custom Metal Kernel design specifically for this model and... show more

Ivan Fioravanti ᯅ

40,444 subscribers

15,846 views • 1 month ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Manus AI is much crazier than Deepseek moment I just got invitation code, this thing is the craziest AI agent I've seen. 10 examples: 1. Clone Apple website It created a copy of Apple website that looks almost the same, with animations and all. (preview link in comment)

Manus AI is much crazier than Deepseek moment I just got invitation code, this thing is the craziest AI agent I've seen. 10 examples: 1. Clone Apple website It created a copy of Apple website that looks almost the same, with animations and all. (preview link in comment)

el.cine

1,128,136 views • 1 year ago

Nah no, this is not a pre-made or baked Blender model. It is created entirely from scratch using extruded geometry and procedural textures for the mustard hatching pattern on the model... all done with Google Gemini 3 pro AI. In fact, every Three.js 3D model you see on this website has been vibe coded with AI 😎 Live: Code:

Nah no, this is not a pre-made or baked Blender model. It is created entirely from scratch using extruded geometry and procedural textures for the mustard hatching pattern on the model... all done with Google Gemini 3 pro AI. In fact, every Three.js 3D model you see on this website has been vibe coded with AI 😎 Live: Code:

The Bugged Dev

54,391 views • 7 months ago

I gave the new Meta Muse Spark model a bunch of assorted assets and it was able to carefully extract them and place it into a functioning app. I've never seen something with a clear intuition for design and product-first thinking. This is the first AI that *gets* UI design.

I gave the new Meta Muse Spark model a bunch of assorted assets and it was able to carefully extract them and place it into a functioning app. I've never seen something with a clear intuition for design and product-first thinking. This is the first AI that gets UI design.

Michael G

38,830 views • 4 months ago

DeepAgent - Researched and built a website in one shot! 🤯🤯 I gave it a one-liner prompt to build a website by looking things up on the internet, and it came up with this. All the information is correct, the links work, and it even grabbed a photo. Still in limited preview and part of ChatLLM. We will open it up later in the week.

DeepAgent - Researched and built a website in one shot! 🤯🤯 I gave it a one-liner prompt to build a website by looking things up on the internet, and it came up with this. All the information is correct, the links work, and it even grabbed a photo. Still in limited preview and part of ChatLLM. We will open it up later in the week.

Bindu Reddy

24,348 views • 1 year ago

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

max fu

70,450 views • 29 days ago

My dual RTX PRO 6000 setup is currently training a Draft model for Qwen 3.6 27B! 🔥 I'm taking the paper DeepSeek dropped on 6/26 and going for a super ambitious application to the 27B scale. Thanks to my homelab, I was able to dive straight in — I read the paper and immediately started experimenting. The amount I've learned has been insane: - How memory bandwidth bottlenecks speed and clever ways to hack around it - Methods to train the draft model and boost its accuracy - Mechanisms to reference tokens all the way back to the previous one to skyrocket draft acceptance rates - The impact of Attention vs. GateDeltaNet on speculative decoding performance and how to handle those differences - The unique approaches and trade-offs of MTP, Dflash, JetSpec, and DSpark I could go on forever, but just from speculative decoding alone I've learned so much. The 27B architecture feels way more DSpark-native than JetSpec, so once draft training finishes, I'm going all-in with DSpark! My goal is to beat existing speculative decoding speeds outright — no task-specific shortcuts or cheating, pure general improvement. If you're into this kind of research, I'd love to hear your thoughts, impressions, and any suggestions — please reply! 🚀

My dual RTX PRO 6000 setup is currently training a Draft model for Qwen 3.6 27B! 🔥 I'm taking the paper DeepSeek dropped on 6/26 and going for a super ambitious application to the 27B scale. Thanks to my homelab, I was able to dive straight in — I read the paper and immediately started experimenting. The amount I've learned has been insane: - How memory bandwidth bottlenecks speed and clever ways to hack around it - Methods to train the draft model and boost its accuracy - Mechanisms to reference tokens all the way back to the previous one to skyrocket draft acceptance rates - The impact of Attention vs. GateDeltaNet on speculative decoding performance and how to handle those differences - The unique approaches and trade-offs of MTP, Dflash, JetSpec, and DSpark I could go on forever, but just from speculative decoding alone I've learned so much. The 27B architecture feels way more DSpark-native than JetSpec, so once draft training finishes, I'm going all-in with DSpark! My goal is to beat existing speculative decoding speeds outright — no task-specific shortcuts or cheating, pure general improvement. If you're into this kind of research, I'd love to hear your thoughts, impressions, and any suggestions — please reply! 🚀

Hikari∣LocalLLM⚡

56,870 views • 1 month ago

/1 Gemma 4 31B just crushed Qwen 3.6 27B in a local LLM gamedev contest inside atomic.chat (prompt is below) Device: MacBook Pro M5 Max, 64GB RAM Results: Qwen 3.6 27B: 32 tokens/sec · 18m 04s · 33,946 tokens Gemma 4 31B: 27 tokens/sec · 3m 51s · 6,209 tokens So what is more important: tokens per second, or the quality of the final answer? Qwen made a very long response and showed more creativity and visual style. But Gemma gave a shorter, clearer, and more logical answer in much less time. In this one-shot Pac-Man gamedev contest, Gemma 4 31B was the clear winner. Its game logic was stronger: click reactions were smoother, and it handled interactions with elements like walls, ghosts, and particle effects better. But this was only one test. Maybe Qwen 3.6 27B can show better results with better settings. Open the comments, try our prompt, and share your result below.

/1 Gemma 4 31B just crushed Qwen 3.6 27B in a local LLM gamedev contest inside atomic.chat (prompt is below) Device: MacBook Pro M5 Max, 64GB RAM Results: Qwen 3.6 27B: 32 tokens/sec · 18m 04s · 33,946 tokens Gemma 4 31B: 27 tokens/sec · 3m 51s · 6,209 tokens So what is more important: tokens per second, or the quality of the final answer? Qwen made a very long response and showed more creativity and visual style. But Gemma gave a shorter, clearer, and more logical answer in much less time. In this one-shot Pac-Man gamedev contest, Gemma 4 31B was the clear winner. Its game logic was stronger: click reactions were smoother, and it handled interactions with elements like walls, ghosts, and particle effects better. But this was only one test. Maybe Qwen 3.6 27B can show better results with better settings. Open the comments, try our prompt, and share your result below.

Chubby♨️

72,566 views • 3 months ago

Introducing Jan v3, with updates to Jan Desktop v0.7.6 💛 Jan v3 is our first v3 model, a 4B base built for fine-tuning and fast local use, with stronger math and coding. This release also includes a small Jan Desktop update, starting with a UI refresh as we move toward a more unified Jan experience. Try it: - Jan v3 is available in Jan Desktop and at - Get the latest desktop app at Model: - Jan-v3-4B: - Jan-v3-4B-GGUF: Thanks Qwen for the base model and Georgi Gerganov for llama.cpp 💛

Introducing Jan v3, with updates to Jan Desktop v0.7.6 💛 Jan v3 is our first v3 model, a 4B base built for fine-tuning and fast local use, with stronger math and coding. This release also includes a small Jan Desktop update, starting with a UI refresh as we move toward a more unified Jan experience. Try it: - Jan v3 is available in Jan Desktop and at - Get the latest desktop app at Model: - Jan-v3-4B: - Jan-v3-4B-GGUF: Thanks Qwen for the base model and Georgi Gerganov for llama.cpp 💛

👋 Jan

53,900 views • 6 months ago

Meet the new look of Cluely. ~1,200 hours, and 3 months of teamwork dedicated to the exploration of the look, design, motion, and development of the new website for the most viral startup. The main challenge was to achieve a "generalized" look while having something special, so the site could be practical and efficient for any type of audience. I hope we succeeded with this. Stay tuned for a massive project breakdown later this month on YT. PS: Roy is a great guy and super easy to work with. The best client ever.

Meet the new look of Cluely. ~1,200 hours, and 3 months of teamwork dedicated to the exploration of the look, design, motion, and development of the new website for the most viral startup. The main challenge was to achieve a "generalized" look while having something special, so the site could be practical and efficient for any type of audience. I hope we succeeded with this. Stay tuned for a massive project breakdown later this month on YT. PS: Roy is a great guy and super easy to work with. The best client ever.

Alex Barashkov

632,265 views • 1 year ago

1/ Gemini 2.5 is here, and it’s our most intelligent AI model ever. Our first 2.5 model, Gemini 2.5 Pro Experimental is a state-of-the-art thinking model, leading in a wide range of benchmarks – with impressive improvements in enhanced reasoning and coding and now #1 on Arena by a significant margin. With a model this intelligent, we wanted to get it to people as quickly as possible. Find it on Google AI Studio and in the Google Gemini for Gemini Advanced users now – and in Vertex in the coming weeks. This is the start of a new era of thinking models – and we can’t wait to see where things go from here.

1/ Gemini 2.5 is here, and it’s our most intelligent AI model ever. Our first 2.5 model, Gemini 2.5 Pro Experimental is a state-of-the-art thinking model, leading in a wide range of benchmarks – with impressive improvements in enhanced reasoning and coding and now #1 on Arena by a significant margin. With a model this intelligent, we wanted to get it to people as quickly as possible. Find it on Google AI Studio and in the Google Gemini for Gemini Advanced users now – and in Vertex in the coming weeks. This is the start of a new era of thinking models – and we can’t wait to see where things go from here.

Sundar Pichai

864,496 views • 1 year ago

I finally got GLM-5.2 to work an entire 3-year patient chart that only Bonsai 27B was allowed to read 🔥 292 encounters live inside Bonsai on my Mac Studio. llama.cpp, Metal, ternary, 7.2GB, Apache-2.0. The chart never leaves the machine. GLM-5.2 can only ask questions. It asked three. Bonsai answered each in ~2s with 19,398 tokens still cached. Then it caught the thing buried 17 months back: metformin + iodinated contrast at eGFR 39. Nephrology warned about it in 2025. The ED booked the CT anyway. A 27B-class model used to need a datacentre. PrismML say the 1-bit build is 3.9GB and fits an iPhone 17 Pro Max. The orchestrator never touched the data. That's the whole point. What should it read next?

I finally got GLM-5.2 to work an entire 3-year patient chart that only Bonsai 27B was allowed to read 🔥 292 encounters live inside Bonsai on my Mac Studio. llama.cpp, Metal, ternary, 7.2GB, Apache-2.0. The chart never leaves the machine. GLM-5.2 can only ask questions. It asked three. Bonsai answered each in ~2s with 19,398 tokens still cached. Then it caught the thing buried 17 months back: metformin + iodinated contrast at eGFR 39. Nephrology warned about it in 2025. The ED booked the CT anyway. A 27B-class model used to need a datacentre. PrismML say the 1-bit build is 3.9GB and fits an iPhone 17 Pro Max. The orchestrator never touched the data. That's the whole point. What should it read next?

Maziyar PANAHI

53,049 views • 1 month ago

I just tested the Hitem3D V2.0 Beta It captures complex details with true print-grade geometry. We’re talking ready for 3D printing, CNC, and even laser cutting straight out of generation. Look at this incredible result down below. On the left side the image, on the right side the 3D Model

I just tested the Hitem3D V2.0 Beta It captures complex details with true print-grade geometry. We’re talking ready for 3D printing, CNC, and even laser cutting straight out of generation. Look at this incredible result down below. On the left side the image, on the right side the 3D Model

Chubby♨️

64,565 views • 7 months ago

A quick test of using 3d drawing with 6DOF controllers as an "instructor" for a generative AI process. There's so many powerful and fun new ways to create just around the corner.. Here I'm using Dreams, Krea and 3daistudio. The 3d model at the end of the video was generated from the Dreams+Krea output in just around 15 seconds. Only the model on the left is a "true" 3d model. #ai #madeindreams

A quick test of using 3d drawing with 6DOF controllers as an "instructor" for a generative AI process. There's so many powerful and fun new ways to create just around the corner.. Here I'm using Dreams, Krea and 3daistudio. The 3d model at the end of the video was generated from the Dreams+Krea output in just around 15 seconds. Only the model on the left is a "true" 3d model. #ai #madeindreams

Martin Nebelong

132,864 views • 2 years ago

Salute to the Qwen team 🫡 We tested Qwen 3.7-Max, Gemini 3.5 Flash, GPT-5.5, and Claude Opus 4.7. The biggest shock came from Qwen. In less than a month (3.6 Max dropped April 20), Qwen went from the worst multimodal output on our sakura tree test, barely keeping up with Gemini, GPT, and Claude , to matching Gemini 3.5 Flash frame for frame on this soccer test, and outperforming GPT-5.5 and Claude Opus 4.7. It rendered a perfectly proportioned soccer player and the most lifelike ball in the entire test. Remarkable spatial reasoning. Also: Gemini 3.5 Flash is now faster than GPT-5.5, which used to be the fastest in our past tests.

Salute to the Qwen team 🫡 We tested Qwen 3.7-Max, Gemini 3.5 Flash, GPT-5.5, and Claude Opus 4.7. The biggest shock came from Qwen. In less than a month (3.6 Max dropped April 20), Qwen went from the worst multimodal output on our sakura tree test, barely keeping up with Gemini, GPT, and Claude , to matching Gemini 3.5 Flash frame for frame on this soccer test, and outperforming GPT-5.5 and Claude Opus 4.7. It rendered a perfectly proportioned soccer player and the most lifelike ball in the entire test. Remarkable spatial reasoning. Also: Gemini 3.5 Flash is now faster than GPT-5.5, which used to be the fastest in our past tests.

GMI Cloud

75,785 views • 2 months ago

HyperNodes Milestone Alert ⚡️ Lumia $ORN is thrilled to announce that we’ve reached a monumental milestone: more than 10,000 nodes sold and $10 MILLION DOLLARS RAISED IN FIRST 10 DAYS! This is a testament to the support from our amazing community and our relentless team. 👇 But this is just the beginning – we’re only getting started. Stay tuned for more exciting updates and innovations. Don’t forget to check out our node calculator on our website to see how you can be a part of our journey. Thank you for being with us every step of the way. The future is bright, and we can’t wait to share it with you! Nodes available for purchase 👉

HyperNodes Milestone Alert ⚡️ Lumia $ORN is thrilled to announce that we’ve reached a monumental milestone: more than 10,000 nodes sold and $10 MILLION DOLLARS RAISED IN FIRST 10 DAYS! This is a testament to the support from our amazing community and our relentless team. 👇 But this is just the beginning – we’re only getting started. Stay tuned for more exciting updates and innovations. Don’t forget to check out our node calculator on our website to see how you can be a part of our journey. Thank you for being with us every step of the way. The future is bright, and we can’t wait to share it with you! Nodes available for purchase 👉

Lumia

404,254 views • 2 years ago

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fang

MolmoAct2 is landing in LeRobot! Ai2's open Action Reasoning Model combines a Molmo2-ER vision-language backbone with a flow-matching continuous action expert to predict robot action chunks from images, language instructions, and proprioceptive state. An open robot foundation model built for real-world control, with strong out-of-the-box performance and easy fine-tuning in LeRobot. Pick-and-place inference running on NVIDIA DGX Spark! Blog: Paper: Thanks to Ai2 Jiafei Duan Haoquan Fang

LeRobot

24,727 views • 2 months ago

Big moment for text-to-speech. Qwen just open-sourced a text-to-speech model that lets you clone voices, design new ones, and control speech using natural language. Let me explain what I mean: You can literally tell it "speak in a cheerful tone with slight nervousness," and it actually does that. No complex audio engineering needed. What makes this special: - 3-second voice cloning - Covers 10 languages: English, German, French, and more - Latency as low as 97ms for real-time applications - Supports both streaming and non-streaming generation The model comes in two sizes (0.6B and 1.7B parameters), so you can pick based on your hardware and quality needs. Three modes to work with: 1. Custom Voice: Use pre-built premium voices with instruction-based style control 2. Voice Design: Describe the voice you want in plain English (or Chinese), and the model creates it 3. Voice Clone: Provide a 3-second reference audio and clone that voice The best part? It integrates with vLLM for production deployment and has a simple Python package you can pip install. I've shared a link to the GitHub repo in the next tweet.

Big moment for text-to-speech. Qwen just open-sourced a text-to-speech model that lets you clone voices, design new ones, and control speech using natural language. Let me explain what I mean: You can literally tell it "speak in a cheerful tone with slight nervousness," and it actually does that. No complex audio engineering needed. What makes this special: - 3-second voice cloning - Covers 10 languages: English, German, French, and more - Latency as low as 97ms for real-time applications - Supports both streaming and non-streaming generation The model comes in two sizes (0.6B and 1.7B parameters), so you can pick based on your hardware and quality needs. Three modes to work with: 1. Custom Voice: Use pre-built premium voices with instruction-based style control 2. Voice Design: Describe the voice you want in plain English (or Chinese), and the model creates it 3. Voice Clone: Provide a 3-second reference audio and clone that voice The best part? It integrates with vLLM for production deployment and has a simple Python package you can pip install. I've shared a link to the GitHub repo in the next tweet.

Akshay 🚀

31,249 views • 6 months ago

My brother and I never imagined would grow this quickly - huge thanks and shoutout to the Tekken community! But, the upsurge of traffic hit us with massive hosting/storage costs we didn't expect. It's just the two of us running and funding this, so if you'd like to support the site (no obligation of course), we've set up a page here: Thank you everyone for visiting our website and using it to lab. It is amazing to have made a meaningful contribution to the community and game we grew up with.

My brother and I never imagined would grow this quickly - huge thanks and shoutout to the Tekken community! But, the upsurge of traffic hit us with massive hosting/storage costs we didn't expect. It's just the two of us running and funding this, so if you'd like to support the site (no obligation of course), we've set up a page here: Thank you everyone for visiting our website and using it to lab. It is amazing to have made a meaningful contribution to the community and game we grew up with.

Okizeme.gg

76,522 views • 11 months ago

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with *decoupled* diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -

World modeling and imitation learning have largely been considered two disparate worlds. In our recent work, Unified World Models, just accepted to #RSS2025, Chuning Zhu provides a dead-simple unifying solution: just train a joint diffusion model over actions and future states, but with decoupled diffusion time steps across these modalities. Manipulating these decoupled time steps then allows for marginalization or conditioning on actions or states; a single model can serve as a policy, forward dynamics model, video prediction model, or inverse dynamics model by simply setting diffusion timesteps carefully. The resulting model can leverage video datasets along with robot training data much more effectively, and shows improved robustness, generalization, and flexibility. This is exciting because it is frustratingly simple, scalable, and shows strong improvement on real-world robotics problems. Please refer to Chuning Zhu 's excellent thread for more details! More details/code can be found on our website and in the paper -

Abhishek Gupta

11,430 views • 1 year ago