Loading video...

Video Failed to Load

Go Home

K-Means is simple. Making it fast on GPU isn't. Flash-KMeans is an IO-aware implementation of exact k-means that rethinks the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 30x speedup over cuML - 200x speedup over FAISS Using the same exact algorithm, just...

DailyDoseOfDS_'s profile picture

Daily Dose of Data Science

48,995 subscribers

23,748 views • 6 months ago •via X (Twitter)

Science & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

I updated my quantization compiler to use e-means as a replacement for k-means in codebook generation and got Qwen3.5-4B from 8.41 GB to 1.23 GB while retaining 91.9% of original performance which I think is approximately SOTA, at least comparable to bonsai 2 but I think we could go lower if someone wanted to donate some GPU time since this is just a hobby for me. at this size, it's small enough that memory bandwidth isn't the bottleneck to decode speed so I'm interested in seeing how fast we can get larger models on consumer hardware. link below.
0:18

I updated my quantization compiler to use e-means as a replacement for k-means in codebook generation and got Qwen3.5-4B from 8.41 GB to 1.23 GB while retaining 91.9% of original performance which I think is approximately SOTA, at least comparable to bonsai 2 but I think we could go lower if someone wanted to donate some GPU time since this is just a hobby for me. at this size, it's small enough that memory bandwidth isn't the bottleneck to decode speed so I'm interested in seeing how fast we can get larger models on consumer hardware. link below.

𝞍 Shin Megami Boson 𝞍

27,667 views • 2 days ago

"We have the highest stock market in the history of our country... That means jobs, and it means 401(k)s" - President Donald J. Trump 🇺🇸
0:15

"We have the highest stock market in the history of our country... That means jobs, and it means 401(k)s" - President Donald J. Trump 🇺🇸

The White House

320,640 views • 10 months ago

“I like it, I need it, I only interested to f**k” Somalian Bro about to cross into the UK - means this doesn’t end well.
0:14

Sensitive content

“I like it, I need it, I only interested to f**k” Somalian Bro about to cross into the UK - means this doesn’t end well.

Concerned Citizen

116,230 views • 2 months ago

“The strike that k*lled over 100 Gazans, Israel declared that the ceasefire is still in effect. Still in effect, because ceasefire to the Israelis means you cease, we fire. And that's what they've been doing.” —Ana Kasparian
0:13

“The strike that k*lled over 100 Gazans, Israel declared that the ceasefire is still in effect. Still in effect, because ceasefire to the Israelis means you cease, we fire. And that's what they've been doing.” —Ana Kasparian

The Resonance

314,437 views • 1 year ago

Mikel Brown Jr. on what it means to play for the city of Brooklyn: When you hear somebody say they from Brooklyn, they take pride in it... to come in and kinda have the same mentality of you representing it and it's not just about you, it means the world"
0:19

Mikel Brown Jr. on what it means to play for the city of Brooklyn: When you hear somebody say they from Brooklyn, they take pride in it... to come in and kinda have the same mentality of you representing it and it's not just about you, it means the world"

SNY Nets

39,154 views • 4 months ago

R2 is designed to take you far. For longer adventures, that means charging along the way, which is why we’ve made charging easy to find and easy to use. Discover what you need to know about fast charging on the go at
0:15

R2 is designed to take you far. For longer adventures, that means charging along the way, which is why we’ve made charging easy to find and easy to use. Discover what you need to know about fast charging on the go at

Rivian

23,282 views • 1 month ago

Gillian Keegan's bonkers response to losing, "Volatility in politics is the story.. What it means and what it means for the future, I don't know" Can anyone translate this for me?
0:14

Gillian Keegan's bonkers response to losing, "Volatility in politics is the story.. What it means and what it means for the future, I don't know" Can anyone translate this for me?

Farrukh

90,965 views • 2 years ago

THIS IS WHAT IT MEANS TO THE DODGERS
0:17

Sensitive content

THIS IS WHAT IT MEANS TO THE DODGERS

FOX Sports: MLB

223,365 views • 2 years ago

Porto v0.1 is out with automatic fee token detection logic! This means that your users don't need to hold the native token of a chain to make an action, great UX! It also means your app doesn't need to make any code changes to leverage that feature, great DX! `pnpm i porto`
0:12

Porto v0.1 is out with automatic fee token detection logic! This means that your users don't need to hold the native token of a chain to make an action, great UX! It also means your app doesn't need to make any code changes to leverage that feature, great DX! `pnpm i porto`

Georgios Konstantopoulos

43,830 views • 1 year ago

If you see this on your For You Page, it means the algorithm is working again! 🎉
0:10

If you see this on your For You Page, it means the algorithm is working again! 🎉

Alex Socoloff

109,004 views • 9 months ago

I love the term #bombogenesis - coined in a white paper in 1980. It sounds scary, but really it means “rapidly intensifying storm”. The reason it matters is, the faster a storm deepens (pressure drops) the stronger the wind gusts due to an imbalance in pressure - it forces air to move quickly. So when meteorologists say a #storm is a bomb it means something. In this case gusts over the ocean will reach 100 mph, with coastal gusts to #hurricane force 75 mph. #blizzard #blizzardof2026
0:17

I love the term #bombogenesis - coined in a white paper in 1980. It sounds scary, but really it means “rapidly intensifying storm”. The reason it matters is, the faster a storm deepens (pressure drops) the stronger the wind gusts due to an imbalance in pressure - it forces air to move quickly. So when meteorologists say a #storm is a bomb it means something. In this case gusts over the ocean will reach 100 mph, with coastal gusts to #hurricane force 75 mph. #blizzard #blizzardof2026

Jeff Berardelli

33,815 views • 8 months ago

Spring is coming which means summer is knock at our door. That means it's time to get that beach body ready so you can get your surf on. 🏄 That said the board that the girl in the video has like too small to be a standard size board. I'm no expert on this, so if anyone knows please share in the comments. 👇
0:15

Spring is coming which means summer is knock at our door. That means it's time to get that beach body ready so you can get your surf on. 🏄 That said the board that the girl in the video has like too small to be a standard size board. I'm no expert on this, so if anyone knows please share in the comments. 👇

Giggling Gannon

19,450 views • 7 months ago

Qwen3.8-Flash-Next now reaches ~43 tok/s after a 122,902-token prompt on ONE DGX Spark. ⚡🚀 MTP k=2 won my draft-depth sweep, with +42.5% mean decode over no draft. The PLE table stays fully on-device. I promised the deeper MTP tests. Here are the results, and now you can explore them in an interactive benchmark page too. 𝗧𝗪𝗢 𝗗𝗥𝗔𝗙𝗧 𝗧𝗢𝗞𝗘𝗡𝗦 𝗪𝗢𝗡 Mean single-request decode with 32K context configured: MTP k=2: 39.21 tok/s MTP k=3: 36.42 tok/s MTP k=1: 35.18 tok/s No draft: 27.51 tok/s k=2 also produced the fastest individual sweep run: 41.34 tok/s. Four runs each for no draft, k=1 and k=2. Seven for k=3. Decode excludes time to first token. Here, k means speculative draft depth, not quantization bits. k=3 produced more tokens per step, but the extra drafting work did not pay off in throughput. k=2 is my current pick for this setup. 𝗧𝗛𝗘 𝟭𝟮𝟯𝗞-𝗧𝗢𝗞𝗘𝗡 𝗣𝗥𝗢𝗠𝗣𝗧 𝗧𝗘𝗦𝗧 I then ran a separate long-prompt comparison: Actual input: 122,902 tokens Configured context: 262,144 Requested output: 128 tokens One request at a time MTP k=2: ~43 tok/s No draft: 26.2 tok/s Time to first token: 110.6 seconds with MTP 107.0 seconds without it The win here is generation speed, not faster prefill. To keep the scope clear: 256K was the configured limit. This was a real ~123K input, not a completely filled 256K window or a full k sweep at that depth. 𝗣𝗟𝗘 𝗦𝗧𝗔𝗬𝗦 𝗢𝗡 𝗧𝗛𝗘 𝗦𝗣𝗔𝗥𝗞 Whole model on-device: 78.57 GiB Packed 5-bit PLE table: 30.4 GiB, included in that total No NVMe PLE offload in this build. This is still turboderp’s 3.05bpw_h5_ng5 EXL3 pack, served through my vllm-exl3 integration. My work here is the serving integration and testing. These are preliminary performance measurements, not a quality evaluation or a claim of bit-exact full-output parity. 𝗘𝗫𝗣𝗟𝗢𝗥𝗘 𝗧𝗛𝗘 𝗥𝗘𝗦𝗨𝗟𝗧𝗦 The benchmark page has the individual sweep values, long-prompt comparison, and measurement scope. You can play the animation, export the charts, or download the HTML and data to render them yourself. No Spark needed to view the results. Credit to turboderp / ExLlamaV3 for the pack and kernels, vLLM for the serving engine, and Qwen Qwen Developers for the model. Recipe + reproduction: Interactive benchmark:
0:19

Qwen3.8-Flash-Next now reaches ~43 tok/s after a 122,902-token prompt on ONE DGX Spark. ⚡🚀 MTP k=2 won my draft-depth sweep, with +42.5% mean decode over no draft. The PLE table stays fully on-device. I promised the deeper MTP tests. Here are the results, and now you can explore them in an interactive benchmark page too. 𝗧𝗪𝗢 𝗗𝗥𝗔𝗙𝗧 𝗧𝗢𝗞𝗘𝗡𝗦 𝗪𝗢𝗡 Mean single-request decode with 32K context configured: MTP k=2: 39.21 tok/s MTP k=3: 36.42 tok/s MTP k=1: 35.18 tok/s No draft: 27.51 tok/s k=2 also produced the fastest individual sweep run: 41.34 tok/s. Four runs each for no draft, k=1 and k=2. Seven for k=3. Decode excludes time to first token. Here, k means speculative draft depth, not quantization bits. k=3 produced more tokens per step, but the extra drafting work did not pay off in throughput. k=2 is my current pick for this setup. 𝗧𝗛𝗘 𝟭𝟮𝟯𝗞-𝗧𝗢𝗞𝗘𝗡 𝗣𝗥𝗢𝗠𝗣𝗧 𝗧𝗘𝗦𝗧 I then ran a separate long-prompt comparison: Actual input: 122,902 tokens Configured context: 262,144 Requested output: 128 tokens One request at a time MTP k=2: ~43 tok/s No draft: 26.2 tok/s Time to first token: 110.6 seconds with MTP 107.0 seconds without it The win here is generation speed, not faster prefill. To keep the scope clear: 256K was the configured limit. This was a real ~123K input, not a completely filled 256K window or a full k sweep at that depth. 𝗣𝗟𝗘 𝗦𝗧𝗔𝗬𝗦 𝗢𝗡 𝗧𝗛𝗘 𝗦𝗣𝗔𝗥𝗞 Whole model on-device: 78.57 GiB Packed 5-bit PLE table: 30.4 GiB, included in that total No NVMe PLE offload in this build. This is still turboderp’s 3.05bpw_h5_ng5 EXL3 pack, served through my vllm-exl3 integration. My work here is the serving integration and testing. These are preliminary performance measurements, not a quality evaluation or a claim of bit-exact full-output parity. 𝗘𝗫𝗣𝗟𝗢𝗥𝗘 𝗧𝗛𝗘 𝗥𝗘𝗦𝗨𝗟𝗧𝗦 The benchmark page has the individual sweep values, long-prompt comparison, and measurement scope. You can play the animation, export the charts, or download the HTML and data to render them yourself. No Spark needed to view the results. Credit to turboderp / ExLlamaV3 for the pack and kernels, vLLM for the serving engine, and Qwen Qwen Developers for the model. Recipe + reproduction: Interactive benchmark:

Cruz

12,258 views • 2 months ago

This Mauricio Dubon misplay has been changed to a double. That means the run Sale allowed in the first inning is now earned.
0:18

This Mauricio Dubon misplay has been changed to a double. That means the run Sale allowed in the first inning is now earned.

Mark Bowman

196,329 views • 3 months ago

HOLY SH*T Trump is asked what he means by there will be "hell to pay" in the Middle East if the hostages aren't released by Jan. 20... "Well, they're gonna have to determine what that means. But - it means it won't be pleasant... it's not gonna be pleasant."
0:16

HOLY SH*T Trump is asked what he means by there will be "hell to pay" in the Middle East if the hostages aren't released by Jan. 20... "Well, they're gonna have to determine what that means. But - it means it won't be pleasant... it's not gonna be pleasant."

Eric Daugherty

924,420 views • 1 year ago

Its not every day you wake up to find that the Pope has made your lifes work the central focus of his papacy: “Disarming AI means freeing it from the mentality of “armed” competition [..] This entails a race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance." - POPE LEO XIV, May 2026 Here, “disarmed” means “neutralised” in the sense that this should not be a differentiator The Innovation Game (TIG) was created to keep data and algorithms open, in order to prevent monopolistic control It's not just an aspiration, it's an economic mechanism that makes open data and open algorithms the rational economic choice • All algorithms are published openly by TIG • If you are willing to make the data you process with an algorithm open, you can use it free-of-charge • Alternatively, if you would like to keep this data private, there is a fee to pay for using the algorithm • All fees are used to fund more open innovation Its an elegant, global, self-reinforcing engine The logical end point of monopoly is that innovation stops We cannot allow that to happen Pope Leo XIV I would be grateful for your thoughts on The Innovation Game
0:18

Its not every day you wake up to find that the Pope has made your lifes work the central focus of his papacy: “Disarming AI means freeing it from the mentality of “armed” competition [..] This entails a race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance." - POPE LEO XIV, May 2026 Here, “disarmed” means “neutralised” in the sense that this should not be a differentiator The Innovation Game (TIG) was created to keep data and algorithms open, in order to prevent monopolistic control It's not just an aspiration, it's an economic mechanism that makes open data and open algorithms the rational economic choice • All algorithms are published openly by TIG • If you are willing to make the data you process with an algorithm open, you can use it free-of-charge • Alternatively, if you would like to keep this data private, there is a fee to pay for using the algorithm • All fees are used to fund more open innovation Its an elegant, global, self-reinforcing engine The logical end point of monopoly is that innovation stops We cannot allow that to happen Pope Leo XIV I would be grateful for your thoughts on The Innovation Game

John Fletcher (𝔦, 𝔦)

11,722 views • 5 months ago

Sterling Lockett makes his first career catch at K-State. That means Kevin Lockett, Aaron Lockett, Tyler Lockett and Sterling Lockett have each caught a pass at K-State. The Locketts have now combined for 604 catches. One of the best football families in collegiate athletics.
0:14

Sterling Lockett makes his first career catch at K-State. That means Kevin Lockett, Aaron Lockett, Tyler Lockett and Sterling Lockett have each caught a pass at K-State. The Locketts have now combined for 604 catches. One of the best football families in collegiate athletics.

D. Scott Fritchen

28,112 views • 1 year ago

This is what it means to see both cars in Q3 for the first time this season 👏
0:14

This is what it means to see both cars in Q3 for the first time this season 👏

Atlassian Williams F1 Team

1,956,706 views • 2 years ago

Transformer and Mixture of Experts, explained visually! Mixture of Experts (MoE) is a popular architecture that uses different experts to improve Transformer models. Transformer and MoE differ in the decoder block: - Transformer uses a feed-forward network. - MoE uses experts, which are feed-forward networks but smaller compared to those Transformer. During inference, a subset of experts are selected. This makes inference faster in MoE. Also, since the network has multiple decoder layers: - The text passes through different experts across layers. - The chosen experts also differ between tokens. But how does the model decide which experts should be ideal? The router does that. It is a multi-class classifier that produces softmax scores over experts to select the top K experts. The router is trained with the network, and it learns to select the best experts. But it isn't straightforward. There are challenges! Challenge 1) Notice this pattern at the start of training: - Say, the model selects "Expert 2" - This expert gets a bit better - It may get selected again since it's the "best" - It learns more - It gets selected again in the next iteration - It learns more, and so on! This means many experts can go under-trained due to the overselection of a few experts! We solve this in two steps: - Add noise to the feed-forward output of the router so that other experts can get higher logits. - Set all but the top K logits to -infinity. After softmax, these scores become zero. This way, other experts also get the opportunity to train. Challenge 2) Some experts may get exposed to more tokens than others, leading to under-trained experts. We prevent this by limiting the number of tokens an expert can process. If an expert reaches the limit, the token is passed to the next best expert. Overall, MoEs have more parameters to load. But a fraction of them are activated during inference. This leads to faster inference. Mixtral 8x7B and Llama 4 are two popular MoE-based LLMs. Have you used MoEs in production yet? To dive deeper into how MoE inference works in production, we wrote a full article covering token dispatch, grouped expert computation, model-weight memory, multi-GPU communication, expert placement, load imbalance, and performance diagnosis. Read it below.
0:11

Transformer and Mixture of Experts, explained visually! Mixture of Experts (MoE) is a popular architecture that uses different experts to improve Transformer models. Transformer and MoE differ in the decoder block: - Transformer uses a feed-forward network. - MoE uses experts, which are feed-forward networks but smaller compared to those Transformer. During inference, a subset of experts are selected. This makes inference faster in MoE. Also, since the network has multiple decoder layers: - The text passes through different experts across layers. - The chosen experts also differ between tokens. But how does the model decide which experts should be ideal? The router does that. It is a multi-class classifier that produces softmax scores over experts to select the top K experts. The router is trained with the network, and it learns to select the best experts. But it isn't straightforward. There are challenges! Challenge 1) Notice this pattern at the start of training: - Say, the model selects "Expert 2" - This expert gets a bit better - It may get selected again since it's the "best" - It learns more - It gets selected again in the next iteration - It learns more, and so on! This means many experts can go under-trained due to the overselection of a few experts! We solve this in two steps: - Add noise to the feed-forward output of the router so that other experts can get higher logits. - Set all but the top K logits to -infinity. After softmax, these scores become zero. This way, other experts also get the opportunity to train. Challenge 2) Some experts may get exposed to more tokens than others, leading to under-trained experts. We prevent this by limiting the number of tokens an expert can process. If an expert reaches the limit, the token is passed to the next best expert. Overall, MoEs have more parameters to load. But a fraction of them are activated during inference. This leads to faster inference. Mixtral 8x7B and Llama 4 are two popular MoE-based LLMs. Have you used MoEs in production yet? To dive deeper into how MoE inference works in production, we wrote a full article covering token dispatch, grouped expert computation, model-weight memory, multi-GPU communication, expert placement, load imbalance, and performance diagnosis. Read it below.

Daily Dose of Data Science

55,651 views • 1 month ago