正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Luminal fuses entire models into a single GPU kernel, automatically. Let's talk about why this matters for inference at the speed of light:

Joe Fioti

2,611 subscribers

24,326 次观看 • 5 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Reasoning LLMs generate very long chains-of-thought, so even small quantization errors add up. With AWQ, Qwen3-4B drops 71.0 → 68.2 on MMLU-Pro (~4% relative loss). 😬 ParoQuant fixes this! It keeps only the critical rotation pairs and fuses everything into a single kernel. Recovers most of the lost reasoning accuracy with minimal overhead — so 4-bit models stay strong at reasoning. 💪💪

Reasoning LLMs generate very long chains-of-thought, so even small quantization errors add up. With AWQ, Qwen3-4B drops 71.0 → 68.2 on MMLU-Pro (~4% relative loss). 😬 ParoQuant fixes this! It keeps only the critical rotation pairs and fuses everything into a single kernel. Recovers most of the lost reasoning accuracy with minimal overhead — so 4-bit models stay strong at reasoning. 💪💪

Zhijian Liu

170,883 次观看 • 4 个月前

Manual QA is dead Engineers used to wait hours, sometimes days, for QA to green-light a single change. That loop is collapsing with agents. Verify in minutes, fix in minutes, and ship with confidence at the speed of inference.

Manual QA is dead Engineers used to wait hours, sometimes days, for QA to green-light a single change. That loop is collapsing with agents. Verify in minutes, fix in minutes, and ship with confidence at the speed of inference.

hayden

19,627 次观看 • 1 个月前

I didn't hear a single Democrat talk about this while it was happening in front of the entire world.

I didn't hear a single Democrat talk about this while it was happening in front of the entire world.

aka

113,705 次观看 • 5 个月前

Why does NVIDIA spend millions of GPU hours training models just to give it all away? Here's a recording to the talk at NeurIPS for anyone who couldn't make it, wish we recorded the questions afterwards!

Why does NVIDIA spend millions of GPU hours training models just to give it all away? Here's a recording to the talk at NeurIPS for anyone who couldn't make it, wish we recorded the questions afterwards!

Nader Khalil🍊

126,208 次观看 • 6 个月前

Is 1500 transactions per second on the XRP Ledger suitable for global finance? Let's talk about how it works - speed matters, but efficiency and reliability define its future.

Is 1500 transactions per second on the XRP Ledger suitable for global finance? Let's talk about how it works - speed matters, but efficiency and reliability define its future.

Jake Claver, QFOP

29,066 次观看 • 8 个月前

$500b wiped out because of a single tweet? Private market valuations of Anthropic, OpenAI, SpaceXai crashed yesterday...let's talk about why. (Please don't make me explain the rug pull joke at the beginning 🤦‍♂️)

$500b wiped out because of a single tweet? Private market valuations of Anthropic, OpenAI, SpaceXai crashed yesterday...let's talk about why. (Please don't make me explain the rug pull joke at the beginning 🤦‍♂️)

Matthew Berman

142,921 次观看 • 1 个月前

Let's talk about the "feminist conservative" women. This is one of the most controversial topics you can talk about. That's why we need to talk about it.

Let's talk about the "feminist conservative" women. This is one of the most controversial topics you can talk about. That's why we need to talk about it.

Jedediah Bila

79,090 次观看 • 1 年前

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,583 次观看 • 2 年前

The Entire Financial System Will Move at the Speed of Light With Blockchain #XRP #Crypto

The Entire Financial System Will Move at the Speed of Light With Blockchain #XRP #Crypto

mickle

12,677 次观看 • 1 年前

at the speed of light (no pun intended) My talk for Deep Tech Week: optics, cancer, diagnostics, and Precigenetics

at the speed of light (no pun intended) My talk for Deep Tech Week: optics, cancer, diagnostics, and Precigenetics

Parmita Mishra

33,942 次观看 • 1 年前

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering

Madison Kanna

119,268 次观看 • 2 个月前

The Metaphysics of Slop Let's talk about why slop is bad.

The Metaphysics of Slop Let's talk about why slop is bad.

Carl Benjamin 🏴󠁧󠁢󠁥󠁮󠁧󠁿

168,898 次观看 • 1 年前

This Shabbat, let's talk about a different kind of candles - how much do you know about the reason behind candle-lighting on Hannukah? Have a listen to find out why even when choosing light, you have to do it for the right reasons. Shabbat shalom! 🕯️🕯️

This Shabbat, let's talk about a different kind of candles - how much do you know about the reason behind candle-lighting on Hannukah? Have a listen to find out why even when choosing light, you have to do it for the right reasons. Shabbat shalom! 🕯️🕯️

Ambassador Yechiel (Michael) Leiter

13,598 次观看 • 6 个月前

2026: the year of explosive AI innovation. Fireworks AI Co-founder & CEO Lin Qiao speaks on how the massive expansion of customized inference and models is creating speed of light production to scale timelines.

2026: the year of explosive AI innovation. Fireworks AI Co-founder & CEO Lin Qiao speaks on how the massive expansion of customized inference and models is creating speed of light production to scale timelines.

NVIDIA

35,716 次观看 • 1 个月前

Let's talk about the thriving business of poverty. In this personal story, I want to show you how easy it is to fall below the poverty line in a struggling economy, and why it is very dangerous for the entire nation.

Let's talk about the thriving business of poverty. In this personal story, I want to show you how easy it is to fall below the poverty line in a struggling economy, and why it is very dangerous for the entire nation.

Jude Bela

65,890 次观看 • 1 年前

Astrophysicist Neil deGrasse Tyson explains, the faster you move through space, the slower you move through time. As you approach the speed of light, time stretches and slows down until it reaches the ultimate cosmic speed limit, where time completely stops. This means that for a photon of light created at the Big Bang 13.8 billion years ago, the exact moment it was born is the exact same instant it strikes your eye; a mind-bending reality proving that for light, the entire history of the universe happens in a single, frozen moment.

Astrophysicist Neil deGrasse Tyson explains, the faster you move through space, the slower you move through time. As you approach the speed of light, time stretches and slows down until it reaches the ultimate cosmic speed limit, where time completely stops. This means that for a photon of light created at the Big Bang 13.8 billion years ago, the exact moment it was born is the exact same instant it strikes your eye; a mind-bending reality proving that for light, the entire history of the universe happens in a single, frozen moment.

Cosmos Archive

87,906 次观看 • 2 个月前

Young people have so much on the line this election. That’s why I was so grateful to talk with the young women of Theta at MSU (at my former sorority house!) about what’s at stake and why their vote matters.

Young people have so much on the line this election. That’s why I was so grateful to talk with the young women of Theta at MSU (at my former sorority house!) about what’s at stake and why their vote matters.

Gretchen Whitmer

557,920 次观看 • 1 年前

BBC: "You changed the Covid misinformation labels... Why do that?" Musk: "Does the BBC hold itself at all responsible for misinformation regarding masking, and side effects of vaccinations?" BBC: "Let's talk about something else." Musk: "You weren't expecting that." BBC: "Let's talk about something else."

BBC: "You changed the Covid misinformation labels... Why do that?" Musk: "Does the BBC hold itself at all responsible for misinformation regarding masking, and side effects of vaccinations?" BBC: "Let's talk about something else." Musk: "You weren't expecting that." BBC: "Let's talk about something else."

Wide Awake Media

4,135,885 次观看 • 1 年前

Slash AI costs by 99%, run 1000+ models on a single GPU! OpenLoRA lets you run thousands of models on one GPU instead of one per model, drastically cutting costs and boosting flexibility. Excited to work with Aethir, Hyperbolic for infrastructure and for verifiability. Watch the demo:

Slash AI costs by 99%, run 1000+ models on a single GPU! OpenLoRA lets you run thousands of models on one GPU instead of one per model, drastically cutting costs and boosting flexibility. Excited to work with Aethir, Hyperbolic for infrastructure and for verifiability. Watch the demo:

OpenLedger

155,267 次观看 • 1 年前