正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Paving the way for unlimited context windows. Introducing ZML /attnd: sparse logarithmic attention, on CPU, over UDP, faster than GPU.

Steeve Morin

8,660 subscribers

121,889 次观看 • 9 个月前 •via X (Twitter)

科学技术新闻政治教育

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Introducing Ziren (formerly zkMIPS) The most performant general-purpose zkVM on CPU, now scaled with GPU, distributed proving, and more. Build with confidence. Deploy without compromise 🟠 Full context: Technical updates:

Introducing Ziren (formerly zkMIPS) The most performant general-purpose zkVM on CPU, now scaled with GPU, distributed proving, and more. Build with confidence. Deploy without compromise 🟠 Full context: Technical updates:

ZKM

5,857,968 次观看 • 11 个月前

$Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.$

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,119,965 次观看 • 1 个月前

$INFQ GPU + CPU + QPU Frontier models are hitting saturation on tokens (data volume) and context windows (memory). $INFQ CTO walks through how they and $NVDA are working together to add a Quantum Processing Unit to the chain as the next inflection point.

$INFQ GPU + CPU + QPU Frontier models are hitting saturation on tokens (data volume) and context windows (memory). $INFQ CTO walks through how they and $NVDA are working together to add a Quantum Processing Unit to the chain as the next inflection point.

YeahDave

57,181 次观看 • 1 个月前

"Agentic AI changes the role of the CPU. The CPU is now the conductor and the GPU is the orchestra". 🎼 NVIDIA Vera is the first CPU built for AI agents — purpose-built from the ground up for how AI works today. Faster. More efficient. Ready for what's next. 🔲 #NVIDIAGTC

"Agentic AI changes the role of the CPU. The CPU is now the conductor and the GPU is the orchestra". 🎼 NVIDIA Vera is the first CPU built for AI agents — purpose-built from the ground up for how AI works today. Faster. More efficient. Ready for what's next. 🔲 #NVIDIAGTC

NVIDIA AI Infrastructure

15,062 次观看 • 23 天前

Just a little bit faster than everyone else on OpenAI GPT-OSS-120B. > 10 times faster than the leading NVIDIA GPU provider

Just a little bit faster than everyone else on OpenAI GPT-OSS-120B. > 10 times faster than the leading NVIDIA GPU provider

Andrew Feldman

124,175 次观看 • 10 个月前

IDF paving the way for Gaza city take over

IDF paving the way for Gaza city take over

Open Source Intel

28,573 次观看 • 10 个月前

CPU vs GPU - very good and intuitive explanation of CPU vs GPU

CPU vs GPU - very good and intuitive explanation of CPU vs GPU

Mohit Mishra

279,324 次观看 • 1 年前

CPU vs GPU - very good and intuitive explanation of CPU vs GPU

CPU vs GPU - very good and intuitive explanation of CPU vs GPU

Mohit Mishra

120,277 次观看 • 1 年前

The AI era needs a new CPU. Meet NVIDIA Vera, 80% faster agentic task completion than x86. Built for AI factories. Built for what's next. The CPU for agents has arrived. 🔗

The AI era needs a new CPU. Meet NVIDIA Vera, 80% faster agentic task completion than x86. Built for AI factories. Built for what's next. The CPU for agents has arrived. 🔗

NVIDIA

101,166 次观看 • 25 天前

Windows ML is now generally available! This milestone brings powerful, local AI capability to every Windows 11 device, harnessing the full power of the GPU, NPU, and CPU. Big thank you to the team and our silicon partners, AMD, Intel, NVIDIA, and Qualcomm, for their collaboration to unlock real-time, secure, and efficient AI experiences for developers on Windows. Let’s go 🚀!

Windows ML is now generally available! This milestone brings powerful, local AI capability to every Windows 11 device, harnessing the full power of the GPU, NPU, and CPU. Big thank you to the team and our silicon partners, AMD, Intel, NVIDIA, and Qualcomm, for their collaboration to unlock real-time, secure, and efficient AI experiences for developers on Windows. Let’s go 🚀!

Pavan Davuluri

11,663 次观看 • 9 个月前

Ubuntu Now Has Higher System Requirements Than Windows As of this month, Ubuntu Linux requires 2X the CPU speed and 50% more RAM than Windows 11.

Ubuntu Now Has Higher System Requirements Than Windows As of this month, Ubuntu Linux requires 2X the CPU speed and 50% more RAM than Windows 11.

The Lunduke Journal

50,202 次观看 • 2 个月前

He's clearly faster than any robot at Paving 📹 : sams_blockpaving

He's clearly faster than any robot at Paving 📹 : sams_blockpaving

Arcfunmi

240,886 次观看 • 1 年前

.Cerebras is pretty fast at Llama 4... 3.5X faster than the next nearest competitor and 20X faster than NVIDIA GPU based solutions.

.Cerebras is pretty fast at Llama 4... 3.5X faster than the next nearest competitor and 20X faster than NVIDIA GPU based solutions.

Andrew Feldman

89,186 次观看 • 1 年前

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,583 次观看 • 2 年前

Introducing Context – The AI Office Suite. Imagine all your files, perfectly organized in one place, directly connected to AI. Search, create, and work faster than ever.

Introducing Context – The AI Office Suite. Imagine all your files, perfectly organized in one place, directly connected to AI. Search, create, and work faster than ever.

Siya 🍯

34,609 次观看 • 11 个月前

The Best Windows is Linux XLibre + XCE + Chicago95 + Wine + Steam. Faster, lighter, and better compatibility with Windows software than Windows 11.

The Best Windows is Linux XLibre + XCE + Chicago95 + Wine + Steam. Faster, lighter, and better compatibility with Windows software than Windows 11.

The Lunduke Journal

13,334 次观看 • 1 个月前

🚀Create complex, modular 3D assets and scenes.🚀 Introducing #MoCA (Mixture-of-Components Attention), a new sparse attention method for compositional 3D generation. It scales to *many * components. paper: code:

🚀Create complex, modular 3D assets and scenes.🚀 Introducing #MoCA (Mixture-of-Components Attention), a new sparse attention method for compositional 3D generation. It scales to many components. paper: code:

Tengfei Wang

27,096 次观看 • 6 个月前

Introducing Skywork 2.0: Infinite Creativity and Infinite Context We have launched Skywork Desktop (Windows version) and replaced the credit system with unlimited usage across Skywork agents. Now, creating and working with Skywork no longer consumes credits.

Introducing Skywork 2.0: Infinite Creativity and Infinite Context We have launched Skywork Desktop (Windows version) and replaced the credit system with unlimited usage across Skywork agents. Now, creating and working with Skywork no longer consumes credits.

Skywork

289,503 次观看 • 4 个月前

Introducing MiniCPM 4.1-8B: First Open-Source Reasoning LLM with Trainable Sparse Attention ✅ Strong Reasoning Capability: Surpasses similar-sized models on 15 tasks! ✅ Fast Generation: 3x decoding speedup for reasoning ✅ Efficient Architecture: Trainable sparse attention, frequency-ranked speculative decoding Download Models: Huggingface: Github: Technical Report: #AI #MiniCPM #LLM #OpenBMB #ArtificialIntelligence #MachineLearning

Introducing MiniCPM 4.1-8B: First Open-Source Reasoning LLM with Trainable Sparse Attention ✅ Strong Reasoning Capability: Surpasses similar-sized models on 15 tasks! ✅ Fast Generation: 3x decoding speedup for reasoning ✅ Efficient Architecture: Trainable sparse attention, frequency-ranked speculative decoding Download Models: Huggingface: Github: Technical Report: #AI #MiniCPM #LLM #OpenBMB #ArtificialIntelligence #MachineLearning

OpenBMB

19,236 次观看 • 9 个月前

CPU vs GPU: Detailed Explanation

CPU vs GPU: Detailed Explanation

Mohit Mishra

92,392 次观看 • 1 年前