Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

$INFQ GPU + CPU + QPU Frontier models are hitting saturation on tokens (data volume) and context windows (memory). $INFQ CTO walks through how they and $NVDA are working together to add a Quantum Processing Unit to the chain as the next inflection point.

57,181 görüntüleme • 2 ay önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

I had to test it myself to believe this unreal inference speed. 3,000 tokens/s for 1 user on standard datacenter GPUs. They leveraged a hidden efficiency gap in how GPUs generate tokens. Kog just achieved 3,000 tokens/s on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 (FP16, no speculative decoding). Their tech preview is on a 2B model, and they show how their techniques will scale to large frontier MoE models at similar speeds. That's a huge number because normal low-batch GPU decoding for 2B to 8B models is usually closer to 100 to 300 tokens/s per request, so Kog is claiming something like a 10X to 30X jump in the speed one user actually feels. Their trick: they are getting the speed by treating LLM decoding as a memory streaming problem, not mainly a math problem. For 1 user at batch size 1, the GPU is not doing big, efficient matrix-matrix work like in training or large-batch serving; it is repeatedly pulling the model’s active weights from high-bandwidth memory for each new token, so speed depends on how smoothly those weights keep flowing. Normal inference stacks keep breaking that flow. They run many separate GPU programs for different parts of the model, move intermediate results through memory, wait at synchronization points, talk back to the CPU for scheduling or sampling, and then repeat this token after token. Kog’s answer is to co-design 3 things that are usually tuned separately: the runtime, the low-level GPU code, and the model architecture. The biggest engineering move is the monokernel, where the whole decode pass runs as 1 persistent GPU-resident program, including sampling, so the system does not keep stopping for kernel launches, CPU scheduling, and intermediate memory round trips. They also rebuilt synchronization, because their own measurements say grid sync was eating around 35% of token-generation time; instead of making every compute unit wait at a broad barrier, each unit waits only for the exact data it needs. On AMD MI300X, they also map memory access around the chiplet layout, because memory latency changes depending on which die makes the request. Then their Laneformer model uses Delayed Tensor Parallelism, which lets cross-GPU communication happen in the background instead of blocking every layer.

Rohan Paul

13,148 görüntüleme • 1 ay önce

My 10x stock idea from GTC isn't photonics?! But it does involve lasers. Say hello to $INFQ. It's a newly IPO'd quantum stock generating tens of millions in revenue in space + defense applications with very unique technology. Infleqtion went public last month but it's trading 40% below its IPO price with a sub $2B market cap. $INFQ trades at roughly 70x trailing sales on $29M in revenue. Compare that to $RGTI at $6B market cap on just $7M in revenue, that's 860x sales. I chatted with $INFQ's Chief Administrative Officer, Julie McGee, to dig in further but here's the TLDR. Most quantum companies need to cool their chips to near absolute zero temps just to operate. Infleqtion uses "neutral atom" technology that traps individual atoms inside a glass cell using lasers and runs them at room temperature. It takes the power of a few hairdryers. No giant refrigerators. Way cheaper and way easier to scale. And unlike most quantum names they're actually shipping products NOW. Quantum clocks for GPS-denied navigation, RF sensors, inertial navigation systems. Selling to NASA, the DoD, and the UK government. Their quantum clock is being qualified by SpaceX for satellite systems. Quantum brings a whole new level of precision that works on the ground, in the sky, and underwater. Their technology can enable submarines to navigate without ever linking up to a satellite. GPS jamming is also becoming a huge problem on the battlefield, showing up in Ukraine and Iran. Quantum timing is inherently unjammable and unspoofable. They also had a dedicated spot inside the Nvidia booth at GTC. $INFQ partnered with Nvidia to demo the first commercial materials science application running on logical qubits and are working with Nvidia's NVQLink to scale quantum-classical hybrid computing. What's next: 30 logical qubits targeted this year, one of the most important milestones in the race to fault tolerant quantum computing by 2028. Plus a new NASA contract to measure Earth's gravity from space. Infleqtion combines an attractive valuation with extremely unique technology (they're the only neutral atom quantum company publicly listed). Could easily see this re-rating fast, I just think the IPO timing was poor with Iran. Could be adding this as a lottery ticket to my Asymmetrical Bets portfolio soon... This post was not sponsored or influenced in any way by $INFQ. All thoughts are my own, NFA / DYOR.

Michael Sikand

321,848 görüntüleme • 3 ay önce