Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

🌸How was the Cyrene Sakura GPU made? Appreciate for @衵曰文STUDIO doing this great work! . #yeston #graphicscard #gpu #hardware #pcgaming #pcbuild #sakura #sakuragpu #cyrene #HardwareDIY

YestonOfficial's profile picture

YESTON盈通

13,990 subscribers

32,561 görüntüleme • 7 ay önce •via X (Twitter)

#yeston#graphicscard#gpu#hardware#pcgaming#pcbuild#sakura#sakuragpu#cyrene#HardwareDIY
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

🌸: i’ve seen her for so long, so seeing this chaewon is so new and fascinating. cw with leadership is so cool. when i first saw her, she was a baby, but she’s grown this much. watching her from the side made me think ‘she’s cool’ 🐯: you're like my mom sakura praising chaewon🥺
0:26

🌸: i’ve seen her for so long, so seeing this chaewon is so new and fascinating. cw with leadership is so cool. when i first saw her, she was a baby, but she’s grown this much. watching her from the side made me think ‘she’s cool’ 🐯: you're like my mom sakura praising chaewon🥺

지오

81,812 görüntüleme • 1 yıl önce

Early on, Jonathan Ross foresaw the unprecedented demand for compute of the 2020s. After pioneering Google's TPUs, he founded Groq, a "Language" Processing Unit dedicated for inference. After joining NVIDIA, he is betting on the GPU+LPU combo to power the agentic chains of AI calling AI calling AI. This interview is part of my larger series on dedicated AI chips, so drop any suggestions/questions below. Thank you NVIDIA for hosting us at your HQ! 00:00 Intro 01:00 The Google TPU & Groq origin story 02:41 How is Groq different from a GPU? 04:43 Static scheduling makes Groq faster 05:47 Does Groq work with Mixture-of-Experts? 09:27 Are LPUs limited to text models? 11:03 Diffusion models 13:41 NVIDIA Vera Rubin: the GPU+LPU combo 15:26 Will Groq still be sold as a standalone chip? 16:49 How does agentic AI impact inference economics? 19:21 Will AI replace CUDA kernel engineers? 21:08 Will AI democratize hardware design? 26:11 Jevon's paradox: An endless demand for compute 29:04 What should kids learn in the AI age?
30:38

Early on, Jonathan Ross foresaw the unprecedented demand for compute of the 2020s. After pioneering Google's TPUs, he founded Groq, a "Language" Processing Unit dedicated for inference. After joining NVIDIA, he is betting on the GPU+LPU combo to power the agentic chains of AI calling AI calling AI. This interview is part of my larger series on dedicated AI chips, so drop any suggestions/questions below. Thank you NVIDIA for hosting us at your HQ! 00:00 Intro 01:00 The Google TPU & Groq origin story 02:41 How is Groq different from a GPU? 04:43 Static scheduling makes Groq faster 05:47 Does Groq work with Mixture-of-Experts? 09:27 Are LPUs limited to text models? 11:03 Diffusion models 13:41 NVIDIA Vera Rubin: the GPU+LPU combo 15:26 Will Groq still be sold as a standalone chip? 16:49 How does agentic AI impact inference economics? 19:21 Will AI replace CUDA kernel engineers? 21:08 Will AI democratize hardware design? 26:11 Jevon's paradox: An endless demand for compute 29:04 What should kids learn in the AI age?

Julia Turc

46,967 görüntüleme • 2 ay önce

Robert Downey Jr. says he looked at Mark Ruffalo and they said to each other how lucky they are to be working for Marvel Studios. “And then we had this whole decade-and-a-half Marvel run, where we were just looking at each other like, ‘God, we’re really lucky. What are we doing? Who’s a wizard? Who’s coming from outer space?’ I’ll never forget seeing this great character indie actor who was always doing this great character work, so alive, so relatable, so human, take that step into a studio world which was totally different back then than it is now.
1:10

Robert Downey Jr. says he looked at Mark Ruffalo and they said to each other how lucky they are to be working for Marvel Studios. “And then we had this whole decade-and-a-half Marvel run, where we were just looking at each other like, ‘God, we’re really lucky. What are we doing? Who’s a wizard? Who’s coming from outer space?’ I’ll never forget seeing this great character indie actor who was always doing this great character work, so alive, so relatable, so human, take that step into a studio world which was totally different back then than it is now.

Tokyo

551,895 görüntüleme • 2 ay önce

GLM-5.2 running on Mac Studio Clusters or RTX GPU rigs isn't how Local AI will get mass adopted. For mass adoption we need to focus on two axes: 1. Intelligence/Speed Frontier. This is being tracked on local dot ai. Aggressive co-design is needed across the entire stack to push up the frontier specifically for local hardware. Better kernels, inference tricks like spec decoding, quantizing models, model routing. 2. Usability / UX. To use Local AI today, you need to install CLIs, select a model, a quantization, a harness that works best with that model, and you need to configure your inference engine. It's not very accessible. More work needs to be done on making this seamless, so the user doesn't even know the AI is running locally.
1:52

GLM-5.2 running on Mac Studio Clusters or RTX GPU rigs isn't how Local AI will get mass adopted. For mass adoption we need to focus on two axes: 1. Intelligence/Speed Frontier. This is being tracked on local dot ai. Aggressive co-design is needed across the entire stack to push up the frontier specifically for local hardware. Better kernels, inference tricks like spec decoding, quantizing models, model routing. 2. Usability / UX. To use Local AI today, you need to install CLIs, select a model, a quantization, a harness that works best with that model, and you need to configure your inference engine. It's not very accessible. More work needs to be done on making this seamless, so the user doesn't even know the AI is running locally.

Alex Cheema

30,820 görüntüleme • 1 ay önce

Aircraft maintenance.. this is my tribute to all my brothers and sisters across the world who work so hard to make it airworthy. You are all doing Gods work and I see you and I appreciate you. This was inspired by Paul Harvey “God made a farmer”. But the saying goes true for all that walk this earth, god made us all for a reason and God wants you to be the best at it. I don’t mean to become melancholy or emotional, but I have a profound love for what I do, and I will always do my best to let the world know how important this job is to me and to my fellow mechanics. 📸 by Stig Aviation
1:52

Aircraft maintenance.. this is my tribute to all my brothers and sisters across the world who work so hard to make it airworthy. You are all doing Gods work and I see you and I appreciate you. This was inspired by Paul Harvey “God made a farmer”. But the saying goes true for all that walk this earth, god made us all for a reason and God wants you to be the best at it. I don’t mean to become melancholy or emotional, but I have a profound love for what I do, and I will always do my best to let the world know how important this job is to me and to my fellow mechanics. 📸 by Stig Aviation

aircraftmaintenancengineer

37,126 görüntüleme • 1 yıl önce

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.
0:42

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.

Akshay 🚀

336,634 görüntüleme • 11 gün önce

251126 | INFO Gen Hoshino mentioned #SAKURA at Gen Hoshino ANN 🧡: Ah, right, and also—just now I mentioned it a little, about KAWAII. When LE SSERAFIM came to this program, I think we talked about MC-ing, right? 🧡: And when the topic came up about, like, how to hype up the audience, I said something like, well, I do say things like “dance!”, but I also kind of mutter like speaking to the person right in front of me “let’s dance” or “dance with me.” 🧡: And I said that in really big venues, that kind of line somehow resonates more, or like, it feels like it reaches people better. 🧡: And then, right before the song KAWAII, Miyawaki Sakura said “let’s dance together,” so I was like, “oh thank you…” 🧡: Yeah, and they came over to greet me afterwards, but I had work so I couldn’t go say hello. 🧡: But that whole thing made me feel that Miyawaki-san was aware that I was there watching, and that made me really happy. 🧡: So yeah, that was the story. #사쿠라 #SAKURA #ックラ #宮脇咲良 #르세라핌 #LE_SSERAFIM
1:20

251126 | INFO Gen Hoshino mentioned #SAKURA at Gen Hoshino ANN 🧡: Ah, right, and also—just now I mentioned it a little, about KAWAII. When LE SSERAFIM came to this program, I think we talked about MC-ing, right? 🧡: And when the topic came up about, like, how to hype up the audience, I said something like, well, I do say things like “dance!”, but I also kind of mutter like speaking to the person right in front of me “let’s dance” or “dance with me.” 🧡: And I said that in really big venues, that kind of line somehow resonates more, or like, it feels like it reaches people better. 🧡: And then, right before the song KAWAII, Miyawaki Sakura said “let’s dance together,” so I was like, “oh thank you…” 🧡: Yeah, and they came over to greet me afterwards, but I had work so I couldn’t go say hello. 🧡: But that whole thing made me feel that Miyawaki-san was aware that I was there watching, and that made me really happy. 🧡: So yeah, that was the story. #사쿠라 #SAKURA #ックラ #宮脇咲良 #르세라핌 #LE_SSERAFIM

SAKURA ISLAND

15,542 görüntüleme • 9 ay önce

I had to test it myself to believe this unreal inference speed. 3,000 tokens/s for 1 user on standard datacenter GPUs. They leveraged a hidden efficiency gap in how GPUs generate tokens. Kog just achieved 3,000 tokens/s on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 (FP16, no speculative decoding). Their tech preview is on a 2B model, and they show how their techniques will scale to large frontier MoE models at similar speeds. That's a huge number because normal low-batch GPU decoding for 2B to 8B models is usually closer to 100 to 300 tokens/s per request, so Kog is claiming something like a 10X to 30X jump in the speed one user actually feels. Their trick: they are getting the speed by treating LLM decoding as a memory streaming problem, not mainly a math problem. For 1 user at batch size 1, the GPU is not doing big, efficient matrix-matrix work like in training or large-batch serving; it is repeatedly pulling the model’s active weights from high-bandwidth memory for each new token, so speed depends on how smoothly those weights keep flowing. Normal inference stacks keep breaking that flow. They run many separate GPU programs for different parts of the model, move intermediate results through memory, wait at synchronization points, talk back to the CPU for scheduling or sampling, and then repeat this token after token. Kog’s answer is to co-design 3 things that are usually tuned separately: the runtime, the low-level GPU code, and the model architecture. The biggest engineering move is the monokernel, where the whole decode pass runs as 1 persistent GPU-resident program, including sampling, so the system does not keep stopping for kernel launches, CPU scheduling, and intermediate memory round trips. They also rebuilt synchronization, because their own measurements say grid sync was eating around 35% of token-generation time; instead of making every compute unit wait at a broad barrier, each unit waits only for the exact data it needs. On AMD MI300X, they also map memory access around the chiplet layout, because memory latency changes depending on which die makes the request. Then their Laneformer model uses Delayed Tensor Parallelism, which lets cross-GPU communication happen in the background instead of blocking every layer.
1:08

I had to test it myself to believe this unreal inference speed. 3,000 tokens/s for 1 user on standard datacenter GPUs. They leveraged a hidden efficiency gap in how GPUs generate tokens. Kog just achieved 3,000 tokens/s on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 (FP16, no speculative decoding). Their tech preview is on a 2B model, and they show how their techniques will scale to large frontier MoE models at similar speeds. That's a huge number because normal low-batch GPU decoding for 2B to 8B models is usually closer to 100 to 300 tokens/s per request, so Kog is claiming something like a 10X to 30X jump in the speed one user actually feels. Their trick: they are getting the speed by treating LLM decoding as a memory streaming problem, not mainly a math problem. For 1 user at batch size 1, the GPU is not doing big, efficient matrix-matrix work like in training or large-batch serving; it is repeatedly pulling the model’s active weights from high-bandwidth memory for each new token, so speed depends on how smoothly those weights keep flowing. Normal inference stacks keep breaking that flow. They run many separate GPU programs for different parts of the model, move intermediate results through memory, wait at synchronization points, talk back to the CPU for scheduling or sampling, and then repeat this token after token. Kog’s answer is to co-design 3 things that are usually tuned separately: the runtime, the low-level GPU code, and the model architecture. The biggest engineering move is the monokernel, where the whole decode pass runs as 1 persistent GPU-resident program, including sampling, so the system does not keep stopping for kernel launches, CPU scheduling, and intermediate memory round trips. They also rebuilt synchronization, because their own measurements say grid sync was eating around 35% of token-generation time; instead of making every compute unit wait at a broad barrier, each unit waits only for the exact data it needs. On AMD MI300X, they also map memory access around the chiplet layout, because memory latency changes depending on which die makes the request. Then their Laneformer model uses Delayed Tensor Parallelism, which lets cross-GPU communication happen in the background instead of blocking every layer.

Rohan Paul

13,244 görüntüleme • 3 ay önce

This was made by a single person without AI,made in Blender, and characters were made in Adobe Animate. there was only one panel present in the manga,so the entire thing was created by myself on how i want gojo hollow Purple to Form. You AI SLoppy bros can write HOLLYWOOD COOK THIS,ANIME STUDIO COOKED THAT,but all you are doing is buttering your food with your own shit and calling the food you are making tasty. Only thing you can create better than others is illusion and only for your sloppy groups.
0:30

This was made by a single person without AI,made in Blender, and characters were made in Adobe Animate. there was only one panel present in the manga,so the entire thing was created by myself on how i want gojo hollow Purple to Form. You AI SLoppy bros can write HOLLYWOOD COOK THIS,ANIME STUDIO COOKED THAT,but all you are doing is buttering your food with your own shit and calling the food you are making tasty. Only thing you can create better than others is illusion and only for your sloppy groups.

Alan Bowe

123,367 görüntüleme • 6 ay önce

🚨 I DON'T KNOW WHAT PEOPLE ARE WAITING FOR! This guy just deployed a full GPU compute farm in a data center Not renting cloud instances, not paying per-token, just racks of consumer GPUs on warehouse shelving doing AI inference 24/7 The setup is wild: open-frame rigs with rows of white fans, PCIe riser cables, server-grade PSUs, and a PDU for power distribution. All mounted on red industrial shelving with raised-floor cooling underneath. It looks like a crypto mining op, because it probably was one Here's what's actually happening: > Ex-mining rigs are being redeployed for AI inference at scale > Each rig runs 6-8 GPUs on PCIe risers connected to a single motherboard > Fans pull cold air from the raised floor through the open frames > A PDU on each rack handles power distribution across all rigs > The whole thing runs in a colocation facility with cheap power rates The economics make sense: cloud GPU pricing runs $2-4/hr per card. Own the hardware, colocate it for $0.08-0.12/kWh, and you're printing compute at a fraction of the cost The crypto-to-AI pipeline is real. Same hardware, same facilities, same power contracts, completely different revenue model Bookmark this!
0:53

🚨 I DON'T KNOW WHAT PEOPLE ARE WAITING FOR! This guy just deployed a full GPU compute farm in a data center Not renting cloud instances, not paying per-token, just racks of consumer GPUs on warehouse shelving doing AI inference 24/7 The setup is wild: open-frame rigs with rows of white fans, PCIe riser cables, server-grade PSUs, and a PDU for power distribution. All mounted on red industrial shelving with raised-floor cooling underneath. It looks like a crypto mining op, because it probably was one Here's what's actually happening: > Ex-mining rigs are being redeployed for AI inference at scale > Each rig runs 6-8 GPUs on PCIe risers connected to a single motherboard > Fans pull cold air from the raised floor through the open frames > A PDU on each rack handles power distribution across all rigs > The whole thing runs in a colocation facility with cheap power rates The economics make sense: cloud GPU pricing runs $2-4/hr per card. Own the hardware, colocate it for $0.08-0.12/kWh, and you're printing compute at a fraction of the cost The crypto-to-AI pipeline is real. Same hardware, same facilities, same power contracts, completely different revenue model Bookmark this!

cristal

16,305 görüntüleme • 2 ay önce

Sequoia’s Shaun Maguire wrote a private hardware manifesto arguing that over the next 25 years, most of the money will be made in hardware: "Every software revolution is preceded by a hardware revolution." "To have the iOS App Store that enabled Uber, DoorDash, and all of these great companies - you needed to have the iPhone." "This AI revolution - we're seeing what it can do from the software layer, but it's still limited by hardware." "The hardware we were doing for a long time was all following Moore's Law. It was all branching out of this decision in the mid-1950s to go all in on the silicon supply chain." "That has created magic, and there's still a couple orders of magnitude of juice to squeeze, but we’re hitting fundamental physics limits - Dennard scaling, things like that." "I think this tech tree is branching into humanoid robots, into silicon photonics, into orbital data centers - all of these new hardware areas where there's going to be 20+ years of progress." "There's going to be incredible businesses built on the back of this. And a lot of dumpster fires."
3:35

Sequoia’s Shaun Maguire wrote a private hardware manifesto arguing that over the next 25 years, most of the money will be made in hardware: "Every software revolution is preceded by a hardware revolution." "To have the iOS App Store that enabled Uber, DoorDash, and all of these great companies - you needed to have the iPhone." "This AI revolution - we're seeing what it can do from the software layer, but it's still limited by hardware." "The hardware we were doing for a long time was all following Moore's Law. It was all branching out of this decision in the mid-1950s to go all in on the silicon supply chain." "That has created magic, and there's still a couple orders of magnitude of juice to squeeze, but we’re hitting fundamental physics limits - Dennard scaling, things like that." "I think this tech tree is branching into humanoid robots, into silicon photonics, into orbital data centers - all of these new hardware areas where there's going to be 20+ years of progress." "There's going to be incredible businesses built on the back of this. And a lot of dumpster fires."

TBPN

218,127 görüntüleme • 5 ay önce

i built a full game on a single GPU with a 3B model and this is the worst local AI will ever be. this was supposed to be a benchmark test. load the model, measure tokens per second, write it up, move on. instead i spent 20 minutes playing Octopus Invaders because the game is genuinely fun and i couldn't stop. a model with 3B active parameters built this from a single prompt. it debugged its own collision system when bullets were phasing through enemies. read the error, found the fix, kept building. this is not a frontier API. this is a quantized open source model running on hardware you can buy used for $800-$1200. no cloud. no subscription. no API costs. just a mass produced consumer GPU doing things that would have been absurd 12 months ago. and here's the part that should keep you up at night: every month the models get smaller and smarter. the quants get tighter. the context windows get longer. the tooling gets cleaner. what 3B active parameters does today on 24gb, a 1B model will do on 8gb within a year. you are looking at the floor. not the ceiling.
0:27

i built a full game on a single GPU with a 3B model and this is the worst local AI will ever be. this was supposed to be a benchmark test. load the model, measure tokens per second, write it up, move on. instead i spent 20 minutes playing Octopus Invaders because the game is genuinely fun and i couldn't stop. a model with 3B active parameters built this from a single prompt. it debugged its own collision system when bullets were phasing through enemies. read the error, found the fix, kept building. this is not a frontier API. this is a quantized open source model running on hardware you can buy used for $800-$1200. no cloud. no subscription. no API costs. just a mass produced consumer GPU doing things that would have been absurd 12 months ago. and here's the part that should keep you up at night: every month the models get smaller and smarter. the quants get tighter. the context windows get longer. the tooling gets cleaner. what 3B active parameters does today on 24gb, a 1B model will do on 8gb within a year. you are looking at the floor. not the ceiling.

Sudo su

36,251 görüntüleme • 6 ay önce

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.
3:14

SITUATION EXPLAINED: Cerebras raised $5.55 billion in their IPO and closing their first day of trading valued at $66 billion, making it the biggest US tech IPO since Snowflake in 2020. Cerebras makes Wafer-Scale Engine chips built for AI inference. We asked Sarah Fong the main difference between wafer-scale chips and traditional GPUs: - GPUs are great at parallel work (graphics, training) - AI inference is sequential, AKA one token at a time This causes the "memory wall" problem: - Every GPU core needs model weights, KV cache, and activations to do its math - On a GPU, that data lives in off-chip memory (HBM) - Cores constantly load and offload from off-chip memory, which is a huge bottleneck; hardware accounts for ~70% of inference latency Cerebras' chips: -Dinner-plate sized (vs. GPUs which are palm-sized) with tens of thousands of cores -Memory sits directly on top of the cores as distributed SRAM -Weights and KV cache can be accessed at on-chip speeds in the PB/s range, compared with off-chip speeds in the TB/s range achieved by GPUs with HBM.

MTS

44,057 görüntüleme • 3 ay önce

Fox & Friends Audience Member Ken: “When you talk about socialism, I go back to the Margaret Thatcher line. Socialism is a wonderful system until you run out of other people’s money... I think these kids grew up getting free stuff and now they’re promised more free stuff and that’s why they’re voting.” “They need to learn what made this country great and how it was hard work but you get rewarded for hard work.”
0:42

Fox & Friends Audience Member Ken: “When you talk about socialism, I go back to the Margaret Thatcher line. Socialism is a wonderful system until you run out of other people’s money... I think these kids grew up getting free stuff and now they’re promised more free stuff and that’s why they’re voting.” “They need to learn what made this country great and how it was hard work but you get rewarded for hard work.”

RedWave Press

83,425 görüntüleme • 2 ay önce