Загрузка видео...

Не удалось загрузить видео

На главную

x86 segmentation maps memory references through segment registers. It is still used in modern x86-64 for FS/GS, especially thread-local storage and kernel per-CPU data. In sandboxing, segmentation can restrict untrusted code to a fixed memory region. MIT 6.858.

tetsuoai's profile picture

tetsuo

236,480 subscribers

17,558 просмотров • 3 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Most C Programmers don't have a clue as to how the CPU works. Here is a lecture from my course on the C Language that introduces how to visualise the CPU as a C (or general software) Programmer.... More Details are at: Being able to model the behaviour of CPU provides a solid base to reason about your code. The summary is - - The CPU is a state machine which maintains it's states in registers. - It has busses to memory for requesting data/instruction read/writes. - Interrupts to attend to events. I discuss this and more in my course on the C Language.
11:41

Most C Programmers don't have a clue as to how the CPU works. Here is a lecture from my course on the C Language that introduces how to visualise the CPU as a C (or general software) Programmer.... More Details are at: Being able to model the behaviour of CPU provides a solid base to reason about your code. The summary is - - The CPU is a state machine which maintains it's states in registers. - It has busses to memory for requesting data/instruction read/writes. - Interrupts to attend to events. I discuss this and more in my course on the C Language.

Piyush Itankar

28,019 просмотров • 1 год назад

Context memory essentially unlocks Agentic AI Much needed for Opus 4.6's "multi-agent swarms" In this SemiDoped pod, Vikram Sekar talks to Val Bercovici from Weka about context storage. - How token warehouses save inference costs - A new networking tier? Context Storage Network! - High Bandwidth Flash for context? - Weka's Augmented Memory Grid for context storage - Where this is all headed The convo is info packed. Don't miss out on it! b/acc, context platform engineer Chapters (00:00) Introduction to Weka and AI Storage Solutions (05:18) The Evolution of Context Memory in AI (09:30) Understanding Memory Hierarchies and Their Impact (16:24) Latency Challenges in Modern Storage Solutions (21:32) The Role of Networking in AI Storage Efficiency (29:42) Dynamic Resource Utilization in AI Networks (30:04) Introducing the Context Memory Network (31:13) High Bandwidth Flash: A Game Changer (32:54) Weka’s Neural Mesh and Storage Solutions (35:01) Axon: Transforming GPU Storage into Memory (39:00) Augmented Memory Grid Explained (42:00) Pooling DRAM and CXL Innovations (46:02) Token Warehouses and Inference Economics (52:10) The Future of Storage Innovations
54:27

Context memory essentially unlocks Agentic AI Much needed for Opus 4.6's "multi-agent swarms" In this SemiDoped pod, Vikram Sekar talks to Val Bercovici from Weka about context storage. - How token warehouses save inference costs - A new networking tier? Context Storage Network! - High Bandwidth Flash for context? - Weka's Augmented Memory Grid for context storage - Where this is all headed The convo is info packed. Don't miss out on it! b/acc, context platform engineer Chapters (00:00) Introduction to Weka and AI Storage Solutions (05:18) The Evolution of Context Memory in AI (09:30) Understanding Memory Hierarchies and Their Impact (16:24) Latency Challenges in Modern Storage Solutions (21:32) The Role of Networking in AI Storage Efficiency (29:42) Dynamic Resource Utilization in AI Networks (30:04) Introducing the Context Memory Network (31:13) High Bandwidth Flash: A Game Changer (32:54) Weka’s Neural Mesh and Storage Solutions (35:01) Axon: Transforming GPU Storage into Memory (39:00) Augmented Memory Grid Explained (42:00) Pooling DRAM and CXL Innovations (46:02) Token Warehouses and Inference Economics (52:10) The Future of Storage Innovations

Semi Doped

12,796 просмотров • 6 месяцев назад

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch
0:22

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch

Bo Wang

178,579 просмотров • 2 лет назад

Jensen Huang just identified the next $200 billion market (Save this). The shift starts with a observation about agentic AI that changes everything about infrastructure. In the era of training and inference, the GPU was everything while CPU was a traffic cop, scheduling work, managing memory, dispatching tasks while the GPU did the heavy lifting. Agentic AI breaks that model entirely. An AI agent does not just run a single inference pass but rather it plans, calls tools, executes code in sandboxes, retrieves data from multiple sources and loops through complex multi-step reasoning sequences often thousands of times per second at scale. Every one of those operations runs through the CPU and the GPU sits idle waiting for the CPU to prepare the next task, supply the right context and execute the retrieval and tool calling logic fast enough to keep the accelerators fed. The CPU is now the conductor and the GPU is the orchestra and the bottleneck is the conductor falling behind. This is showing up in production AI factory utilization right now, which is exactly why Jensen built Vera from scratch rather than licensing x86. Vera achieves 40% lower peak memory latency than x86, 50% faster core to core communication, and 1.8 times the agentic sandbox performance of current x86 processors on a purpose-built architecture designed around the agentic loop. Now here is where the investment thesis gets interesting. The obvious beneficiary is Nvidia itself, and that thesis is real. Nvidia's CFO has guided for nearly $20 billion in Vera CPU revenue this fiscal year alone, a market Nvidia had zero presence in just three years ago. Intel held 60% of server CPU market share as recently as Q4 2025 and that transition is now happening at a pace Intel structurally cannot respond to. But the deeper question is, what architecture is Vera actually built on? Vera's Olympus cores are ARM compatible and every single Vera CPU deployed in every Vera Rubin rack in every data center in the world runs on ARM architecture. And ARM Holdings collects a royalty on every one of them. ARM does not make chips but rather licenses the instruction set architecture and CPU core designs that others build on top of. Every time Nvidia ships a Vera CPU, every time a hyperscaler deploys a Vera Rubin rack, every time an enterprise qualifies Vera for their AI factory, ARM earns a royalty. The secular tailwind here is almost perfectly constructed for ARM's business model. Amazon's Graviton, Microsoft's Cobalt, Google's Axion, Apple's silicon stack, and Qualcomm's data center push all run on ARM. And now Nvidia's Vera, which is projected to displace Intel as the largest server CPU supplier by revenue in a single fiscal year, is ARM. ARM's royalty rate on high end server chips is estimated at roughly 1 to 2% of chip selling price. At $5,000 per Vera CPU and 4 million units projected for FY2027, that is a royalty line growing from near zero to potentially $400 million to $800 million annually from Nvidia's data center CPU business alone before counting Amazon, Microsoft, Google, Apple, and Qualcomm. The total ARM addressable royalty base across all the silicon it already licenses is compounding at a rate that the current $130 billion market cap does not fully reflect. Jensen's CPU thesis is the most underappreciated catalyst in ARM's fundamental story, and the royalty compounding has barely started. Come join Milk Road Pro and get our full ARM royalty model and our entire AI trade thesis. Link below!
3:41

Jensen Huang just identified the next $200 billion market (Save this). The shift starts with a observation about agentic AI that changes everything about infrastructure. In the era of training and inference, the GPU was everything while CPU was a traffic cop, scheduling work, managing memory, dispatching tasks while the GPU did the heavy lifting. Agentic AI breaks that model entirely. An AI agent does not just run a single inference pass but rather it plans, calls tools, executes code in sandboxes, retrieves data from multiple sources and loops through complex multi-step reasoning sequences often thousands of times per second at scale. Every one of those operations runs through the CPU and the GPU sits idle waiting for the CPU to prepare the next task, supply the right context and execute the retrieval and tool calling logic fast enough to keep the accelerators fed. The CPU is now the conductor and the GPU is the orchestra and the bottleneck is the conductor falling behind. This is showing up in production AI factory utilization right now, which is exactly why Jensen built Vera from scratch rather than licensing x86. Vera achieves 40% lower peak memory latency than x86, 50% faster core to core communication, and 1.8 times the agentic sandbox performance of current x86 processors on a purpose-built architecture designed around the agentic loop. Now here is where the investment thesis gets interesting. The obvious beneficiary is Nvidia itself, and that thesis is real. Nvidia's CFO has guided for nearly $20 billion in Vera CPU revenue this fiscal year alone, a market Nvidia had zero presence in just three years ago. Intel held 60% of server CPU market share as recently as Q4 2025 and that transition is now happening at a pace Intel structurally cannot respond to. But the deeper question is, what architecture is Vera actually built on? Vera's Olympus cores are ARM compatible and every single Vera CPU deployed in every Vera Rubin rack in every data center in the world runs on ARM architecture. And ARM Holdings collects a royalty on every one of them. ARM does not make chips but rather licenses the instruction set architecture and CPU core designs that others build on top of. Every time Nvidia ships a Vera CPU, every time a hyperscaler deploys a Vera Rubin rack, every time an enterprise qualifies Vera for their AI factory, ARM earns a royalty. The secular tailwind here is almost perfectly constructed for ARM's business model. Amazon's Graviton, Microsoft's Cobalt, Google's Axion, Apple's silicon stack, and Qualcomm's data center push all run on ARM. And now Nvidia's Vera, which is projected to displace Intel as the largest server CPU supplier by revenue in a single fiscal year, is ARM. ARM's royalty rate on high end server chips is estimated at roughly 1 to 2% of chip selling price. At $5,000 per Vera CPU and 4 million units projected for FY2027, that is a royalty line growing from near zero to potentially $400 million to $800 million annually from Nvidia's data center CPU business alone before counting Amazon, Microsoft, Google, Apple, and Qualcomm. The total ARM addressable royalty base across all the silicon it already licenses is compounding at a rate that the current $130 billion market cap does not fully reflect. Jensen's CPU thesis is the most underappreciated catalyst in ARM's fundamental story, and the royalty compounding has barely started. Come join Milk Road Pro and get our full ARM royalty model and our entire AI trade thesis. Link below!

Milk Road AI

11,819 просмотров • 2 месяцев назад

What Lisa Su actually held on stage: A mini PC the size of a lunchbox running Qwen3-235B locally, with no cloud and no discrete GPU Inside: the Ryzen AI Max+ 395, 128GB unified memory, 110GB usable as VRAM on Linux The first x86 chip that handles 200+ billion parameters on a single die AMD claims it beats the RTX 5080 by several times on memory-bound models — because the 5080 simply cannot fit them $1,400 to $2,500 once. cloud bills run $200 to $400 a month It pays for itself in a few months, then costs nothing per request This is not a faster GPU. it is the first real argument that your AI does not belong in someone else's data center
0:31

What Lisa Su actually held on stage: A mini PC the size of a lunchbox running Qwen3-235B locally, with no cloud and no discrete GPU Inside: the Ryzen AI Max+ 395, 128GB unified memory, 110GB usable as VRAM on Linux The first x86 chip that handles 200+ billion parameters on a single die AMD claims it beats the RTX 5080 by several times on memory-bound models — because the 5080 simply cannot fit them $1,400 to $2,500 once. cloud bills run $200 to $400 a month It pays for itself in a few months, then costs nothing per request This is not a faster GPU. it is the first real argument that your AI does not belong in someone else's data center

plutos

1,237,247 просмотров • 2 месяцев назад

One of the unique features of the Neo Geo was its memory card support. Most games used it mainly for saving high scores, but some titles could even store stage progress. In Fatal Fury Special, after defeating Ryo Sakazaki at the end and saving to the memory card, you could reload the data and select Ryo Sakazaki in VS mode. Since the MVS and AES shared almost the same hardware architecture, the same memory card could be used on both systems. That was a really interesting feature. However, unless you had an MVS cabinet equipped with a memory card slot, you couldn’t use it, and because the Neo Geo AES itself was extremely expensive, most gamers didn’t even know this feature existed. The original memory cards used battery-backed saves, but the upcoming AES+ memory card uses ROM storage, so I’m really looking forward to it👌
0:35

One of the unique features of the Neo Geo was its memory card support. Most games used it mainly for saving high scores, but some titles could even store stage progress. In Fatal Fury Special, after defeating Ryo Sakazaki at the end and saving to the memory card, you could reload the data and select Ryo Sakazaki in VS mode. Since the MVS and AES shared almost the same hardware architecture, the same memory card could be used on both systems. That was a really interesting feature. However, unless you had an MVS cabinet equipped with a memory card slot, you couldn’t use it, and because the Neo Geo AES itself was extremely expensive, most gamers didn’t even know this feature existed. The original memory cards used battery-backed saves, but the upcoming AES+ memory card uses ROM storage, so I’m really looking forward to it👌

Gamer_lafan

72,588 просмотров • 3 месяцев назад

Nvidia announces the new RTX Spark, a new platform powered by the NX1 CPU, and shows off Spark laptops running 007 First Light and Forza 6. The CPU has 20 ARM based cores and a Blackwell RTX GPU with 6144 CUDA Cores. This is the same core count as a 5070, but with 128GB of unified LPDDR5X RAM memory sitting in the same package as the CPU and GPU. The entire Nvidia software stack is available, particularly CUDA, vital for AI. Nvidia's new laptops will likely be ideal for running local LLMs be cause the unified memory means you can load models up to 120-180B parameters (quantized). These laptops are expected to ship later this year and could become strong competitors to high-end MacBooks and even Mac Studios for local AI workloads, thanks to CUDA support and unified memory. Price is unannounced.
2:24

Nvidia announces the new RTX Spark, a new platform powered by the NX1 CPU, and shows off Spark laptops running 007 First Light and Forza 6. The CPU has 20 ARM based cores and a Blackwell RTX GPU with 6144 CUDA Cores. This is the same core count as a 5070, but with 128GB of unified LPDDR5X RAM memory sitting in the same package as the CPU and GPU. The entire Nvidia software stack is available, particularly CUDA, vital for AI. Nvidia's new laptops will likely be ideal for running local LLMs be cause the unified memory means you can load models up to 120-180B parameters (quantized). These laptops are expected to ship later this year and could become strong competitors to high-end MacBooks and even Mac Studios for local AI workloads, thanks to CUDA support and unified memory. Price is unannounced.

Grummz

30,573 просмотров • 2 месяцев назад

Google Ironwood TPU Memory Hierarchy in 9 levels by hand ✍️ 1. Bit – The most basic unit of information, the on–off decision from which every number, tensor, and model state is ultimately constructed. 2. FP8 (1×8 → 8 bits) – Eight bits are grouped to form a floating-point value, typically used for inference, where reduced precision is a deliberate trade-off to maximize throughput and efficiency. 3. BF16 (×2 → 16 bits) – Two FP8-scale chunks are combined to gain more dynamic range and stability, while still staying friendly to high-throughput hardware. 4. Tensor tile (×1024 → 1K) – Data moves through the chip in blocks of 1024 values at a time, defining the granularity at which tensors are fetched and manipulated. 5. Matrix Multiplication Unit (MXU) (×64 → 64K) – A systolic array where matrix multiplication is not abstract but physical, with tensor tiles flowing through fixed hardware to achieve the highest possible throughput. 6. Vector Memory (VMEM) (×2048 → 128M) – On-chip working memory that holds activations, partial results, and intermediates, sized specifically to keep the systolic array busy without stalling. 7. Common Memory (CMEM) (×8 → 1 GB) – A small but critical shared memory sitting between VMEM and HBM, used for staging, accumulation, synchronization, and cross-lane coordination. 8. HBM (×96 → 96 GB) – Off-chip high-bandwidth memory where model weights and large states live, implemented as HBM3e with 16 stacks at 6 GB each, for a total of 96 GB. 9. Dual-Die (x2 → 192GB) – Two tightly coupled compute dies operate as a single logical accelerator, each with its own local HBM, effectively doubling memory capacity and bandwidth while allowing tensors and activations to stream seamlessly across dies as if they lived on one chip. I created this drawing for this week's seminar. I’ll take you through these 9 levels in a beginner-friendly way by hand ✍️. RSVP 👉
0:27

Google Ironwood TPU Memory Hierarchy in 9 levels by hand ✍️ 1. Bit – The most basic unit of information, the on–off decision from which every number, tensor, and model state is ultimately constructed. 2. FP8 (1×8 → 8 bits) – Eight bits are grouped to form a floating-point value, typically used for inference, where reduced precision is a deliberate trade-off to maximize throughput and efficiency. 3. BF16 (×2 → 16 bits) – Two FP8-scale chunks are combined to gain more dynamic range and stability, while still staying friendly to high-throughput hardware. 4. Tensor tile (×1024 → 1K) – Data moves through the chip in blocks of 1024 values at a time, defining the granularity at which tensors are fetched and manipulated. 5. Matrix Multiplication Unit (MXU) (×64 → 64K) – A systolic array where matrix multiplication is not abstract but physical, with tensor tiles flowing through fixed hardware to achieve the highest possible throughput. 6. Vector Memory (VMEM) (×2048 → 128M) – On-chip working memory that holds activations, partial results, and intermediates, sized specifically to keep the systolic array busy without stalling. 7. Common Memory (CMEM) (×8 → 1 GB) – A small but critical shared memory sitting between VMEM and HBM, used for staging, accumulation, synchronization, and cross-lane coordination. 8. HBM (×96 → 96 GB) – Off-chip high-bandwidth memory where model weights and large states live, implemented as HBM3e with 16 stacks at 6 GB each, for a total of 96 GB. 9. Dual-Die (x2 → 192GB) – Two tightly coupled compute dies operate as a single logical accelerator, each with its own local HBM, effectively doubling memory capacity and bandwidth while allowing tensors and activations to stream seamlessly across dies as if they lived on one chip. I created this drawing for this week's seminar. I’ll take you through these 9 levels in a beginner-friendly way by hand ✍️. RSVP 👉

Tom Yeh

30,489 просмотров • 7 месяцев назад

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”
2:27

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Daniel Romero

38,220 просмотров • 1 месяц назад

The infamous hyperspeed bug in Peter Jackson's King Kong is now fixed! The game tracks time with CPU cycles, but casts the frequency to a 32 bit value. On modern high-clock CPUs that overflows, so the engine thinks your CPU is way slower than it actually is. It then crams way too many physics & gameplay events into each tick - and since the real CPU is still blazing fast, everything goes into insane overdrive. Now properly fixed for smooth gameplay on today's hardware. Download link:
0:22

The infamous hyperspeed bug in Peter Jackson's King Kong is now fixed! The game tracks time with CPU cycles, but casts the frequency to a 32 bit value. On modern high-clock CPUs that overflows, so the engine thinks your CPU is way slower than it actually is. It then crams way too many physics & gameplay events into each tick - and since the real CPU is still blazing fast, everything goes into insane overdrive. Now properly fixed for smooth gameplay on today's hardware. Download link:

Fusion Fix

72,290 просмотров • 5 месяцев назад

Why is Redis Fast? Redis is fast for in-memory data storage. Its speed has made it popular for caching, session storage, and real-time analytics. But what gives Redis its blazing speed? Let's explore: RAM-Based Storage At its core, Redis primarily uses main memory for storing data. Accessing data from RAM is orders of magnitude faster than from disk. This is a major reason for Redis's speed. However, RAM is volatile. To persist data, Redis supports disk snapshots and append-only file logging. This combines RAM's performance with disk's permanence. There is a tradeoff though - recovery from disk is slow. If a Redis instance fails, restarting from disk can be slow compared to failing over to a replica instance fully in memory. So while Redis offers durability via disk, it comes at the cost of slower recovery. A better solution is Redis replication. With a synchronized replica kept in memory, failover is instant with no rehydration. This maintains speed and near-instant recovery. IO Multiplexing & Single-threaded Read/Write Redis uses an event-driven, single-threaded model for its core operations. A main event loop handles all client requests and data operations sequentially. This single-threaded execution avoids context switching and synchronization overhead typical of multi-threaded systems. Redis uses non-blocking I/O to handle multiple connections asynchronously. This allows it to support many client connections with very low overhead, Redis does leverage threading in certain areas: - Background tasks like taking snapshots. - I/O threads are used for certain operations. - Modules can use threads. - Since Redis 6.0, it supports multi-threaded I/O for network communication, improving performance on multi-core systems. Redis also uses pipelining for high throughput. Clients pipeline commands without waiting for each response. This allows more efficient network round trips, boosting overall performance. Efficient Data Structures Redis supports various optimized data structures, from linked lists, zip lists, and skip lists to sets, hashes, and sorted sets, among others. Each is carefully designed for specific use cases for quick and efficient data access. Over to you: With Redis now supporting some multi-threading, how should we configure it to fully utilize all the CPU cores of modern hardware when deploying in production? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages):
1:17

Why is Redis Fast? Redis is fast for in-memory data storage. Its speed has made it popular for caching, session storage, and real-time analytics. But what gives Redis its blazing speed? Let's explore: RAM-Based Storage At its core, Redis primarily uses main memory for storing data. Accessing data from RAM is orders of magnitude faster than from disk. This is a major reason for Redis's speed. However, RAM is volatile. To persist data, Redis supports disk snapshots and append-only file logging. This combines RAM's performance with disk's permanence. There is a tradeoff though - recovery from disk is slow. If a Redis instance fails, restarting from disk can be slow compared to failing over to a replica instance fully in memory. So while Redis offers durability via disk, it comes at the cost of slower recovery. A better solution is Redis replication. With a synchronized replica kept in memory, failover is instant with no rehydration. This maintains speed and near-instant recovery. IO Multiplexing & Single-threaded Read/Write Redis uses an event-driven, single-threaded model for its core operations. A main event loop handles all client requests and data operations sequentially. This single-threaded execution avoids context switching and synchronization overhead typical of multi-threaded systems. Redis uses non-blocking I/O to handle multiple connections asynchronously. This allows it to support many client connections with very low overhead, Redis does leverage threading in certain areas: - Background tasks like taking snapshots. - I/O threads are used for certain operations. - Modules can use threads. - Since Redis 6.0, it supports multi-threaded I/O for network communication, improving performance on multi-core systems. Redis also uses pipelining for high throughput. Clients pipeline commands without waiting for each response. This allows more efficient network round trips, boosting overall performance. Efficient Data Structures Redis supports various optimized data structures, from linked lists, zip lists, and skip lists to sets, hashes, and sorted sets, among others. Each is carefully designed for specific use cases for quick and efficient data access. Over to you: With Redis now supporting some multi-threading, how should we configure it to fully utilize all the CPU cores of modern hardware when deploying in production? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages):

Sahn Lam

46,910 просмотров • 2 лет назад

Gemma 4 31B is way too slow on my MacBook Pro M5 Max with 128GB of unified memory. 42 seconds to generate the first line of code. 21 tokens per second after that. Uses 26GB of memory when loaded. I sat there waiting nearly a minute before it even started writing code. The output? Actually decent. The lava lamp it generated looks good. But 42 seconds to first token is a dealbreaker. Claude Opus 4.7 starts generating in under 2 seconds. GPT 5.5 is instant. Good model. Unusable speed. Local inference on a $5,399 machine still can't match the cloud.
2:37

Gemma 4 31B is way too slow on my MacBook Pro M5 Max with 128GB of unified memory. 42 seconds to generate the first line of code. 21 tokens per second after that. Uses 26GB of memory when loaded. I sat there waiting nearly a minute before it even started writing code. The output? Actually decent. The lava lamp it generated looks good. But 42 seconds to first token is a dealbreaker. Claude Opus 4.7 starts generating in under 2 seconds. GPT 5.5 is instant. Good model. Unusable speed. Local inference on a $5,399 machine still can't match the cloud.

BridgeMind

40,282 просмотров • 3 месяцев назад

For the last few weeks, I've been writing a multi-CPU single-board computer like explorer. I'm calling it CROSSWOZ (Hat tip to Woz and the WOZMON). Seven vintage CPU cores (6502, 65C02, Z80, 8080, 8085, 6809, 1802) share a 64K memory bus and the same WOZMON-style monitor. I can swap CPUs in place. It's got a mini-assembler per CPU, JMON-style debug, and animated paper-tape I/O. I'm pretty pleased with how it's turning out.
0:29

For the last few weeks, I've been writing a multi-CPU single-board computer like explorer. I'm calling it CROSSWOZ (Hat tip to Woz and the WOZMON). Seven vintage CPU cores (6502, 65C02, Z80, 8080, 8085, 6809, 1802) share a 64K memory bus and the same WOZMON-style monitor. I can swap CPUs in place. It's got a mini-assembler per CPU, JMON-style debug, and animated paper-tape I/O. I'm pretty pleased with how it's turning out.

mrdoornbos

15,112 просмотров • 3 месяцев назад

Groq's LPU is faster than GPUs, handling requests and responding more quickly. Groq's LPUs don't need speedy data delivery like Nvidia GPUs do because they don't have HBM in their system. They use SRAM, which is about 20 times faster than what GPUs use. Since inference runs use way less data than model training, Groq's LPU is more energy-efficient. It reads less from external memory and uses less power than a Nvidia GPU for inference tasks. The LPU works differently from GPUs. It uses a Temporal Instruction Set Computer architecture, so it doesn't have to reload data from memory as often as GPUs do with High Bandwidth Memory (HBM). This helps avoid issues with HBM shortages and keeps costs down. If Groq's LPU is used in places that do AI processing, you might not need special storage for Nvidia GPUs. The LPU doesn't demand super-fast storage like GPUs do. Groq is claiming that its technology could replace GPUs in AI tasks with its powerful chip and software.
7:57

Groq's LPU is faster than GPUs, handling requests and responding more quickly. Groq's LPUs don't need speedy data delivery like Nvidia GPUs do because they don't have HBM in their system. They use SRAM, which is about 20 times faster than what GPUs use. Since inference runs use way less data than model training, Groq's LPU is more energy-efficient. It reads less from external memory and uses less power than a Nvidia GPU for inference tasks. The LPU works differently from GPUs. It uses a Temporal Instruction Set Computer architecture, so it doesn't have to reload data from memory as often as GPUs do with High Bandwidth Memory (HBM). This helps avoid issues with HBM shortages and keeps costs down. If Groq's LPU is used in places that do AI processing, you might not need special storage for Nvidia GPUs. The LPU doesn't demand super-fast storage like GPUs do. Groq is claiming that its technology could replace GPUs in AI tasks with its powerful chip and software.

Brian Roemmele

87,214 просмотров • 2 лет назад

Americans are being priced out of playing videos games and the reason is Data Centers The Nintendo Switch 2 is $500 A high-end Xbox is $800 The PlayStation 5 Pro is $900 It’s causing major shifts: - PlayStation is ending physical disks - NVIDIA GeForce is implementing a gaming cap for a maximum amount of hours played per month - XBox is divesting in 5 gaming studios Data Centers are buying and using so much RAM memory that one stick that used to cost $200 now costs over $1,000 “Huge increases across the board” 3 companies make almost all memory chips “these chips are really only made by about 3 companies in the entire world, and those 3 companies have limited output. So they have to decide, do they want to build high-end memory chips for AI data centers or consumer-grade memory chips for all of our devices? As you can probably guess, They didn't choose us. OpenAI reportedly committed to buying 40% of all the memory chips in the entire world just for their Stargate multi-state data center project. For memory chip companies, this was a huge payday, but it meant absolute scarcity for consumer memory chips, which of course means massive price increases for you and I” Gaming companies aren’t going to invest in next gen consoles that will only sell a fraction of the amount of previous consoles because of price. It wouldn’t make sense because game developers are going to focus on the consoles with all the users, the older consoles So gaming console manufacturers are moving towards new revenue streams like controlling how you can buy a game, more subscription costs and more
5:32

Americans are being priced out of playing videos games and the reason is Data Centers The Nintendo Switch 2 is $500 A high-end Xbox is $800 The PlayStation 5 Pro is $900 It’s causing major shifts: - PlayStation is ending physical disks - NVIDIA GeForce is implementing a gaming cap for a maximum amount of hours played per month - XBox is divesting in 5 gaming studios Data Centers are buying and using so much RAM memory that one stick that used to cost $200 now costs over $1,000 “Huge increases across the board” 3 companies make almost all memory chips “these chips are really only made by about 3 companies in the entire world, and those 3 companies have limited output. So they have to decide, do they want to build high-end memory chips for AI data centers or consumer-grade memory chips for all of our devices? As you can probably guess, They didn't choose us. OpenAI reportedly committed to buying 40% of all the memory chips in the entire world just for their Stargate multi-state data center project. For memory chip companies, this was a huge payday, but it meant absolute scarcity for consumer memory chips, which of course means massive price increases for you and I” Gaming companies aren’t going to invest in next gen consoles that will only sell a fraction of the amount of previous consoles because of price. It wouldn’t make sense because game developers are going to focus on the consoles with all the users, the older consoles So gaming console manufacturers are moving towards new revenue streams like controlling how you can buy a game, more subscription costs and more

Wall Street Apes

80,738 просмотров • 27 дней назад

The AI boom just hit a wall nobody saw coming. And it's not software. It's not regulation. It's not even energy... It's memory chips. Right now, Dell is raising PC prices by 30%. Intel can't ship chips. Nvidia is slashing GPU production by 40%. And almost nobody understands why. Here's the "hidden" crisis the AI industry is trying to hide: AI data centers are hoarding memory. Not GPUs. Not processors. MEMORY. Every AI server needs massive amounts of high-bandwidth memory (HBM) to run those models everyone's hyping. One problem: There are only 3 companies in the world that can make it. Samsung. SK Hynix. Micron. That's it. And all 3 just diverted their entire production capacity away from normal RAM to feed AI data centers. The math that breaks everything: 1 gigabyte of HBM takes 4X the manufacturing capacity of regular DRAM. AI will consume 20% of global DRAM production in 2026. But the thing is, consumer demand for RAM didn't disappear. PCs still need memory. Phones still need memory. Cars still need memory. But there's no capacity left to make it. The price explosion: RAM prices are up 246% in the last 6 months. DDR5 contract prices jumped 100% month-over-month in some cases. Dell's CFO said he's "never witnessed costs escalating at this pace." SK Hynix and Micron? Sold out through all of 2026. Micron straight up EXITED the consumer memory market entirely to focus on AI customers. If you're not building an AI data center, you're not getting memory chips. AI data centers pay 3-5X margins compared to consumer products. So memory manufacturers are rationally choosing: Serve Microsoft and Google's AI buildout, or serve Dell's laptop business? Easy choice. Every wafer allocated to an Nvidia H100 GPU is a wafer DENIED to your next laptop. It's a zero-sum game. And consumers are losing. The dangerous cascade effect: Nvidia is cutting RTX 50-series GPU production by 30-40% because they can't get GDDR7 memory. Dell, Lenovo, HP are all raising PC prices 15-30% in early 2026. Xiaomi and other smartphone makers are cutting shipment targets. Even Intel's crash last week? Partially driven by memory shortages limiting chip production. This is a PERMANENT reallocation of the world's silicon capacity. Not a temporary supply hiccup. For decades, consumer electronics (phones, PCs, laptops) drove memory production. Now? AI data centers are the priority customer. And that priority shift is reshaping the entire tech economy. The timeline Is worse than you think: Industry analysts project shortages lasting through 2027, maybe 2028. Why? Because building new memory fabs takes 3-5 YEARS. Micron's new Idaho fab won't meaningfully impact supply until 2028. Samsung and SK Hynix are too busy ramping up HBM4 production to expand consumer DRAM. So we're stuck. AI companies need memory to scale. But producing that memory DESTROYS the supply chain for everything else. My question here: Everyone's betting on AI scaling infinitely. But what if the AI boom STALLS because there's not enough memory to support it? What if we're not in an "AI supercycle" but a "memory shortage that kills the AI buildout"? Intel crashed 17% because they can't manufacture enough chips. The root cause though? Memory shortages limiting what they can even produce. Nvidia is cutting GPU production by 40%. AMD is struggling to get GDDR6 for Radeon cards. This isn't just a consumer problem. It's an AI infrastructure problem. And if memory doesn't scale, AI doesn't scale. The AI industry sold you on infinite scaling. But they forgot to mention the part where there's only 3 companies making the memory chips that power everything. And all 3 just chose AI data centers over you. Even Nvidia can't make enough GPUs to meet demand. Not because of energy. Not because of regulation... But because the memory supply chain is BROKEN. And it won't be fixed until 2028.
0:58

The AI boom just hit a wall nobody saw coming. And it's not software. It's not regulation. It's not even energy... It's memory chips. Right now, Dell is raising PC prices by 30%. Intel can't ship chips. Nvidia is slashing GPU production by 40%. And almost nobody understands why. Here's the "hidden" crisis the AI industry is trying to hide: AI data centers are hoarding memory. Not GPUs. Not processors. MEMORY. Every AI server needs massive amounts of high-bandwidth memory (HBM) to run those models everyone's hyping. One problem: There are only 3 companies in the world that can make it. Samsung. SK Hynix. Micron. That's it. And all 3 just diverted their entire production capacity away from normal RAM to feed AI data centers. The math that breaks everything: 1 gigabyte of HBM takes 4X the manufacturing capacity of regular DRAM. AI will consume 20% of global DRAM production in 2026. But the thing is, consumer demand for RAM didn't disappear. PCs still need memory. Phones still need memory. Cars still need memory. But there's no capacity left to make it. The price explosion: RAM prices are up 246% in the last 6 months. DDR5 contract prices jumped 100% month-over-month in some cases. Dell's CFO said he's "never witnessed costs escalating at this pace." SK Hynix and Micron? Sold out through all of 2026. Micron straight up EXITED the consumer memory market entirely to focus on AI customers. If you're not building an AI data center, you're not getting memory chips. AI data centers pay 3-5X margins compared to consumer products. So memory manufacturers are rationally choosing: Serve Microsoft and Google's AI buildout, or serve Dell's laptop business? Easy choice. Every wafer allocated to an Nvidia H100 GPU is a wafer DENIED to your next laptop. It's a zero-sum game. And consumers are losing. The dangerous cascade effect: Nvidia is cutting RTX 50-series GPU production by 30-40% because they can't get GDDR7 memory. Dell, Lenovo, HP are all raising PC prices 15-30% in early 2026. Xiaomi and other smartphone makers are cutting shipment targets. Even Intel's crash last week? Partially driven by memory shortages limiting chip production. This is a PERMANENT reallocation of the world's silicon capacity. Not a temporary supply hiccup. For decades, consumer electronics (phones, PCs, laptops) drove memory production. Now? AI data centers are the priority customer. And that priority shift is reshaping the entire tech economy. The timeline Is worse than you think: Industry analysts project shortages lasting through 2027, maybe 2028. Why? Because building new memory fabs takes 3-5 YEARS. Micron's new Idaho fab won't meaningfully impact supply until 2028. Samsung and SK Hynix are too busy ramping up HBM4 production to expand consumer DRAM. So we're stuck. AI companies need memory to scale. But producing that memory DESTROYS the supply chain for everything else. My question here: Everyone's betting on AI scaling infinitely. But what if the AI boom STALLS because there's not enough memory to support it? What if we're not in an "AI supercycle" but a "memory shortage that kills the AI buildout"? Intel crashed 17% because they can't manufacture enough chips. The root cause though? Memory shortages limiting what they can even produce. Nvidia is cutting GPU production by 40%. AMD is struggling to get GDDR6 for Radeon cards. This isn't just a consumer problem. It's an AI infrastructure problem. And if memory doesn't scale, AI doesn't scale. The AI industry sold you on infinite scaling. But they forgot to mention the part where there's only 3 companies making the memory chips that power everything. And all 3 just chose AI data centers over you. Even Nvidia can't make enough GPUs to meet demand. Not because of energy. Not because of regulation... But because the memory supply chain is BROKEN. And it won't be fixed until 2028.

Ricardo

594,643 просмотров • 7 месяцев назад

$MU $00060 $00530 $TSM Evolution of the AI Memory Hierarchy: HBM, DRAM, & SRAM Overview and Market Analysis The video outlines the strategic shift of modern computing toward a memory-centric architecture driven by the demands of generative AI. It explains that the industry is moving away from a traditional commodity model to a tightly coupled hierarchy of specialized technologies, where High Bandwidth Memory (HBM) and advanced packaging act as primary bottlenecks for performance. Leading manufacturers like SK hynix, Samsung, and Micron are navigating complex transitions to HBM4 and CXL-based expansion to overcome data movement constraints. Beyond the memory producers, the ecosystem relies on integration enablers like TSMC for packaging and specialized equipment providers for high-precision manufacturing. Ultimately, the analysis suggests that memory intensity and power efficiency have become the decisive factors in the success of gigawatt-scale data centers.
14:38

$MU $00060 $00530 $TSM Evolution of the AI Memory Hierarchy: HBM, DRAM, & SRAM Overview and Market Analysis The video outlines the strategic shift of modern computing toward a memory-centric architecture driven by the demands of generative AI. It explains that the industry is moving away from a traditional commodity model to a tightly coupled hierarchy of specialized technologies, where High Bandwidth Memory (HBM) and advanced packaging act as primary bottlenecks for performance. Leading manufacturers like SK hynix, Samsung, and Micron are navigating complex transitions to HBM4 and CXL-based expansion to overcome data movement constraints. Beyond the memory producers, the ecosystem relies on integration enablers like TSMC for packaging and specialized equipment providers for high-precision manufacturing. Ultimately, the analysis suggests that memory intensity and power efficiency have become the decisive factors in the success of gigawatt-scale data centers.

TheValueist

23,901 просмотров • 8 месяцев назад

Dylan on why memory prices will double again: "Memory can only grow capacity low double digit percentages a year. 20-30% a year, even less for NAND, a little bit higher for DRAM, but whatever. Even though the demand signal was very strong at the end of 2025, the memory companies immediately started reacting. None of that incremental capacity really gets here until 2028, even if they wanted to build as fast as possible. So the result is memory prices have gone through the roof. They're going to double and triple again. DRAM, especially. People are like, "oh, the memory story is overplayed." No, you don't get it. DRAM will double or triple from here still, because that's how much capacity is required. And they have to steal capacity from somewhere else. And the only way to steal capacity from somewhere else in a capitalist economy is demand destruction via higher pricing."
2:17

Dylan on why memory prices will double again: "Memory can only grow capacity low double digit percentages a year. 20-30% a year, even less for NAND, a little bit higher for DRAM, but whatever. Even though the demand signal was very strong at the end of 2025, the memory companies immediately started reacting. None of that incremental capacity really gets here until 2028, even if they wanted to build as fast as possible. So the result is memory prices have gone through the roof. They're going to double and triple again. DRAM, especially. People are like, "oh, the memory story is overplayed." No, you don't get it. DRAM will double or triple from here still, because that's how much capacity is required. And they have to steal capacity from somewhere else. And the only way to steal capacity from somewhere else in a capitalist economy is demand destruction via higher pricing."

Patrick OShaughnessy

189,478 просмотров • 4 месяцев назад

🚨 SOMEONE JUST KILLED THE COACHING INDUSTRY a developer spent 22,000 hours building a Personal AI Operating System on top of Claude Code now anyone with a terminal can install it for FREE it knows your goals, remembers every decision you've made, and prepares your morning briefing while you sleep [ the numbers are insane ]: - hours of dev work in it: 22,000 - sessions logged: 6,000 - time saved per day: 2-3 hours - GitHub stars: 12,100 - skills built in: 45 - workflows wired up: 171 - safety hooks: 37 - cost to install: $0 [ the science is wild too ]: no embeddings, no vector databases, no AI magic you can't read every memory, decision, and context lives in plain markdown files you read it with cat, search it with ripgrep, version it with git 4 memory types compound over time: - work memory (active projects, open decisions) - knowledge memory (domain expertise, research) - people memory (contacts, companies, relationships) - learning memory (patterns, mistakes, what works for YOU) every complex task routes through a 7-step cycle: OBSERVE → THINK → PLAN → BUILD → EXECUTE → VERIFY → LEARN privacy is enforced by CODE, not prompts a hook called ContainmentGuard physically blocks sensitive data from being written outside designated zones [ the grift opportunity is even wilder ]: freelancers are already charging $500-2,000 per personal AI setup for executives, founders, and busy operators one person + one weekend = a consulting business that didn't exist 6 months ago every AI productivity app you're paying $30/month for is replaceable by 4 hours of setup work and this one repo REPO: 100% OPEN SOURCE, FREE
0:47

🚨 SOMEONE JUST KILLED THE COACHING INDUSTRY a developer spent 22,000 hours building a Personal AI Operating System on top of Claude Code now anyone with a terminal can install it for FREE it knows your goals, remembers every decision you've made, and prepares your morning briefing while you sleep [ the numbers are insane ]: - hours of dev work in it: 22,000 - sessions logged: 6,000 - time saved per day: 2-3 hours - GitHub stars: 12,100 - skills built in: 45 - workflows wired up: 171 - safety hooks: 37 - cost to install: $0 [ the science is wild too ]: no embeddings, no vector databases, no AI magic you can't read every memory, decision, and context lives in plain markdown files you read it with cat, search it with ripgrep, version it with git 4 memory types compound over time: - work memory (active projects, open decisions) - knowledge memory (domain expertise, research) - people memory (contacts, companies, relationships) - learning memory (patterns, mistakes, what works for YOU) every complex task routes through a 7-step cycle: OBSERVE → THINK → PLAN → BUILD → EXECUTE → VERIFY → LEARN privacy is enforced by CODE, not prompts a hook called ContainmentGuard physically blocks sensitive data from being written outside designated zones [ the grift opportunity is even wilder ]: freelancers are already charging $500-2,000 per personal AI setup for executives, founders, and busy operators one person + one weekend = a consulting business that didn't exist 6 months ago every AI productivity app you're paying $30/month for is replaceable by 4 hours of setup work and this one repo REPO: 100% OPEN SOURCE, FREE

Ronin

614,821 просмотров • 3 месяцев назад