正在加载视频...

视频加载失败

返回首页

Running GLM 4.7 Flash (8-bit) with Tensor Parallel / RDMA on 2 M4 Pro Mac Minis at 60 tok/sec. mlx-lm 0.30.5 features huge speedups for GLM 4.7 Flash for long context (h/t N8 Programs & Awni Hannun). M5 Pro (~28 Jan) will have ~4x faster prefill and ~1.3x faster decode.

alexocheema's profile picture

Alex Cheema

49,275 subscribers

56,555 次观看 • 6 个月前 •via X (Twitter)

游戏教育科学技术
LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Running GLM-4.7-Flash with OpenCode locally on M4 Max MacBook Pro. 4-bit model runs at 82 tok/sec. Prefill will get ~4x faster with M5 Max MacBook Pro (~28 Jan). EXO will also support disaggregating prefill and decode across devices, e.g. DGX Spark.
0:43

Running GLM-4.7-Flash with OpenCode locally on M4 Max MacBook Pro. 4-bit model runs at 82 tok/sec. Prefill will get ~4x faster with M5 Max MacBook Pro (~28 Jan). EXO will also support disaggregating prefill and decode across devices, e.g. DGX Spark.

Alex Cheema

128,404 次观看 • 6 个月前

Running GLM-4.7-Flash on 4 x M4 Pro Mac Minis using EXO Labs. Uses tensor parallelism with RDMA over Thunderbolt & MLX backend (h/t Awni Hannun). Runs at 100 tok/sec. We're working on optimizing this at EXO Labs. Aiming to hit ~200 tok/sec on this setup soon.
0:22

Running GLM-4.7-Flash on 4 x M4 Pro Mac Minis using EXO Labs. Uses tensor parallelism with RDMA over Thunderbolt & MLX backend (h/t Awni Hannun). Runs at 100 tok/sec. We're working on optimizing this at EXO Labs. Aiming to hit ~200 tok/sec on this setup soon.

Alex Cheema

62,221 次观看 • 6 个月前

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.
0:37

GLM 4.7 Flash is supported in mlx-lm 0.30.3 (h/t Ivan Fioravanti ᯅ) The 4-bit runs fast (43 tok/s generation, ~800 tok/s prefill) on a base M5 32GB laptop.

Awni Hannun

141,194 次观看 • 6 个月前

For even higher throughput and lower latency: batch generation + tensor parallel with mlx-lm + and mlx.distributed. Here it's generating at 63 tok/sec (throughput) with GLM 4.7 in 6-bit and batch size 4 on 4 M3 Ultras:
0:41

For even higher throughput and lower latency: batch generation + tensor parallel with mlx-lm + and mlx.distributed. Here it's generating at 63 tok/sec (throughput) with GLM 4.7 in 6-bit and batch size 4 on 4 M3 Ultras:

Awni Hannun

22,721 次观看 • 7 个月前

Timelapse #156 (36 hrs) - Worked with the tiny corp on getting GLM 5.2 running on 8xMI300X (sglang won here) - Launched KernelBench-Mega and updated Kernelbench-Hard with h100 and b200 sweeps - Took care of boring business stuff - Did some training sweeps for specialized technical vocab audio model - Some bugs with putting kernelbench-mega and hard on cloud instances so had to do some reruns. Learned a lot though - Setting up my own local rl infra and profiling concurrency 128 rollouts with vllm. Became clear to me that I need to serve in nvfp4, use MoE only for throughput, reap so training doesn’t OOM, dig into the vllm kernel graph itself to not underutilize my hardware from poor flashinfer/cutlass selections for my rtx pro 6000 sm120 architecture - Might do online distillation from glm 5.2 but for now taking it one step at a time - Slept for a bit then woke up and showered - Fixed an issue with SGLang tensor parallel deadlock on GLM 5.2 architecture with MTP enabled - GLM 5.2 inference is 2-3x faster than coding plans and running on amd boxes - Spent time with family - Hung out with some friends - Recorded some yoctogpt lectures with the revamped notebook (high taste btw) - Setting up dflash training for GLM 5.2
0:30

Timelapse #156 (36 hrs) - Worked with the tiny corp on getting GLM 5.2 running on 8xMI300X (sglang won here) - Launched KernelBench-Mega and updated Kernelbench-Hard with h100 and b200 sweeps - Took care of boring business stuff - Did some training sweeps for specialized technical vocab audio model - Some bugs with putting kernelbench-mega and hard on cloud instances so had to do some reruns. Learned a lot though - Setting up my own local rl infra and profiling concurrency 128 rollouts with vllm. Became clear to me that I need to serve in nvfp4, use MoE only for throughput, reap so training doesn’t OOM, dig into the vllm kernel graph itself to not underutilize my hardware from poor flashinfer/cutlass selections for my rtx pro 6000 sm120 architecture - Might do online distillation from glm 5.2 but for now taking it one step at a time - Slept for a bit then woke up and showered - Fixed an issue with SGLang tensor parallel deadlock on GLM 5.2 architecture with MTP enabled - GLM 5.2 inference is 2-3x faster than coding plans and running on amd boxes - Spent time with family - Hung out with some friends - Recorded some yoctogpt lectures with the revamped notebook (high taste btw) - Setting up dflash training for GLM 5.2

Elliot Arledge

45,447 次观看 • 1 个月前

I just crammed the updated Gemma 4 26B A4B QAT (MoE) with 180k context into an 8GB RTX 4060 (8 GB VRAM + 16 GB RAM only!!) and optimized the batch size. 23 tokens/sec decode, 300 tokens/sec prefill Yesterday I showed you a Gemma 4 31B dense model running flawlessly on an RTX 4090. Today, we're breaking the VRAM bank on a budget card using Unsloth’s new Gemma 4 26B (A4B) QAT quants. Following Google’s chat template update that boosted agentic benchmarks by +10%, I pushed this model to its absolute limits. Here is how you squeeze 250k context out of 8GB of VRAM. # The Setup & The Optimization - Hardware: Nvidia RTX 4060 (8GB VRAM) + 16GB System RAM - Environment: CUDA 13.0 build of llama.cpp - Model: gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf - Prompt: 28,000 tokens of prompt for each run If you read my L2 cache breakdown (attached in replies), you know the 4060’s 24MB cache maxes out at `-b 1024 -ub 1024`. Push past that, and prefill crashes. I locked those flags in for every test below to ensure maximum GEMM throughput. # 1. The Raw Context Push (Unquantized KV Cache) First, I wanted to see how far pure 8GB VRAM + 16GB RAM could stretch without touching the KV cache: - 80k Context: Prefill 385 t/s | Decode 25.5 t/s - 120k Context: Prefill 270 t/s | Decode 24 t/s llama.cpp flags: .\llama-server -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 120000 --port 8080 -ub 1024 -b 1024 Without KV quantization, 120k is your hard ceiling. push past that prefill throughput drops off a cliff, making the model practically unusable for large agentic workloads. # 2. The Q8 KV Cache Lifeline To survive 250k context on a budget card, you have to quantize the KV cache. I enabled 8 bit KV cache (`-ctk q8_0 -ctv q8_0`) and re ran: - 180k Context: Prefill 280 t/s | Decode 22.8 t/s - 250k Context: Prefill 115 t/s | Decode 20 t/s llama.cpp flags: .\llama-server -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 180000 --port 8080 -b 1024 -ub 1024 -ctk q8_0 -ctv q8_0 Result: Q8 KV cache brings 250k context back from the dead. Decode speed stabilizes at a highly usable 20 t/s. You are trading a very small bit amount of reasoning precision for an extra 130,000 tokens of context window. if you own a single rtx 3050, 3060, 3070, 4050, 4060, 5050 or 5060, you must try this model and optimize your batch size for higher prefill. Hugging Face links to the updated Unsloth's QAT quants and performance graph are in the replies below. What model are you running on your 6GB, 8GB or 12GB cards right now? Let's see your setups.
0:20

I just crammed the updated Gemma 4 26B A4B QAT (MoE) with 180k context into an 8GB RTX 4060 (8 GB VRAM + 16 GB RAM only!!) and optimized the batch size. 23 tokens/sec decode, 300 tokens/sec prefill Yesterday I showed you a Gemma 4 31B dense model running flawlessly on an RTX 4090. Today, we're breaking the VRAM bank on a budget card using Unsloth’s new Gemma 4 26B (A4B) QAT quants. Following Google’s chat template update that boosted agentic benchmarks by +10%, I pushed this model to its absolute limits. Here is how you squeeze 250k context out of 8GB of VRAM. # The Setup & The Optimization - Hardware: Nvidia RTX 4060 (8GB VRAM) + 16GB System RAM - Environment: CUDA 13.0 build of llama.cpp - Model: gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf - Prompt: 28,000 tokens of prompt for each run If you read my L2 cache breakdown (attached in replies), you know the 4060’s 24MB cache maxes out at `-b 1024 -ub 1024`. Push past that, and prefill crashes. I locked those flags in for every test below to ensure maximum GEMM throughput. # 1. The Raw Context Push (Unquantized KV Cache) First, I wanted to see how far pure 8GB VRAM + 16GB RAM could stretch without touching the KV cache: - 80k Context: Prefill 385 t/s | Decode 25.5 t/s - 120k Context: Prefill 270 t/s | Decode 24 t/s llama.cpp flags: .\llama-server -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 120000 --port 8080 -ub 1024 -b 1024 Without KV quantization, 120k is your hard ceiling. push past that prefill throughput drops off a cliff, making the model practically unusable for large agentic workloads. # 2. The Q8 KV Cache Lifeline To survive 250k context on a budget card, you have to quantize the KV cache. I enabled 8 bit KV cache (`-ctk q8_0 -ctv q8_0`) and re ran: - 180k Context: Prefill 280 t/s | Decode 22.8 t/s - 250k Context: Prefill 115 t/s | Decode 20 t/s llama.cpp flags: .\llama-server -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 180000 --port 8080 -b 1024 -ub 1024 -ctk q8_0 -ctv q8_0 Result: Q8 KV cache brings 250k context back from the dead. Decode speed stabilizes at a highly usable 20 t/s. You are trading a very small bit amount of reasoning precision for an extra 130,000 tokens of context window. if you own a single rtx 3050, 3060, 3070, 4050, 4060, 5050 or 5060, you must try this model and optimize your batch size for higher prefill. Hugging Face links to the updated Unsloth's QAT quants and performance graph are in the replies below. What model are you running on your 6GB, 8GB or 12GB cards right now? Let's see your setups.

Alok

36,473 次观看 • 11 天前

People made fun of Alex Finn for buying three Mac Studios to run AI at home. Then Fable got banned for a week, GLM 5.2 dropped, and those exact Mac Studios started reselling for 4x what he paid. He showed me how he built his home AI lab from scratch. Here's the playbook: 1) The hardware. three 512GB Mac Studios, an NVIDIA DGX Spark, a custom RTX 5090 build, and a few Mac Minis. ~$30k all in. 2) The buying framework... - Mac Studio: huge memory, runs GLM 5.2 (open weights, near Opus 4.8 on benchmarks), but slow. - DGX Spark ($4,800): the sweet spot for most people. - RTX 5090: smaller models at blazing speed (Qwen's 29B now hits Sonnet 4 level). 3) Tailscale networks every machine into one private network with root access to each other. Only one machine is plugged into a monitor. 4) A Nous Research Hermes agent is his IT guy. New model drops? It SSHs into the right box, loads 5 candidates, runs evals overnight, and reports back which task belongs on which machine. Alex has literally never loaded a model himself. 5) The whole point: achieving "ambient intelligence." Always-on jobs that would bankrupt you on per-token billing. A security sweep of his API endpoints every hour. Code optimization every 20 minutes. Database anomaly & churn detection. Hourly scraping of X, Reddit & Hacker News for business opportunities. 6) Running those workloads on frontier models would cost thousands a month. His actual cost: ~$60 more in electricity. 7) Btw he's not anti-frontier. He still maxes out his Claude plan. The way he sees it: frontier is for hard thinking, local is for the foot soldiers that never sleep. 8) "We own everything except for the intelligence. Why can't we own the intelligence?" 9) He thinks frontier-level intelligence runs on consumer hardware within 6 months.
36:48

People made fun of Alex Finn for buying three Mac Studios to run AI at home. Then Fable got banned for a week, GLM 5.2 dropped, and those exact Mac Studios started reselling for 4x what he paid. He showed me how he built his home AI lab from scratch. Here's the playbook: 1) The hardware. three 512GB Mac Studios, an NVIDIA DGX Spark, a custom RTX 5090 build, and a few Mac Minis. ~$30k all in. 2) The buying framework... - Mac Studio: huge memory, runs GLM 5.2 (open weights, near Opus 4.8 on benchmarks), but slow. - DGX Spark ($4,800): the sweet spot for most people. - RTX 5090: smaller models at blazing speed (Qwen's 29B now hits Sonnet 4 level). 3) Tailscale networks every machine into one private network with root access to each other. Only one machine is plugged into a monitor. 4) A Nous Research Hermes agent is his IT guy. New model drops? It SSHs into the right box, loads 5 candidates, runs evals overnight, and reports back which task belongs on which machine. Alex has literally never loaded a model himself. 5) The whole point: achieving "ambient intelligence." Always-on jobs that would bankrupt you on per-token billing. A security sweep of his API endpoints every hour. Code optimization every 20 minutes. Database anomaly & churn detection. Hourly scraping of X, Reddit & Hacker News for business opportunities. 6) Running those workloads on frontier models would cost thousands a month. His actual cost: ~$60 more in electricity. 7) Btw he's not anti-frontier. He still maxes out his Claude plan. The way he sees it: frontier is for hard thinking, local is for the foot soldiers that never sleep. 8) "We own everything except for the intelligence. Why can't we own the intelligence?" 9) He thinks frontier-level intelligence runs on consumer hardware within 6 months.

Alex Lieberman

57,044 次观看 • 26 天前

A single RTX 4090 (24 GB VRAM) can run the updated gemma 4 31B (dense) model with a 190,000 context window at 33 tokens/second. The VRAM barrier is dying. Google quietly updated Gemma 4, and Unsloth immediately compiled the new quants. I built llama.cpp from source on Ubuntu 22 to benchmark it. Google's stealth update 2 days ago enabled uniform Flash Attention 4 on Hopper to boost prefill and patched the chat template to improve tool calling. The agentic reasoning gains on the benchmark charts are massive: TB2 (Agents): +4.5% (to 25.8%) Tau2 (Telecom): +10.1% (to 62.7%) Running on Ubuntu 22, CUDA 13.0 with a single NVIDIA GeForce RTX 4090. Here is the exact step by step benchmarking process with a massive 28k tokens prompt and the commands I used to squeeze out maximum context without killing my throughput: # 1. The Baseline (Unquantized KV Cache) I started with full GPU offload (-ngl 99) and pushed the context to 40k. llama.cpp flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -ngl 99 -c 40000 -fa on --port 8080 -v VRAM: 23.8 GB (maxed out on card) Throughput: Prefill: 2198.81 t/s | Decode: 35.77 t/s (with 28k tokens prompt) # 2. The CPU Split Trap I tried stretching to 80k context by offloading layers to the CPU (-ngl 52). llama.cpp flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 80000 -ngl 52 -fa on --port 8080 -v Throughput: Prefill: 1212.73 t/s | Decode: 5 t/s (with 28k tokens prompt) # 3. The KV Quantization Breakthrough Instead of spilling layers to the CPU, I kept the model fully on card (-ngl 99) but enabled 8-bit KV cache quantization to free up VRAM. flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 100000 --cache-type-k q8_0 --cache-type-v q8_0 -ngl 99 --port 8080 -v VRAM: 23.9 GB Throughput: Prefill: 2139.68 t/s | Decode: 32 t/s (with 28k tokens prompt) Result: 100k tokens of context on a single GPU with practically zero speed loss (and minimal intelligence loss). # 4. The Limit Test (Q4 KV Cache) To find the absolute breaking point, I dropped the KV cache to 4 bit (q4_0) and set -c 190000. flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 190000 --cache-type-k q4_0 --cache-type-v q4_0 -ngl 99 --port 8080 -v VRAM: 23.8 GB Throughput: Prefill: 2206.66 t/s | Decode: 33 t/s (with 28k tokens prompt) (Note: Pushing it to 220k required dropping to -ngl 58 again, which immediately penalized decode down to 17 t/s). # The Tradeoff: For Max Reasoning: Keep your KV cache unquantized (f16). You get pristine reasoning but hit a strict 40k context ceiling. For Massive Document Retrieval: If you need to feed the model giant codebases, use --cache-type-k q4_0. Getting 190k context at 33 tokens/second on a consumer desktop with a 31b dense model is a cheat code. If you’re rocking a single 3090 or 4090 and slept on Gemma 4 earlier, this update is your cue to dust off the terminal. Hugging Face links to the Unsloth QAT quants are in the replies below.
0:20

A single RTX 4090 (24 GB VRAM) can run the updated gemma 4 31B (dense) model with a 190,000 context window at 33 tokens/second. The VRAM barrier is dying. Google quietly updated Gemma 4, and Unsloth immediately compiled the new quants. I built llama.cpp from source on Ubuntu 22 to benchmark it. Google's stealth update 2 days ago enabled uniform Flash Attention 4 on Hopper to boost prefill and patched the chat template to improve tool calling. The agentic reasoning gains on the benchmark charts are massive: TB2 (Agents): +4.5% (to 25.8%) Tau2 (Telecom): +10.1% (to 62.7%) Running on Ubuntu 22, CUDA 13.0 with a single NVIDIA GeForce RTX 4090. Here is the exact step by step benchmarking process with a massive 28k tokens prompt and the commands I used to squeeze out maximum context without killing my throughput: # 1. The Baseline (Unquantized KV Cache) I started with full GPU offload (-ngl 99) and pushed the context to 40k. llama.cpp flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -ngl 99 -c 40000 -fa on --port 8080 -v VRAM: 23.8 GB (maxed out on card) Throughput: Prefill: 2198.81 t/s | Decode: 35.77 t/s (with 28k tokens prompt) # 2. The CPU Split Trap I tried stretching to 80k context by offloading layers to the CPU (-ngl 52). llama.cpp flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 80000 -ngl 52 -fa on --port 8080 -v Throughput: Prefill: 1212.73 t/s | Decode: 5 t/s (with 28k tokens prompt) # 3. The KV Quantization Breakthrough Instead of spilling layers to the CPU, I kept the model fully on card (-ngl 99) but enabled 8-bit KV cache quantization to free up VRAM. flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 100000 --cache-type-k q8_0 --cache-type-v q8_0 -ngl 99 --port 8080 -v VRAM: 23.9 GB Throughput: Prefill: 2139.68 t/s | Decode: 32 t/s (with 28k tokens prompt) Result: 100k tokens of context on a single GPU with practically zero speed loss (and minimal intelligence loss). # 4. The Limit Test (Q4 KV Cache) To find the absolute breaking point, I dropped the KV cache to 4 bit (q4_0) and set -c 190000. flags: ./build/bin/llama-server -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -c 190000 --cache-type-k q4_0 --cache-type-v q4_0 -ngl 99 --port 8080 -v VRAM: 23.8 GB Throughput: Prefill: 2206.66 t/s | Decode: 33 t/s (with 28k tokens prompt) (Note: Pushing it to 220k required dropping to -ngl 58 again, which immediately penalized decode down to 17 t/s). # The Tradeoff: For Max Reasoning: Keep your KV cache unquantized (f16). You get pristine reasoning but hit a strict 40k context ceiling. For Massive Document Retrieval: If you need to feed the model giant codebases, use --cache-type-k q4_0. Getting 190k context at 33 tokens/second on a consumer desktop with a 31b dense model is a cheat code. If you’re rocking a single 3090 or 4090 and slept on Gemma 4 earlier, this update is your cue to dust off the terminal. Hugging Face links to the Unsloth QAT quants are in the replies below.

Alok

75,704 次观看 • 16 天前

Ex-NFL Player John Madsen on how he makes $4M/mo with this fitness coaching business 👇 0:00 NFL to Fitness Entrepreneurship 0:23 Why Fitness Businesses Plateau 0:56 Scaling with High-Ticket Pricing 2:35 Mastering Acquisition Costs 6:10 Pivoting to Online Coaching 8:56 Targeting High-Net-Worth Leads 13:36 $40k Programs & Concierge Medical 15:40 Scaling Lessons & Team Hiring 21:14 Expanding to the Female Market 26:24 Brand Faces vs. Ambassadors 28:40 Organic Content vs. Paid Ads 34:30 Building Proprietary Tech & AI 38:00 AI for Copywriting & Sales Ops 41:38 Optimizing Sales & Show Rates 46:24 Fixing Sales Team Performance 57:58 Pro Sports to Business Mindset 1:00:37 NLP & Performance Coaching 1:06:04 Creating a High-Converting VSL - Sponsored by If you have a sales team and are using Calendly/Oncehub to book calls...check out Saleskicks booking system. Saleskick is a booking/calendar system designed specifically for sales teams to: -Decrease cost per booked call (faster load times, better CRO, etc) -Increase show rate -Auto-confirm appointments via AI -AI score every lead so it's delegated correctly... And more. Check it out at
1:15:42

Ex-NFL Player John Madsen on how he makes $4M/mo with this fitness coaching business 👇 0:00 NFL to Fitness Entrepreneurship 0:23 Why Fitness Businesses Plateau 0:56 Scaling with High-Ticket Pricing 2:35 Mastering Acquisition Costs 6:10 Pivoting to Online Coaching 8:56 Targeting High-Net-Worth Leads 13:36 $40k Programs & Concierge Medical 15:40 Scaling Lessons & Team Hiring 21:14 Expanding to the Female Market 26:24 Brand Faces vs. Ambassadors 28:40 Organic Content vs. Paid Ads 34:30 Building Proprietary Tech & AI 38:00 AI for Copywriting & Sales Ops 41:38 Optimizing Sales & Show Rates 46:24 Fixing Sales Team Performance 57:58 Pro Sports to Business Mindset 1:00:37 NLP & Performance Coaching 1:06:04 Creating a High-Converting VSL - Sponsored by If you have a sales team and are using Calendly/Oncehub to book calls...check out Saleskicks booking system. Saleskick is a booking/calendar system designed specifically for sales teams to: -Decrease cost per booked call (faster load times, better CRO, etc) -Increase show rate -Auto-confirm appointments via AI -AI score every lead so it's delegated correctly... And more. Check it out at

Cole Gordon

18,185 次观看 • 3 个月前

This is one of the craziest AI launches of 2026 and it came out of basically nowhere (Save this). A company called Subquadratic just shipped SubQ, and the benchmarks are almost hard to believe. To understand why this is such a big deal, you have to understand the fundamental problem that has defined AI for the last decade. Every large language model in existence is built on transformer architecture, and transformers use a mechanism called standard attention that checks every single word in a sequence against every other word. Double the context length and compute doesn't double, it quadruples, triple it and compute goes up nine times. This quadratic scaling is why frontier models have been stuck at roughly 1 million tokens, why running them at those lengths gets expensive fast, and why the AI labs have essentially been printing money charging you more the longer you need the model to think. The industry has known this problem existed since 2017 but they scaled it anyway. SubQ is built from the ground up to solve it. Instead of processing every possible token relationship, SubQ's sparse attention architecture identifies which relationships actually matter and ignores the rest meaning compute is used where it counts and wasted nowhere else. The result is that compute scales linearly with context length instead of exponentially, and the implications of that one architectural shift are enormous. At 12 million tokens, SubQ reduces attention compute by nearly 1,000x compared to standard frontier models and at 1 million tokens, it runs 52x faster than FlashAttention. And it does all of this while posting frontier level accuracy, scoring 95% on the RULER 128K long-context benchmark versus Claude Opus 4.6's 94.8%, and an 81.8 on SWE-Bench Verified coding tasks, besting Opus 4.6 (80.8) and DeepSeek 4.0 Pro. The cost comparison is where it gets genuinely insane. SubQ runs at under $1.50 per million tokens less than 5% of what Claude Opus charges. On the RULER benchmark, running the test with SubQ cost $8, running the same test with Claude Opus cost $2,600 and that's a 300x cost reduction at equivalent or better accuracy.. Subquadratic launched with $29 million in funding, SubQ is available today for early access via API, and SubQ Code, a coding agent built on the architecture ships alongside it. The transformer has been the unchallenged foundation of every major AI system since 2017. SubQ is the first serious evidence that something structurally better might have just arrived.
1:24

This is one of the craziest AI launches of 2026 and it came out of basically nowhere (Save this). A company called Subquadratic just shipped SubQ, and the benchmarks are almost hard to believe. To understand why this is such a big deal, you have to understand the fundamental problem that has defined AI for the last decade. Every large language model in existence is built on transformer architecture, and transformers use a mechanism called standard attention that checks every single word in a sequence against every other word. Double the context length and compute doesn't double, it quadruples, triple it and compute goes up nine times. This quadratic scaling is why frontier models have been stuck at roughly 1 million tokens, why running them at those lengths gets expensive fast, and why the AI labs have essentially been printing money charging you more the longer you need the model to think. The industry has known this problem existed since 2017 but they scaled it anyway. SubQ is built from the ground up to solve it. Instead of processing every possible token relationship, SubQ's sparse attention architecture identifies which relationships actually matter and ignores the rest meaning compute is used where it counts and wasted nowhere else. The result is that compute scales linearly with context length instead of exponentially, and the implications of that one architectural shift are enormous. At 12 million tokens, SubQ reduces attention compute by nearly 1,000x compared to standard frontier models and at 1 million tokens, it runs 52x faster than FlashAttention. And it does all of this while posting frontier level accuracy, scoring 95% on the RULER 128K long-context benchmark versus Claude Opus 4.6's 94.8%, and an 81.8 on SWE-Bench Verified coding tasks, besting Opus 4.6 (80.8) and DeepSeek 4.0 Pro. The cost comparison is where it gets genuinely insane. SubQ runs at under $1.50 per million tokens less than 5% of what Claude Opus charges. On the RULER benchmark, running the test with SubQ cost $8, running the same test with Claude Opus cost $2,600 and that's a 300x cost reduction at equivalent or better accuracy.. Subquadratic launched with $29 million in funding, SubQ is available today for early access via API, and SubQ Code, a coding agent built on the architecture ships alongside it. The transformer has been the unchallenged foundation of every major AI system since 2017. SubQ is the first serious evidence that something structurally better might have just arrived.

Milk Road AI

278,244 次观看 • 2 个月前

** MEGA Parodius Scaling Effects Part 2 ** Well - this is the BIG one - literally !! Huge thanks to my team Pyron & Vector Orbitex for their efforts. Pyron has provided all the source frames for Puyon and his spikes / explosion and a lot of analysis video on how the spikes behave / move which was really helpful. Pyron also used his CRT setup for this video as we felt an emulator video would not do it justice - running on 100% real MD hardware FTW. Vector has provided the catchy boss music and its sounding great - as always ! The coding on this has been a bit insane - things done since part 1 post previous: Implemented dual buffering - Last video was single buffered - so VRAM is very tight now , we only have about 40/2048 tiles free. For the longest time I didn't think it would fit - I found a vram jigsaw puzzle that made it work in the end. Double buffering has cleaned up the stability of the animation and matches the arcade scaling effect now albiet costing 2x more VRAM . Vertical Scaling Implemented - the vertical scaler was taken from Lufthoheit ( my other shooter ) and its heavily optimised to reduce cpu usage. During the scaling the vertical scaler partitions the 68k processor registers into 2 sets, 4 registers are allocated to feeding the fast horizontal interrupt (h-int) that drives the vertical scaling , remaining 12 registers are for the horizontal scaler running in the background. This setup is much faster than normal backup / restore register methods, as we do not need to backup / restore registers in the h-int which would double CPU costs. The catch is the momment any background routine tries to use the H-ints registers it would break things so it has to be carefully timed. Added the spike projectiles - this was very tricky to get close to the arcade, they are semi heat seeking missiles basically and hence needed code that worked out angle differences to player at speed theres no time for arc-tan or similar so it uses faster lookup tables to work out the angles . They speed up over time and get larger and whats more we can't keep all the scales in VRAM - we have room for 2 spike buffers only. Also what was a real pain was working out scaled coordinates for the circular launch of the spikes . Added Fish Damage - Shock frame and Explosion frames from Arcade . Very proud of the fact we have the full arcade quality explosion is which is fully scaled also. Added Temporal Masking - which is fancy wording for don't draw nothing to buffer if nothing is there already there for the scaling . So empty Corners and edges can be optimised out to lower cpu costs and rom costs. I had to make some scripting for this and work out what areas did not need drawing at all in the frame, which should be force cleared by cpu and which areas should just be copied from rom. This reduced rom size by 30 kb and with a bit more work we could extend that to 60 kb & get a bit more speedup even doing so. Added a frame limiter . In the last video update we let the 68k burn hot and just pump out frames as fast as it could - here we match the arcade animation rate which does leave the cpu idling at times , particularly in the smaller frames - even at large though we could be running the animation 25 % faster , issue is though that would speed up the game logic and make it less arcade accurate. We had some real bullet / Spike hell simulations going without the limiter but yeah we had to tone it down a bit. Maybe in a hardcore mode we could let it run wild though ! Code is 95% 68k assembly with about 5% C code (v-int as its cold path ) driving things . This sort of thing needs all the speed it can get !! Well now after all that I can return to finish off Level 1 haha - just a wee sidetrack there . No doubt we will polish stage 8 boss some more in time too !! #SegaMegadrive #SegaGenesis #Parodius #SGDK
0:58

** MEGA Parodius Scaling Effects Part 2 ** Well - this is the BIG one - literally !! Huge thanks to my team Pyron & Vector Orbitex for their efforts. Pyron has provided all the source frames for Puyon and his spikes / explosion and a lot of analysis video on how the spikes behave / move which was really helpful. Pyron also used his CRT setup for this video as we felt an emulator video would not do it justice - running on 100% real MD hardware FTW. Vector has provided the catchy boss music and its sounding great - as always ! The coding on this has been a bit insane - things done since part 1 post previous: Implemented dual buffering - Last video was single buffered - so VRAM is very tight now , we only have about 40/2048 tiles free. For the longest time I didn't think it would fit - I found a vram jigsaw puzzle that made it work in the end. Double buffering has cleaned up the stability of the animation and matches the arcade scaling effect now albiet costing 2x more VRAM . Vertical Scaling Implemented - the vertical scaler was taken from Lufthoheit ( my other shooter ) and its heavily optimised to reduce cpu usage. During the scaling the vertical scaler partitions the 68k processor registers into 2 sets, 4 registers are allocated to feeding the fast horizontal interrupt (h-int) that drives the vertical scaling , remaining 12 registers are for the horizontal scaler running in the background. This setup is much faster than normal backup / restore register methods, as we do not need to backup / restore registers in the h-int which would double CPU costs. The catch is the momment any background routine tries to use the H-ints registers it would break things so it has to be carefully timed. Added the spike projectiles - this was very tricky to get close to the arcade, they are semi heat seeking missiles basically and hence needed code that worked out angle differences to player at speed theres no time for arc-tan or similar so it uses faster lookup tables to work out the angles . They speed up over time and get larger and whats more we can't keep all the scales in VRAM - we have room for 2 spike buffers only. Also what was a real pain was working out scaled coordinates for the circular launch of the spikes . Added Fish Damage - Shock frame and Explosion frames from Arcade . Very proud of the fact we have the full arcade quality explosion is which is fully scaled also. Added Temporal Masking - which is fancy wording for don't draw nothing to buffer if nothing is there already there for the scaling . So empty Corners and edges can be optimised out to lower cpu costs and rom costs. I had to make some scripting for this and work out what areas did not need drawing at all in the frame, which should be force cleared by cpu and which areas should just be copied from rom. This reduced rom size by 30 kb and with a bit more work we could extend that to 60 kb & get a bit more speedup even doing so. Added a frame limiter . In the last video update we let the 68k burn hot and just pump out frames as fast as it could - here we match the arcade animation rate which does leave the cpu idling at times , particularly in the smaller frames - even at large though we could be running the animation 25 % faster , issue is though that would speed up the game logic and make it less arcade accurate. We had some real bullet / Spike hell simulations going without the limiter but yeah we had to tone it down a bit. Maybe in a hardcore mode we could let it run wild though ! Code is 95% 68k assembly with about 5% C code (v-int as its cold path ) driving things . This sort of thing needs all the speed it can get !! Well now after all that I can return to finish off Level 1 haha - just a wee sidetrack there . No doubt we will polish stage 8 boss some more in time too !! #SegaMegadrive #SegaGenesis #Parodius #SGDK

Shannon Birt

26,539 次观看 • 5 个月前

10 free Google AI tools nobody talks about. while everyone's burning $20/mo on chatgpt and claude, google quietly shipped a stack worth $200+/mo. all free. all yours. — 1️⃣ NotebookLM — your second brain upload sources (PDFs, websites, audio, YouTube). it summarizes, builds mind maps, generates quizzes, drafts slide decks, even turns your notes into a podcast you can listen to on a walk. free tier: 100 notebooks, 50 sources each, 50 chats/day, 3 audio overviews/day. replaces: notion AI + perplexity + readwise — 2️⃣ Google AI Studio — the free gemini playground web playground for gemini 3 pro and flash with a free API key. generous limits. paste a 1M-token context window and watch it actually use it. faster than the openai playground and free where openai charges per token. replaces: openai playground + paid API credits — 3️⃣ Gemini CLI — google's open-source terminal agent apache 2.0 licensed. one command (npx @google/gemini-cli) and you've got an agent in your terminal that reads your codebase, runs shell commands, and ships PRs. drop-in claude code alternative. replaces: claude code ($20/mo by default) — 4️⃣ Jules — async coding agent assign jules a github issue. it spins up a cloud VM, clones your repo, writes the plan, makes the changes, opens a PR. free tier: 15 tasks/day, 3 concurrent, runs on gemini flash. replaces: devin ($20/mo+) + cursor agent 5️⃣ Stitch — text → UI → code google's free figma killer. describe an interface, get production-ready HTML/CSS/Tailwind + figma export. march 2026 update added voice canvas, infinite canvas, and MCP integration with cursor. 350 standard + 200 experimental generations/month free. replaces: galileo AI + early-stage figma work — 6️⃣ Gemma 4 — open-weight LLM google's flagship open model. apache 2.0. 2B, 4B, 26B-MoE, and 31B variants. 256K context. runs on ollama with one command. quantized versions run on a 4090 or beefy laptop. replaces: paying for hosted LLM inference — 7️⃣ Illuminate — papers → podcasts paste an arxiv preprint link. illuminate turns dense research papers into a 6-8 min conversation between two AI hosts breaking it down. perfect for commute reading you can't do at a desk. note: still in waitlist for some regions. replaces: snipd + manual research reading — 8️⃣ Learn About (LearnLM) — adaptive AI tutor drop in any topic you're stuck on. highlight a word, click "go deeper," and the interface adapts in real time to your comprehension level. visual explanations, follow-up questions, the works. replaces: paid tutoring on niche topics — 9️⃣ Google Labs FX (ImageFX + Flow + MusicFX) — free imagen, veo, musicLM google labs creative suite. text-to-image (imagen 4), text-to-video (veo via Flow), text-to-music (musicLM). free tier: limited daily generations. the heavy veo 3.1 features are paid (AI Pro $19.99/mo). still worth using for image and music — those stay free. replaces: midjourney + suno (free tier only — runway-level video gen is paid) — 🔟 Google Colab — free GPU notebooks free T4 GPU + 12GB RAM in a browser tab. enough to fine-tune small models, run stable diffusion, prototype agents. the launching pad for half the ML projects on github. replaces: paid cloud GPU rentals — a quick honest note: these tools aren't 1:1 better than the paid versions they replace. but they're decent enough to get most things done — especially if you're not a heavy user or you've got little funds to play with. i've put all 10 in a public github repo (link in comments). follow + turn on post notifications for more useful posts like this 🔔
0:34

10 free Google AI tools nobody talks about. while everyone's burning $20/mo on chatgpt and claude, google quietly shipped a stack worth $200+/mo. all free. all yours. — 1️⃣ NotebookLM — your second brain upload sources (PDFs, websites, audio, YouTube). it summarizes, builds mind maps, generates quizzes, drafts slide decks, even turns your notes into a podcast you can listen to on a walk. free tier: 100 notebooks, 50 sources each, 50 chats/day, 3 audio overviews/day. replaces: notion AI + perplexity + readwise — 2️⃣ Google AI Studio — the free gemini playground web playground for gemini 3 pro and flash with a free API key. generous limits. paste a 1M-token context window and watch it actually use it. faster than the openai playground and free where openai charges per token. replaces: openai playground + paid API credits — 3️⃣ Gemini CLI — google's open-source terminal agent apache 2.0 licensed. one command (npx @google/gemini-cli) and you've got an agent in your terminal that reads your codebase, runs shell commands, and ships PRs. drop-in claude code alternative. replaces: claude code ($20/mo by default) — 4️⃣ Jules — async coding agent assign jules a github issue. it spins up a cloud VM, clones your repo, writes the plan, makes the changes, opens a PR. free tier: 15 tasks/day, 3 concurrent, runs on gemini flash. replaces: devin ($20/mo+) + cursor agent 5️⃣ Stitch — text → UI → code google's free figma killer. describe an interface, get production-ready HTML/CSS/Tailwind + figma export. march 2026 update added voice canvas, infinite canvas, and MCP integration with cursor. 350 standard + 200 experimental generations/month free. replaces: galileo AI + early-stage figma work — 6️⃣ Gemma 4 — open-weight LLM google's flagship open model. apache 2.0. 2B, 4B, 26B-MoE, and 31B variants. 256K context. runs on ollama with one command. quantized versions run on a 4090 or beefy laptop. replaces: paying for hosted LLM inference — 7️⃣ Illuminate — papers → podcasts paste an arxiv preprint link. illuminate turns dense research papers into a 6-8 min conversation between two AI hosts breaking it down. perfect for commute reading you can't do at a desk. note: still in waitlist for some regions. replaces: snipd + manual research reading — 8️⃣ Learn About (LearnLM) — adaptive AI tutor drop in any topic you're stuck on. highlight a word, click "go deeper," and the interface adapts in real time to your comprehension level. visual explanations, follow-up questions, the works. replaces: paid tutoring on niche topics — 9️⃣ Google Labs FX (ImageFX + Flow + MusicFX) — free imagen, veo, musicLM google labs creative suite. text-to-image (imagen 4), text-to-video (veo via Flow), text-to-music (musicLM). free tier: limited daily generations. the heavy veo 3.1 features are paid (AI Pro $19.99/mo). still worth using for image and music — those stay free. replaces: midjourney + suno (free tier only — runway-level video gen is paid) — 🔟 Google Colab — free GPU notebooks free T4 GPU + 12GB RAM in a browser tab. enough to fine-tune small models, run stable diffusion, prototype agents. the launching pad for half the ML projects on github. replaces: paid cloud GPU rentals — a quick honest note: these tools aren't 1:1 better than the paid versions they replace. but they're decent enough to get most things done — especially if you're not a heavy user or you've got little funds to play with. i've put all 10 in a public github repo (link in comments). follow + turn on post notifications for more useful posts like this 🔔

m0h

11,847 次观看 • 2 个月前

The Concorde could fly London to New York in ~3.5 hours (half the time of a normal airliner). In operation from 1976 to 2003, the plane was a technical marvel and faster than the speed of sound. But it had poor business economics. Let’s start with the insane specs: ▫️Got as high as 10k miles ▫️Flew around earth in 30 hours ▫️Mach 2.04 speed (more than 2x speed of sound) ▫️Arrived in New York at earlier time than it left London (banker folk loved this) ▫️Delta wings and nose tip that could point down (so pilots could see runway, because it had to land at specific angle) Jointly run by British Airways and Air France, there were only 14 Concordes put to work (flew 50,000 total flights). What was wrong with the business model? ▫️SMALL MARKET: The thin aerodynamic design could only seat 109, so tickets were very expensive: $11k (a Boeing 747 can do >800 passengers) ▫️HUGE MAINTENANCE: Planes got so hot at the max height (110 degrees), that they expanded 30cm and sealant for fuel hardened. It took 28 hours to turnaround the plane (a normal one can do <2 hours). ▫️OUTRAGEOUS FUEL NEEDS: Each flight required 28,000 litres, for max 109 passengers. Commmercial planes needed 4x less fuel on a per passenger basis. ▫️NOISE RESTRICTIONS: Many cities wouldn’t allow the Concorde to fly because of how glass-shatteringly long it is on take-off, which limited destinations. *** Development of the Concorde — it had an “e” to placate the French — cost $2.8B and was paid for by the UK and French governments. The airlines were actually able to fly profitably for a number of years. But the economics were warped because the planes were given to them for free and the airlines didn’t have to capitalize costs. The business ultimately shuttered in 2003 following a tragic crash in 2000 and industry-wide slowdown post-9/11. Will we ever have sub-4 hour flights from London to NY again? A batch of sonic plane startups could make it happen by the end of this decade.
0:39

The Concorde could fly London to New York in ~3.5 hours (half the time of a normal airliner). In operation from 1976 to 2003, the plane was a technical marvel and faster than the speed of sound. But it had poor business economics. Let’s start with the insane specs: ▫️Got as high as 10k miles ▫️Flew around earth in 30 hours ▫️Mach 2.04 speed (more than 2x speed of sound) ▫️Arrived in New York at earlier time than it left London (banker folk loved this) ▫️Delta wings and nose tip that could point down (so pilots could see runway, because it had to land at specific angle) Jointly run by British Airways and Air France, there were only 14 Concordes put to work (flew 50,000 total flights). What was wrong with the business model? ▫️SMALL MARKET: The thin aerodynamic design could only seat 109, so tickets were very expensive: $11k (a Boeing 747 can do >800 passengers) ▫️HUGE MAINTENANCE: Planes got so hot at the max height (110 degrees), that they expanded 30cm and sealant for fuel hardened. It took 28 hours to turnaround the plane (a normal one can do <2 hours). ▫️OUTRAGEOUS FUEL NEEDS: Each flight required 28,000 litres, for max 109 passengers. Commmercial planes needed 4x less fuel on a per passenger basis. ▫️NOISE RESTRICTIONS: Many cities wouldn’t allow the Concorde to fly because of how glass-shatteringly long it is on take-off, which limited destinations. *** Development of the Concorde — it had an “e” to placate the French — cost $2.8B and was paid for by the UK and French governments. The airlines were actually able to fly profitably for a number of years. But the economics were warped because the planes were given to them for free and the airlines didn’t have to capitalize costs. The business ultimately shuttered in 2003 following a tragic crash in 2000 and industry-wide slowdown post-9/11. Will we ever have sub-4 hour flights from London to NY again? A batch of sonic plane startups could make it happen by the end of this decade.

Trung Phan

3,100,805 次观看 • 2 年前

$AMD $5 Trillion MC Is Inevitable Long Term👑 This thread will focus more on Inference! 2026 EPYC "Venice" $TSM 2nm to save Large GW Scale Inference by 40% more than Prior Turin gen. Context: EPYC Turin achieves ~$0.001 per million tokens for batch inference vs $0.02-$0.12/ million tokens as I wrote the thread below. Venice is going to lower cost down to $0.0005-$0.0006/Million Tokens. OpenAI spent roughly $20B on Inference and Training, where 80-90% of that was for Inference per Analysts. AKA Renting Compute is Expensive AF! In this thread, I want to focus on why most analysts and investors are underestimating the role EPYC "Venice" and future Gen on overall Data center revenue. And $TSM ramping up 2nm supply early is a confirmation that AMD will be a major buyer long term. I will also link the thread the Gap between AMD Analysts & Reality and 2nm Ramp Thread so you have more comprehensive view of what I'm writing here. Before I go into detail this is my 2026 Projection: AI GPUs: $35-$50B EPYC Data Center: $15B-$17B Client Segment: $12-$13B Gaming: $6B Embedded: $4B-$5B Total Revenue $70-$100B Non-GAAP net income $18B-$25B Non-GAAP EPS $10.97-$15.40 Foward P/E 55x-70x= $603-$1,078 AMD's Analysts are projecting $0 Revenue for MI450 and sluggish EPYC Growth. Meaning, all analysts are either full of 💩 or Sexist, you decide! Analysts are also projecting 0% growth on AMD "Secret Weapon" Chip as $MSFT said we are at significant Windows refresh and upgrade cycle. Do you think TSMC would allocate more 2nm supply to $AMD at $0 MI450 revenue and sluggish EPYC? 1. EPYC is going to be the leader in lowest Inference! Current Turin cost saving is 95% vs $NVDA or 98-99% on Inference cost when you factor in renting Inference compute from Amazon Web Services, Microsoft Azure, or $NVDA Neocloud pets. TSMC claimed: 10-15% higher performance at iso-power, 25-30% lower power at iso-speed, and ~15% higher transistor density compared to 3nm. This reduces operational expenses (energy, cooling) while increasing throughput per chip. EPYC Turin achieves ~$0.001 per million tokens for batch inference (via vLLM on models like Llama 3 70B), driven by high core counts and low hardware costs. EPYC Venice offers ~1.7x overall performance and up to 70% more compute capability per core, with up to 256 cores (512 threads). Enhanced vector/AI instructions and open-source firmware (openSIL) optimize for inference workloads. AMD Incorporates AI Engines (now part of AMD's XDNA) for on-chip acceleration, improving efficiency for low-latency and edge inference. This reduces reliance on discrete GPUs, lowering system complexity and TCO. Venice SKUs are projected at $3,000-$15,000 ($5,000 for 256-core flagship), far below NVIDIA Rubin ($50,000-$90,000) or AMD's own MI450 GPUs ($40,000-$50,000). High memory bandwidth (up to 1.6 TB/s) supports efficient batch inference. Venice is designed exactly for Large customers that want to lower Inference Cost and MI450 Helios is for Customers that want Training at lowest TCO, TDP as well as lower Upfront 1GW scale(Full build $35-$40B vs $NVDA $55B-$80B). 2. Real World Example: OpenAI's 2025 inference spend reached ~$20B, escalating to even higher total compute rental (mostly inference) amid token volume growth(from video generating). By 2026, with usage doubling (consistent with industry trends: token demand grows 2-5x YoY), assume OpenAI processes ~1,800 billion million-tokens annually $NVDA Blackwell at $0.02-$0.12 is $36B(most optimized) Rubin is projected to be at $0.01/million tokens or $18B annual Inference Cost vs $AMD Venice $0.0005/million tokens or $0.9B annual Inference Cost => Massive saving for OpenAI or anyone that are paying 80-90% Annual Bill for Inference compute. In short, it is unsustainable to pay this much rent vs owning for all current AI players for the medium to long term. Rubin excels in low-latency decode (if Groq integration from $20B deal in 2027-2028), but Venice dominates batch (80% of inference by 2030). Actual savings depend on deployment scale (OpenAI's 6GW AMD plans), electricity rates, and software maturity. If Rubin only hits $0.03, savings swell to $53.1B vs. $17.1B. 3. Will running Inference on Venice and future Gen slow down response generation in 2026 and beyond? Human perception of "fast enough" for chat, agents, search augmentation, summarization, coding assistance is roughly Meaning, EPYC may generate $100B a year on data center revenue, Hence $MSFT $AMZN $META $GOOGL OpenAI xAI and 42+ Countries are leaning AMD for Inference, because the cost saving is MASSIVE! 4. Regular users (you, me, people using ChatGPT, Claude, Gemini, Grok, Perplexity...) are extremely unlikely to notice any slowdown and in many cases might even experience slightly faster or more consistent response times if the industry heavily shifts toward AMD EPYC for inference. What actually happens when companies save massively on inference? When OpenAI , Anthropic , Gemini , Grok Meta .... save billions on the batch/enterprise/RAG layer using EPYC Venice, they typically do one or more of these things with the savings, none of which make your chat slower but enhancing their bottom line(Profit) ~Keep prices the same → make more profit ~Lower subscription prices / increase free tier limits ~Train bigger & better models more frequently ~Offer longer context windows ~Add more reasoning steps / tool calls / agents per query ~Improve multimodal capabilities ~Build more data centers / reduce throttling during peaks In practice the consumer experience usually gets better, not worse, when inference becomes dramatically cheaper. Prime example is $META leaning AMD heavily or currently AMD largest customer. or Grok 2 to Grok 3 heavily used AMD for Inference saving. And most Grok Users reported Groke responses snappier, not slower. 5. What does this mean for potential Revenue? Noted that TSMC is massively ramping 2nm supply for $AMD both MI450 and EPYC. EPYC Conservative projection: FY2025: $10.5B(best Est) FY2026: $16B FY2027: $29B FY2028: $49B FY2029: $75B FY2030: $100B Large customers: $META OpenAI $MSFT $AMZN $GOOGL xAI (Apple?) Smaller customer: $DELL $HPE $SMCI and 42+ other countries. The roadmap to $5 Trillion is very much inevitable as Inference Cost from Renting or owning $NVDA are too high, but $NVDA will still dominate Training market share, where MI families are likely to take 15-20% market share, but the TAM is also expanding Rapidly. Most Institutions are projecting $2-$3Trillion TAM by 2030. $NVDA said $4 Trillion. Dr. Lisa Su said $1 Trillion+ by 2030. So you decide on how much TAM. If you enjoy this kind of analysis, Slap the Like/Repost and Bookmark to please the X Algo as it is Free.99! If you want to support my work further, consider subscribe to see more in-depth analysis! Alright, that is it. Not Financial Advice!
3:09

$AMD $5 Trillion MC Is Inevitable Long Term👑 This thread will focus more on Inference! 2026 EPYC "Venice" $TSM 2nm to save Large GW Scale Inference by 40% more than Prior Turin gen. Context: EPYC Turin achieves ~$0.001 per million tokens for batch inference vs $0.02-$0.12/ million tokens as I wrote the thread below. Venice is going to lower cost down to $0.0005-$0.0006/Million Tokens. OpenAI spent roughly $20B on Inference and Training, where 80-90% of that was for Inference per Analysts. AKA Renting Compute is Expensive AF! In this thread, I want to focus on why most analysts and investors are underestimating the role EPYC "Venice" and future Gen on overall Data center revenue. And $TSM ramping up 2nm supply early is a confirmation that AMD will be a major buyer long term. I will also link the thread the Gap between AMD Analysts & Reality and 2nm Ramp Thread so you have more comprehensive view of what I'm writing here. Before I go into detail this is my 2026 Projection: AI GPUs: $35-$50B EPYC Data Center: $15B-$17B Client Segment: $12-$13B Gaming: $6B Embedded: $4B-$5B Total Revenue $70-$100B Non-GAAP net income $18B-$25B Non-GAAP EPS $10.97-$15.40 Foward P/E 55x-70x= $603-$1,078 AMD's Analysts are projecting $0 Revenue for MI450 and sluggish EPYC Growth. Meaning, all analysts are either full of 💩 or Sexist, you decide! Analysts are also projecting 0% growth on AMD "Secret Weapon" Chip as $MSFT said we are at significant Windows refresh and upgrade cycle. Do you think TSMC would allocate more 2nm supply to $AMD at $0 MI450 revenue and sluggish EPYC? 1. EPYC is going to be the leader in lowest Inference! Current Turin cost saving is 95% vs $NVDA or 98-99% on Inference cost when you factor in renting Inference compute from Amazon Web Services, Microsoft Azure, or $NVDA Neocloud pets. TSMC claimed: 10-15% higher performance at iso-power, 25-30% lower power at iso-speed, and ~15% higher transistor density compared to 3nm. This reduces operational expenses (energy, cooling) while increasing throughput per chip. EPYC Turin achieves ~$0.001 per million tokens for batch inference (via vLLM on models like Llama 3 70B), driven by high core counts and low hardware costs. EPYC Venice offers ~1.7x overall performance and up to 70% more compute capability per core, with up to 256 cores (512 threads). Enhanced vector/AI instructions and open-source firmware (openSIL) optimize for inference workloads. AMD Incorporates AI Engines (now part of AMD's XDNA) for on-chip acceleration, improving efficiency for low-latency and edge inference. This reduces reliance on discrete GPUs, lowering system complexity and TCO. Venice SKUs are projected at $3,000-$15,000 ($5,000 for 256-core flagship), far below NVIDIA Rubin ($50,000-$90,000) or AMD's own MI450 GPUs ($40,000-$50,000). High memory bandwidth (up to 1.6 TB/s) supports efficient batch inference. Venice is designed exactly for Large customers that want to lower Inference Cost and MI450 Helios is for Customers that want Training at lowest TCO, TDP as well as lower Upfront 1GW scale(Full build $35-$40B vs $NVDA $55B-$80B). 2. Real World Example: OpenAI's 2025 inference spend reached ~$20B, escalating to even higher total compute rental (mostly inference) amid token volume growth(from video generating). By 2026, with usage doubling (consistent with industry trends: token demand grows 2-5x YoY), assume OpenAI processes ~1,800 billion million-tokens annually $NVDA Blackwell at $0.02-$0.12 is $36B(most optimized) Rubin is projected to be at $0.01/million tokens or $18B annual Inference Cost vs $AMD Venice $0.0005/million tokens or $0.9B annual Inference Cost => Massive saving for OpenAI or anyone that are paying 80-90% Annual Bill for Inference compute. In short, it is unsustainable to pay this much rent vs owning for all current AI players for the medium to long term. Rubin excels in low-latency decode (if Groq integration from $20B deal in 2027-2028), but Venice dominates batch (80% of inference by 2030). Actual savings depend on deployment scale (OpenAI's 6GW AMD plans), electricity rates, and software maturity. If Rubin only hits $0.03, savings swell to $53.1B vs. $17.1B. 3. Will running Inference on Venice and future Gen slow down response generation in 2026 and beyond? Human perception of "fast enough" for chat, agents, search augmentation, summarization, coding assistance is roughly Meaning, EPYC may generate $100B a year on data center revenue, Hence $MSFT $AMZN $META $GOOGL OpenAI xAI and 42+ Countries are leaning AMD for Inference, because the cost saving is MASSIVE! 4. Regular users (you, me, people using ChatGPT, Claude, Gemini, Grok, Perplexity...) are extremely unlikely to notice any slowdown and in many cases might even experience slightly faster or more consistent response times if the industry heavily shifts toward AMD EPYC for inference. What actually happens when companies save massively on inference? When OpenAI , Anthropic , Gemini , Grok Meta .... save billions on the batch/enterprise/RAG layer using EPYC Venice, they typically do one or more of these things with the savings, none of which make your chat slower but enhancing their bottom line(Profit) ~Keep prices the same → make more profit ~Lower subscription prices / increase free tier limits ~Train bigger & better models more frequently ~Offer longer context windows ~Add more reasoning steps / tool calls / agents per query ~Improve multimodal capabilities ~Build more data centers / reduce throttling during peaks In practice the consumer experience usually gets better, not worse, when inference becomes dramatically cheaper. Prime example is $META leaning AMD heavily or currently AMD largest customer. or Grok 2 to Grok 3 heavily used AMD for Inference saving. And most Grok Users reported Groke responses snappier, not slower. 5. What does this mean for potential Revenue? Noted that TSMC is massively ramping 2nm supply for $AMD both MI450 and EPYC. EPYC Conservative projection: FY2025: $10.5B(best Est) FY2026: $16B FY2027: $29B FY2028: $49B FY2029: $75B FY2030: $100B Large customers: $META OpenAI $MSFT $AMZN $GOOGL xAI (Apple?) Smaller customer: $DELL $HPE $SMCI and 42+ other countries. The roadmap to $5 Trillion is very much inevitable as Inference Cost from Renting or owning $NVDA are too high, but $NVDA will still dominate Training market share, where MI families are likely to take 15-20% market share, but the TAM is also expanding Rapidly. Most Institutions are projecting $2-$3Trillion TAM by 2030. $NVDA said $4 Trillion. Dr. Lisa Su said $1 Trillion+ by 2030. So you decide on how much TAM. If you enjoy this kind of analysis, Slap the Like/Repost and Bookmark to please the X Algo as it is Free.99! If you want to support my work further, consider subscribe to see more in-depth analysis! Alright, that is it. Not Financial Advice!

Mike

102,223 次观看 • 7 个月前

I’ve been using GPT-5.6 Sol internally for the past two months, I've spent probably 25+ billion tokens. Here’s my review and comparison to Fable 5: > Let's start with the analogy because everyone seems to be giving theirs - GPT-5.6 is likely the last version of the GPT-5 training run series. It's kind of like an athlete at their peak. Through years of experience in the game, they've become the most reliable player and has the highest game IQ. But, there's no more room to grow. Fable on the other hand, being essentially the first version of a new training run, is the first round draft pick rookie. Raw talent mixed with the energy only a young person would have results in some incredible plays we didn't think possible, but also mistakes due to lack of experience. But that rookie will only improve and likely will be better than the veteran ever was because it's a new game and a new era. > GPT-5.6 is genuinely better at long, sustained work. With /goal, I've had it running complex projects for days with almost no intervention. It built a Minecraft-style game, kept adding features and mobs after the core game worked, and only stopped because I stopped the run. I never felt as though I had to jump in and guide it back to the right path. > It keeps finding useful work when you give it a concrete finish line. I had it recreate Excel with a loop. It inspected the real desktop excel app with Computer Use, comparing that against its own build, and closing the gaps. I stopped it after six days after it had built an incredible amount of functionality. > It's faster than other models in two different ways. The raw generation speed is higher, something OpenAI has been putting effort into. But it also takes a shorter path to solutions. It wanders less, changes less code, and generally knows how to get things done directly. In daily use, it feels about 2-3x times faster than Fable. That's my impression, not a controlled benchmark. The difference is large enough that I notice it constantly. > It works well across a wide range of tasks. I use it for one-line edits, quick questions, browser chores, and multi-day builds without changing my prompting style. Speaking of browser control, its the best ever I've used. To the point where I actually use it often. If a task lives on a website, GPT-5.6 usually opens the browser and does it there instead of asking for an API key or forcing everything through the terminal. When I switched back to GPT-5.5, it went straight to the command line even when the browser was clearly the better tool. > And it can handle real browser work, not just toy demos. During a data import, I had it monitor Supabase and resize instances as the load changed. It stayed on the dashboard, adjusted capacity, and checked the result without an API or a custom script. > I also gave it a full Google Workspace migration. It moved Forward Future from to preserved the old aliases, and configured MX, SPF, and DKIM. Before a consequential save, it stopped, explained exactly what would change, and waited for confirmation. > The reasoning setting matters a lot. Light is good for questions and small edits. High and Extra High are the sweet spots for serious work. Ultra usually takes longer than the extra thinking is worth and burns tokens. > I love that 5.6 is split into 3 sizes. Not only can you control speed and cost that way, but you still also have the thinking effort setting for each of them. Very precise controls. I just wish Codex automatically routed my prompts for me. > Its personality is blunt and a little bland. Claude feels warmer and more natural to talk to. GPT-5.6 is more clinical, but I like that for work. It gives me enough explanation and rarely pads the answer. I usually have to ask Fable to explain things more simply and/or more concise. > Its front-end taste has improved, but the default is predictable. Left alone, it turns websites into PowerPoint decks with huge statements and hard section breaks. The good news is that it takes design direction well and can revise without destroying the parts that already work. > It still makes confident mistakes. I asked it to rebuild parts of a system, and it told me the job was finished. Later, I found out it wasn't. Bits of its internal process also leak into the answer occasionally. > Claude Fable is more naturally autonomous on large, open-ended projects. GPT-5.6 is easier to reach for. I don't need to invent a huge project to justify using it. It works just as well for a small edit or browser chore. > GPT-5.6 is also cheaper. Sol costs $5 per million input tokens and $30 per million output tokens. Fable costs $10 and $50. Cached input is cheaper too. Still, cost per finished task matters more than cost per token. > GPT-5.6 isn't the best at everything, and it still needs supervision. But it generates faster, wanders less, works at almost any scale, and wastes less of my time. It's the model I have the most confidence in to get the job done right the first time. I put together a full breakdown with all the tests, prompts, and examples on a site. You can read it here:
7:46

I’ve been using GPT-5.6 Sol internally for the past two months, I've spent probably 25+ billion tokens. Here’s my review and comparison to Fable 5: > Let's start with the analogy because everyone seems to be giving theirs - GPT-5.6 is likely the last version of the GPT-5 training run series. It's kind of like an athlete at their peak. Through years of experience in the game, they've become the most reliable player and has the highest game IQ. But, there's no more room to grow. Fable on the other hand, being essentially the first version of a new training run, is the first round draft pick rookie. Raw talent mixed with the energy only a young person would have results in some incredible plays we didn't think possible, but also mistakes due to lack of experience. But that rookie will only improve and likely will be better than the veteran ever was because it's a new game and a new era. > GPT-5.6 is genuinely better at long, sustained work. With /goal, I've had it running complex projects for days with almost no intervention. It built a Minecraft-style game, kept adding features and mobs after the core game worked, and only stopped because I stopped the run. I never felt as though I had to jump in and guide it back to the right path. > It keeps finding useful work when you give it a concrete finish line. I had it recreate Excel with a loop. It inspected the real desktop excel app with Computer Use, comparing that against its own build, and closing the gaps. I stopped it after six days after it had built an incredible amount of functionality. > It's faster than other models in two different ways. The raw generation speed is higher, something OpenAI has been putting effort into. But it also takes a shorter path to solutions. It wanders less, changes less code, and generally knows how to get things done directly. In daily use, it feels about 2-3x times faster than Fable. That's my impression, not a controlled benchmark. The difference is large enough that I notice it constantly. > It works well across a wide range of tasks. I use it for one-line edits, quick questions, browser chores, and multi-day builds without changing my prompting style. Speaking of browser control, its the best ever I've used. To the point where I actually use it often. If a task lives on a website, GPT-5.6 usually opens the browser and does it there instead of asking for an API key or forcing everything through the terminal. When I switched back to GPT-5.5, it went straight to the command line even when the browser was clearly the better tool. > And it can handle real browser work, not just toy demos. During a data import, I had it monitor Supabase and resize instances as the load changed. It stayed on the dashboard, adjusted capacity, and checked the result without an API or a custom script. > I also gave it a full Google Workspace migration. It moved Forward Future from to preserved the old aliases, and configured MX, SPF, and DKIM. Before a consequential save, it stopped, explained exactly what would change, and waited for confirmation. > The reasoning setting matters a lot. Light is good for questions and small edits. High and Extra High are the sweet spots for serious work. Ultra usually takes longer than the extra thinking is worth and burns tokens. > I love that 5.6 is split into 3 sizes. Not only can you control speed and cost that way, but you still also have the thinking effort setting for each of them. Very precise controls. I just wish Codex automatically routed my prompts for me. > Its personality is blunt and a little bland. Claude feels warmer and more natural to talk to. GPT-5.6 is more clinical, but I like that for work. It gives me enough explanation and rarely pads the answer. I usually have to ask Fable to explain things more simply and/or more concise. > Its front-end taste has improved, but the default is predictable. Left alone, it turns websites into PowerPoint decks with huge statements and hard section breaks. The good news is that it takes design direction well and can revise without destroying the parts that already work. > It still makes confident mistakes. I asked it to rebuild parts of a system, and it told me the job was finished. Later, I found out it wasn't. Bits of its internal process also leak into the answer occasionally. > Claude Fable is more naturally autonomous on large, open-ended projects. GPT-5.6 is easier to reach for. I don't need to invent a huge project to justify using it. It works just as well for a small edit or browser chore. > GPT-5.6 is also cheaper. Sol costs $5 per million input tokens and $30 per million output tokens. Fable costs $10 and $50. Cached input is cheaper too. Still, cost per finished task matters more than cost per token. > GPT-5.6 isn't the best at everything, and it still needs supervision. But it generates faster, wanders less, works at almost any scale, and wastes less of my time. It's the model I have the most confidence in to get the job done right the first time. I put together a full breakdown with all the tests, prompts, and examples on a site. You can read it here:

Matthew Berman

186,712 次观看 • 24 天前

AGI? One day, but not yet. The only AI that works well right now is the one behind the screen [12-17]. But passing the Turing Test [9] behind a screen is easy compared to Real AI for real robots in the real world. No current AI-driven robot could be certified as a plumber [13-17]. Hence, the Turing Test isn't a good measure of intelligence (and neither is IQ). And AGI without mastery of the physical world is no AGI. That’s why I created the TUM CogBotLab for learning robots in 2004 [5], co-founded a company for AI in the physical world in 2014 [6], and had teams at TUM, IDSIA, and now KAUST work towards baby robots [4,10-11,18]. Such soft robots don't just slavishly imitate humans and they don't work by just downloading the web like LLMs/VLMs. No. Instead, they exploit the principles of Artificial Curiosity to improve their neural World Models (two terms I used back in 1990 [1-4]). These robots work with lots of sensors, but only weak actuators, such that they cannot easily harm themselves [18] when they collect useful data by devising and running their own self-invented experiments. Remarkably, since the 1970s, many have made fun of my old goal to build a self-improving AGI smarter than myself and then retire. Recently, however, many have finally started to take this seriously, and now some of them are suddenly TOO optimistic. These people are often blissfully unaware of the remaining challenges we have to solve to achieve Real AI. My 2024 TED talk [15] summarises some of that. REFERENCES (easy to find on the web): [1] J. Schmidhuber. Making the world differentiable: On using fully recurrent self-supervised neural networks (NNs) for dynamic reinforcement learning and planning in non-stationary environments. TR FKI-126-90, TUM, Feb 1990, revised Nov 1990. This paper also introduced artificial curiosity and intrinsic motivation through generative adversarial networks where a generator NN is fighting a predictor NN in a minimax game. [2] J. S. A possibility for implementing curiosity and boredom in model-building neural controllers. In J. A. Meyer and S. W. Wilson, editors, Proc. of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats, pages 222-227. MIT Press/Bradford Books, 1991. Based on [1]. [3] J.S. AI Blog (2020). 1990: Planning & Reinforcement Learning with Recurrent World Models and Artificial Curiosity. Summarising aspects of [1][2] and lots of later papers including [7][8]. [4] J.S. AI Blog (2021): Artificial Curiosity & Creativity Since 1990. Summarising aspects of [1][2] and lots of later papers including [7][8]. [5] J.S. TU Munich CogBotLab for learning robots (2004-2009) [6] NNAISENSE, founded in 2014, for AI in the physical world [7] J.S. (2015). On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning (RL) Controllers and Recurrent Neural World Models. arXiv 1210.0118. Sec. 5.3 describes an RL prompt engineer which learns to query its model for abstract reasoning and planning and decision making. Today this is called "chain of thought." [8] J.S. (2018). One Big Net For Everything. arXiv 1802.08864. See also patent US11853886B2 and my DeepSeek tweet: DeepSeek uses elements of the 2015 reinforcement learning prompt engineer [7] and its 2018 refinement [8] which collapses the RL machine and world model of [7] into a single net. This uses my neural net distillation procedure of 1991: a distilled chain of thought system. [9] J.S. Turing Oversold. It's not Turing's fault, though. AI Blog (2021, was #1 on Hacker News) [10] J.S. Intelligente Roboter werden vom Leben fasziniert sein. (Intelligent robots will be fascinated by life.) F.A.Z., 2015 [11] J.S. at Falling Walls: The Past, Present and Future of Artificial Intelligence. Scientific American, Observations, 2017. [12] J.S. KI ist eine Riesenchance für Deutschland. (AI is a huge chance for Germany.) F.A.Z., 2018 [13] H. Jones. J.S. Says His Life's Work Won't Lead To Dystopia. Forbes Magazine, 2023. [14] Interview with J.S. Jazzyear, Shanghai, 2024. [15] J.S. TED talk at TED AI Vienna (2024): Why 2042 will be a big year for AI. See the attached video clip. [16] J.S. Baut den KI-gesteuerten Allzweckroboter! (Build the AI-controlled all-purpose robot!) F.A.Z., 2024 [17] J.S. 1995-2025: The Decline of Germany & Japan vs US & China. Can All-Purpose Robots Fuel a Comeback? AI Blog, Jan 2025, based on [16]. [18] M. Alhakami, D. R. Ashley, J. Dunham, Y. Dai, F. Faccio, E. Feron, J. Schmidhuber. Towards an Extremely Robust Baby Robot With Rich Interaction Ability for Advanced Machine Learning Algorithms. Preprint arxiv 2404.08093, 2024.
1:41

AGI? One day, but not yet. The only AI that works well right now is the one behind the screen [12-17]. But passing the Turing Test [9] behind a screen is easy compared to Real AI for real robots in the real world. No current AI-driven robot could be certified as a plumber [13-17]. Hence, the Turing Test isn't a good measure of intelligence (and neither is IQ). And AGI without mastery of the physical world is no AGI. That’s why I created the TUM CogBotLab for learning robots in 2004 [5], co-founded a company for AI in the physical world in 2014 [6], and had teams at TUM, IDSIA, and now KAUST work towards baby robots [4,10-11,18]. Such soft robots don't just slavishly imitate humans and they don't work by just downloading the web like LLMs/VLMs. No. Instead, they exploit the principles of Artificial Curiosity to improve their neural World Models (two terms I used back in 1990 [1-4]). These robots work with lots of sensors, but only weak actuators, such that they cannot easily harm themselves [18] when they collect useful data by devising and running their own self-invented experiments. Remarkably, since the 1970s, many have made fun of my old goal to build a self-improving AGI smarter than myself and then retire. Recently, however, many have finally started to take this seriously, and now some of them are suddenly TOO optimistic. These people are often blissfully unaware of the remaining challenges we have to solve to achieve Real AI. My 2024 TED talk [15] summarises some of that. REFERENCES (easy to find on the web): [1] J. Schmidhuber. Making the world differentiable: On using fully recurrent self-supervised neural networks (NNs) for dynamic reinforcement learning and planning in non-stationary environments. TR FKI-126-90, TUM, Feb 1990, revised Nov 1990. This paper also introduced artificial curiosity and intrinsic motivation through generative adversarial networks where a generator NN is fighting a predictor NN in a minimax game. [2] J. S. A possibility for implementing curiosity and boredom in model-building neural controllers. In J. A. Meyer and S. W. Wilson, editors, Proc. of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats, pages 222-227. MIT Press/Bradford Books, 1991. Based on [1]. [3] J.S. AI Blog (2020). 1990: Planning & Reinforcement Learning with Recurrent World Models and Artificial Curiosity. Summarising aspects of [1][2] and lots of later papers including [7][8]. [4] J.S. AI Blog (2021): Artificial Curiosity & Creativity Since 1990. Summarising aspects of [1][2] and lots of later papers including [7][8]. [5] J.S. TU Munich CogBotLab for learning robots (2004-2009) [6] NNAISENSE, founded in 2014, for AI in the physical world [7] J.S. (2015). On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning (RL) Controllers and Recurrent Neural World Models. arXiv 1210.0118. Sec. 5.3 describes an RL prompt engineer which learns to query its model for abstract reasoning and planning and decision making. Today this is called "chain of thought." [8] J.S. (2018). One Big Net For Everything. arXiv 1802.08864. See also patent US11853886B2 and my DeepSeek tweet: DeepSeek uses elements of the 2015 reinforcement learning prompt engineer [7] and its 2018 refinement [8] which collapses the RL machine and world model of [7] into a single net. This uses my neural net distillation procedure of 1991: a distilled chain of thought system. [9] J.S. Turing Oversold. It's not Turing's fault, though. AI Blog (2021, was #1 on Hacker News) [10] J.S. Intelligente Roboter werden vom Leben fasziniert sein. (Intelligent robots will be fascinated by life.) F.A.Z., 2015 [11] J.S. at Falling Walls: The Past, Present and Future of Artificial Intelligence. Scientific American, Observations, 2017. [12] J.S. KI ist eine Riesenchance für Deutschland. (AI is a huge chance for Germany.) F.A.Z., 2018 [13] H. Jones. J.S. Says His Life's Work Won't Lead To Dystopia. Forbes Magazine, 2023. [14] Interview with J.S. Jazzyear, Shanghai, 2024. [15] J.S. TED talk at TED AI Vienna (2024): Why 2042 will be a big year for AI. See the attached video clip. [16] J.S. Baut den KI-gesteuerten Allzweckroboter! (Build the AI-controlled all-purpose robot!) F.A.Z., 2024 [17] J.S. 1995-2025: The Decline of Germany & Japan vs US & China. Can All-Purpose Robots Fuel a Comeback? AI Blog, Jan 2025, based on [16]. [18] M. Alhakami, D. R. Ashley, J. Dunham, Y. Dai, F. Faccio, E. Feron, J. Schmidhuber. Towards an Extremely Robust Baby Robot With Rich Interaction Ability for Advanced Machine Learning Algorithms. Preprint arxiv 2404.08093, 2024.

Jürgen Schmidhuber

72,331 次观看 • 1 年前

$AMD is easily a $1,200 stock IMO| CPUs TAM 🧵 Not Financial Advice! DYOR! In this thread, I want to discuss the actual TAM for CPUs data center for just 2026, where many are giving different ranges, where I don't agree with. I will explain in detail why I disagree with these research firms and financial analysts using Math. And this thread should not be treated as Financial Advice. I'm just explaining my research and thought process so we can have a discussion. In 2024/2025, I gave out $620 PT for FY2026 was too conservative for AMD potential. At the time, It was early and many were just laughing, that PT was unrealistic and the AI world is run on GPUs only. Today, most of these folks are laughing with me. That is ok, I dont offer financial advice, and I do not need everyone to agree with me. I respect other opinions. If you enjoy this kind of thread, slap the like/repost/bookmark. If you want to support my work further and gain more in-depth analysis, consider subscribe! In early 2026, hyperscalers, enterprises, and OEMs are scrambling as Intel and AMD server CPUs are largely sold out for the year, with prices jumping 10–20% and lead times stretching from weeks to months (or longer for certain SKUs). What was once a GPU dominated story has flipped: the shift to explosive Agentic AI with its multi-step reasoning loops, tool calling, multi-agent orchestration, real-time data movement, and reinforcement learning, is dramatically tightening CPU:GPU ratios from the old training-era 1:4–8 all the way to 1:1 to 5:1 or even CPU-heavy configurations. CEOs across NVIDIA, AMD, Intel, Google, Meta, Microsoft, and public companies have been sounding the alarm on CNBC, Bloomberg, and earnings calls. CPUs are “cool again,” and in many agentic deployments they are becoming the new bottleneck alongside (or even ahead of) GPUs and custom ASICs. In 2025, roughly 12-15m AI GPUs + AI ASICs GPUs shipped, and is expect to be 15-20m units by 2026, where it suggesting Training demand is not going away. The actual TAM is structural, multiplicative demand that has already forced AMD to double its long-term server CPU TAM forecast to >$120 billion by 2030 (>35% CAGR), with Dr. Lisa Su noting Q2 2026 server CPU sales expected to surge 70%+ year-over-year and demand “far exceeding expectations.” At the same time, AMD’s secured 30–40% share of TSMC’s initial 2nm capacity (behind only Apple’s >50%) positions it to ramp Zen 6-based EPYC Venice exactly when this agentic wave hits hardest but even that aggressive five-fab 2nm expansion (with plans scaling toward 11 total advanced facilities) cannot instantly close the gap in the near-term. Supply constraints on wafers, advanced packaging, and power are compounding the squeeze, just as hyperscalers forward-buy and lock in long-term deals. 1. The actual potential TAM Various sources and institutions are giving $50-$160-$200B CPUs TAM toward 2030, and i disagree, where supply is severely behind vs Demand by at least 2-3 years or even longer by some estimates. The actual TAM will probably be 15-20m for FY2026. The typical average selling price from low to high end is $5,000 to $15,000, but due to rising memory, and different inflationary pressures on Semi, it would be more logical to think between $7,000-17,000. A. CPU:GPU Ratio at 1:1 A basic calucation at mid range =12,000 x 15-20m CPUs= $180-$240B TAM B. CPU:GPU Ratio at 5:1 = $12,000 x 75m-100m CPUs= $900B-$1.2T TAM Of course TSMC cannot even supply 20% of this massive inflection TAM in 2026. But do we think of Demand for TAM or Supply for TAM? Hence we are seeing massive 2nm Ramp from TSMC for $AMD. IMO, conservatively, I would take down 15-20% on 1:1 or $135-$192B TAM for just 2026. Im not even talking about 2030. We are just months into this, it is impossible to estimate Cagr atm, but this is 1-5 agents running tasks, I wrote a thread on 24/7 autonomous agents thread, where companies could use 50-250 agents to run tasks for them 24/7. It would require a different structural CPU:GPU to bring down the cost of token as well as handling the Orchestration bottleneck. GPUs would be useless and sit idle waiting for CPU due to highly CPU-intensive nature. The cost per Million tokens must come down more rapidly for this 50-250 autonomous agents to work, otherwise the token cost would be too enormous. Helios Rack is estimated to bring inference cost down to $0.0003-$0.0005/M tokens with 18 EPYC Venices along with 72 MI455x and other chips+ Components. A heavier or CPUs dense rack would bring down inference cost further. EPYC Verano(2027 gen 7 AI-optimized) is expected to drive inference costs meaningfully lower than the Venice baseline likely to the $0.00002–$0.00025 per million tokens range (or even sub-$0.00015 in highly optimized agentic/batch workloads). Verano have higher core counts than Venice, LPDDR5X SOCAMM2 memory support, more AI optimized and Next-Gen rack density & efficiency. 2. $AMD secured at least 30-40% of TSMC 2nm capacity and Memory from Samsung through 2028-2030. 2 2nm fabs are entering ramping phase toward 60-65k wafers per months and 5 dedicated 2nm fabs entering mass production/ramp in 2026. Will link sub threads below if you are interest for full detail. Apple is reported to secure 50%+ 2nm capacity for Iphone 18 and Mac chips and AMD secured at least 30-40% capacity while $NVDA $AVGO $ARM $AMZN $GOOGL and others are on 3nm. This broader aggressive ramp from TSMC to target up to 11 fabs is to address $AMD massive growth ahead. Where $ARM is facing massive CPUs supply constraints as they have to compete with other Mega Cap players on 3nm allocation. And $INTC is also facing supply constraints for data center CPUs and PC per management with lead times extrended to longer than 12 weeks. Dr. Su is aiming for higher than 50%+ Market share, and I believe it is achievable in 2026 or 2027 as AMD has the strongest CPUs offerings. Dr. Su did not want to take advantage of the shortage and she said during the Q1 earning call, AMD is prioritizing Units shipped while guiding margin to be inching 60%. If Jensen were in charge, I'm sure margin would be 70-75% in this kind of severe CPUs shortage condition. But that is not how Dr. Su operates for more than a decade. She wants most market share. So we will see it in revenue growth, but as TSMC ramps faster and faster, AMD Operating and FCF margin will massively improve vs prior decade. A significantly higher margin profile than before. 3. How I came up with $1,200 withint 12-18 months? At $1,200/ share, that would be around $2 Trillion MC. I expect FY2027 revenue to be $124-$144B where data center revenue dominates overall revenue. AI GPUs: I will stick to the lowest end so show u that I'm conservative at $18B for each GW vs $NVDA Rubin is $30B+ (most likely Helios Rack in the $20B+ due to memory price rising). We know deals with OpenAI and Meta are around 12GW and additional multi-customers at multi-GW scale were hinted and will be revealed as we get to July 22-23 2026 Advancing AI event. For now I will conservatively add a bit more to this model. (3-6GW Helios Rack Range) EPYC Venice is reported to be in $15,000-$20,000. However large customers will likely to enjoy $10-$12k discount. I expect AMD to be able to ramp 7m EPYC Venice for entire 2026 and 3-4m of EPYC Verano(higher price than Venice). If we take an average selling price of $10,000 to be on the conservative side. Take down another 30% to be even more conservative on projection. I like to be conservative. That would be ~ 7m EPYC CPUs(Venice + Verano) for FY2027 or 583,000 units per month or 15,000 additional 2nm wafers per month which is completely reasonable for current TSMC Ramp, and I may be too conservative here. EPYC Verano and MI500 series will also be on 2nm. AI GPUs: 3GW x $18B= $54B EPYC CPUs: $10k x 7m CPUs= $70B = Data center revenue alone is $124B Other segments= probably in the $20-$25B FY 2027. FY2027 revenue = $124-$149B At 7m EPYC CPUs for entire 2027, that would be more than 50% market share when we comp it to availability from supply side, not from total Demand. It is possible that TSMC could significantly ramp even more capacity in 2027, so we will see. Metric Q1 2026 FY2027 Gross Margin 55-56% 60-62% Operating Margin 25-26% 32-35% Net Income Margin ~22% 26-30% FCF Margin 25% 28-30% At $124-$149B Revenue FY 2027 Net Income would be $32-$44B EPS would be $20-$27 (GAAP) Non-GAAP would be $25-$31 At $1,200 a share or $2T valuation that would be: 13.4-16x Price to Sales (P/S) 38-48 P/E At this kind of growth of AI SuperCycle, I think it is very reasonable valuation. If we use today at $406/share or $661B MC: 2027 P/S = 4.4x-5.3x 2027 P/E = 13x-16x Is AMD today expensive or cheap to you? Above is already a very conservative where I trimmed 20-30% of doable units. Meaning, there could be upside if TSMC is able to ramp meaningfully like they are planning. Conclusion: A $1,200 per share valuation IMO for AMD in FY2027 is not expensive at all; it is, in fact, conservative when viewed against the structural explosion in agentic AI demand we have mapped out. With server CPU TAM potentially scaling into the $100–$200B+ range in just CPU:GPU 1:1 Ratio for just 2026. AMD positioned to capture 50%+ share thanks to its 2nm TSMC allocation advantage and full-stack leadership, the company could realistically deliver $124–149B in total revenue and $25–$31+ non-GAAP EPS. At those levels, $1,200 implies a 2027 P/E = 13x-16x. Entirely reasonable for a company that will have become the clear Inference Queen (and in many workloads the preferred) AI infrastructure provider, with operating margins expanding above 30% and tens of billions in high-margin rack-scale AI revenue. Dr. Lisa Su was right presciently so about the Agentic AI inflection all the way back to her early 2022–2023 commentary on the coming shift from pure training to inference and orchestration-heavy workloads. While the broader market only fully woke up to this in 2026 when she doubled AMD’s long-term server CPU TAM forecast to >$120B by 2030 (with >35% CAGR), Dr. Su and her team have consistently positioned the company at the center of the CPU renaissance. The explosive demand we are seeing today, sold-out lines, rising ASPs, and hyperscalers forward-buying entire gigawatts of Helios-class systems is exactly the outcome she forecasted years ago. Not Financial Advice! DYOR!
13:03

$AMD is easily a $1,200 stock IMO| CPUs TAM 🧵 Not Financial Advice! DYOR! In this thread, I want to discuss the actual TAM for CPUs data center for just 2026, where many are giving different ranges, where I don't agree with. I will explain in detail why I disagree with these research firms and financial analysts using Math. And this thread should not be treated as Financial Advice. I'm just explaining my research and thought process so we can have a discussion. In 2024/2025, I gave out $620 PT for FY2026 was too conservative for AMD potential. At the time, It was early and many were just laughing, that PT was unrealistic and the AI world is run on GPUs only. Today, most of these folks are laughing with me. That is ok, I dont offer financial advice, and I do not need everyone to agree with me. I respect other opinions. If you enjoy this kind of thread, slap the like/repost/bookmark. If you want to support my work further and gain more in-depth analysis, consider subscribe! In early 2026, hyperscalers, enterprises, and OEMs are scrambling as Intel and AMD server CPUs are largely sold out for the year, with prices jumping 10–20% and lead times stretching from weeks to months (or longer for certain SKUs). What was once a GPU dominated story has flipped: the shift to explosive Agentic AI with its multi-step reasoning loops, tool calling, multi-agent orchestration, real-time data movement, and reinforcement learning, is dramatically tightening CPU:GPU ratios from the old training-era 1:4–8 all the way to 1:1 to 5:1 or even CPU-heavy configurations. CEOs across NVIDIA, AMD, Intel, Google, Meta, Microsoft, and public companies have been sounding the alarm on CNBC, Bloomberg, and earnings calls. CPUs are “cool again,” and in many agentic deployments they are becoming the new bottleneck alongside (or even ahead of) GPUs and custom ASICs. In 2025, roughly 12-15m AI GPUs + AI ASICs GPUs shipped, and is expect to be 15-20m units by 2026, where it suggesting Training demand is not going away. The actual TAM is structural, multiplicative demand that has already forced AMD to double its long-term server CPU TAM forecast to >$120 billion by 2030 (>35% CAGR), with Dr. Lisa Su noting Q2 2026 server CPU sales expected to surge 70%+ year-over-year and demand “far exceeding expectations.” At the same time, AMD’s secured 30–40% share of TSMC’s initial 2nm capacity (behind only Apple’s >50%) positions it to ramp Zen 6-based EPYC Venice exactly when this agentic wave hits hardest but even that aggressive five-fab 2nm expansion (with plans scaling toward 11 total advanced facilities) cannot instantly close the gap in the near-term. Supply constraints on wafers, advanced packaging, and power are compounding the squeeze, just as hyperscalers forward-buy and lock in long-term deals. 1. The actual potential TAM Various sources and institutions are giving $50-$160-$200B CPUs TAM toward 2030, and i disagree, where supply is severely behind vs Demand by at least 2-3 years or even longer by some estimates. The actual TAM will probably be 15-20m for FY2026. The typical average selling price from low to high end is $5,000 to $15,000, but due to rising memory, and different inflationary pressures on Semi, it would be more logical to think between $7,000-17,000. A. CPU:GPU Ratio at 1:1 A basic calucation at mid range =12,000 x 15-20m CPUs= $180-$240B TAM B. CPU:GPU Ratio at 5:1 = $12,000 x 75m-100m CPUs= $900B-$1.2T TAM Of course TSMC cannot even supply 20% of this massive inflection TAM in 2026. But do we think of Demand for TAM or Supply for TAM? Hence we are seeing massive 2nm Ramp from TSMC for $AMD. IMO, conservatively, I would take down 15-20% on 1:1 or $135-$192B TAM for just 2026. Im not even talking about 2030. We are just months into this, it is impossible to estimate Cagr atm, but this is 1-5 agents running tasks, I wrote a thread on 24/7 autonomous agents thread, where companies could use 50-250 agents to run tasks for them 24/7. It would require a different structural CPU:GPU to bring down the cost of token as well as handling the Orchestration bottleneck. GPUs would be useless and sit idle waiting for CPU due to highly CPU-intensive nature. The cost per Million tokens must come down more rapidly for this 50-250 autonomous agents to work, otherwise the token cost would be too enormous. Helios Rack is estimated to bring inference cost down to $0.0003-$0.0005/M tokens with 18 EPYC Venices along with 72 MI455x and other chips+ Components. A heavier or CPUs dense rack would bring down inference cost further. EPYC Verano(2027 gen 7 AI-optimized) is expected to drive inference costs meaningfully lower than the Venice baseline likely to the $0.00002–$0.00025 per million tokens range (or even sub-$0.00015 in highly optimized agentic/batch workloads). Verano have higher core counts than Venice, LPDDR5X SOCAMM2 memory support, more AI optimized and Next-Gen rack density & efficiency. 2. $AMD secured at least 30-40% of TSMC 2nm capacity and Memory from Samsung through 2028-2030. 2 2nm fabs are entering ramping phase toward 60-65k wafers per months and 5 dedicated 2nm fabs entering mass production/ramp in 2026. Will link sub threads below if you are interest for full detail. Apple is reported to secure 50%+ 2nm capacity for Iphone 18 and Mac chips and AMD secured at least 30-40% capacity while $NVDA $AVGO $ARM $AMZN $GOOGL and others are on 3nm. This broader aggressive ramp from TSMC to target up to 11 fabs is to address $AMD massive growth ahead. Where $ARM is facing massive CPUs supply constraints as they have to compete with other Mega Cap players on 3nm allocation. And $INTC is also facing supply constraints for data center CPUs and PC per management with lead times extrended to longer than 12 weeks. Dr. Su is aiming for higher than 50%+ Market share, and I believe it is achievable in 2026 or 2027 as AMD has the strongest CPUs offerings. Dr. Su did not want to take advantage of the shortage and she said during the Q1 earning call, AMD is prioritizing Units shipped while guiding margin to be inching 60%. If Jensen were in charge, I'm sure margin would be 70-75% in this kind of severe CPUs shortage condition. But that is not how Dr. Su operates for more than a decade. She wants most market share. So we will see it in revenue growth, but as TSMC ramps faster and faster, AMD Operating and FCF margin will massively improve vs prior decade. A significantly higher margin profile than before. 3. How I came up with $1,200 withint 12-18 months? At $1,200/ share, that would be around $2 Trillion MC. I expect FY2027 revenue to be $124-$144B where data center revenue dominates overall revenue. AI GPUs: I will stick to the lowest end so show u that I'm conservative at $18B for each GW vs $NVDA Rubin is $30B+ (most likely Helios Rack in the $20B+ due to memory price rising). We know deals with OpenAI and Meta are around 12GW and additional multi-customers at multi-GW scale were hinted and will be revealed as we get to July 22-23 2026 Advancing AI event. For now I will conservatively add a bit more to this model. (3-6GW Helios Rack Range) EPYC Venice is reported to be in $15,000-$20,000. However large customers will likely to enjoy $10-$12k discount. I expect AMD to be able to ramp 7m EPYC Venice for entire 2026 and 3-4m of EPYC Verano(higher price than Venice). If we take an average selling price of $10,000 to be on the conservative side. Take down another 30% to be even more conservative on projection. I like to be conservative. That would be ~ 7m EPYC CPUs(Venice + Verano) for FY2027 or 583,000 units per month or 15,000 additional 2nm wafers per month which is completely reasonable for current TSMC Ramp, and I may be too conservative here. EPYC Verano and MI500 series will also be on 2nm. AI GPUs: 3GW x $18B= $54B EPYC CPUs: $10k x 7m CPUs= $70B = Data center revenue alone is $124B Other segments= probably in the $20-$25B FY 2027. FY2027 revenue = $124-$149B At 7m EPYC CPUs for entire 2027, that would be more than 50% market share when we comp it to availability from supply side, not from total Demand. It is possible that TSMC could significantly ramp even more capacity in 2027, so we will see. Metric Q1 2026 FY2027 Gross Margin 55-56% 60-62% Operating Margin 25-26% 32-35% Net Income Margin ~22% 26-30% FCF Margin 25% 28-30% At $124-$149B Revenue FY 2027 Net Income would be $32-$44B EPS would be $20-$27 (GAAP) Non-GAAP would be $25-$31 At $1,200 a share or $2T valuation that would be: 13.4-16x Price to Sales (P/S) 38-48 P/E At this kind of growth of AI SuperCycle, I think it is very reasonable valuation. If we use today at $406/share or $661B MC: 2027 P/S = 4.4x-5.3x 2027 P/E = 13x-16x Is AMD today expensive or cheap to you? Above is already a very conservative where I trimmed 20-30% of doable units. Meaning, there could be upside if TSMC is able to ramp meaningfully like they are planning. Conclusion: A $1,200 per share valuation IMO for AMD in FY2027 is not expensive at all; it is, in fact, conservative when viewed against the structural explosion in agentic AI demand we have mapped out. With server CPU TAM potentially scaling into the $100–$200B+ range in just CPU:GPU 1:1 Ratio for just 2026. AMD positioned to capture 50%+ share thanks to its 2nm TSMC allocation advantage and full-stack leadership, the company could realistically deliver $124–149B in total revenue and $25–$31+ non-GAAP EPS. At those levels, $1,200 implies a 2027 P/E = 13x-16x. Entirely reasonable for a company that will have become the clear Inference Queen (and in many workloads the preferred) AI infrastructure provider, with operating margins expanding above 30% and tens of billions in high-margin rack-scale AI revenue. Dr. Lisa Su was right presciently so about the Agentic AI inflection all the way back to her early 2022–2023 commentary on the coming shift from pure training to inference and orchestration-heavy workloads. While the broader market only fully woke up to this in 2026 when she doubled AMD’s long-term server CPU TAM forecast to >$120B by 2030 (with >35% CAGR), Dr. Su and her team have consistently positioned the company at the center of the CPU renaissance. The explosive demand we are seeing today, sold-out lines, rising ASPs, and hyperscalers forward-buying entire gigawatts of Helios-class systems is exactly the outcome she forecasted years ago. Not Financial Advice! DYOR!

Mike

301,322 次观看 • 2 个月前

$AMD $AMZN partnership will 🚀 in 2026 🔥 Amazon/AMD partnership is hidden among hot headlines from OpenAI $NVDA $ORCL... TLDR: Amazon refused to bid up the overpriced $NVDA chips among other hyperscalers, and decided to work closely with $AMD. Amazon is expected to spend up to $10-$20B a year on 2026 EPYC breakthrough Gen and Future Gen. Dr. Su confirmed "we have plenty for other large customers". For its 2026 EPYC "Venice" processors, AMD is using a multi-node manufacturing strategy: the CPU core complex dies (CCDs) are built on TSMC's 2 nm-class node (N2), while the I/O die (IOD) uses the N3P (3 nm) process. Context: Andy Jassy Amazon Web Services has been working with AMD on EPYC processors since November 2018. With this "secret weapon" breakthrough(patented), this long time partnership has expanded to New breakthrough 2026 EPYC Gen. AMD's 6th Gen EPYC "Venice" processors, slated for 2026, introduce New Chiplet design breakthrough. a revolutionary chiplet interconnect fabric that redefines server scalability for AI. This isn't just faster silicon; it's a paradigm shift for AWS, enabling hyper-efficient, rack-scale AI inference that slashes costs and latency while boosting throughput. AMD to benefit AWS's $100B+ AI opportunity along with $ORCL $MSFT $GOOGL $META Saudi, UAE ,38+ countries and startups. In early October, Amazon/AWS announced the new EC2 M8a instances as their latest-generation, general-purpose compute instances now powered by AMD EPYC 9005 "Turin" processors. Amazon announced the M8a as having up to 30% higher performance and up to 19% better price performance over M7a. With my testing of both at 32 vCPUs, the new AMD EPYC Turin instance provided 1.59x the performance over the prior-generation EPYC Genoa instance! How will this impact AWS AI Inference? ~Cost Efficiency: Inference is 80%+ of AI workloads and latency-sensitive (e.g., chatbots need <1s responses). "Secret weapon" enables 35x better inference perf (per AMD's CDNA roadmap tie-in), cutting AWS's energy use by 50%+ in clusters. With $118B 2025 capex, this could save $20–$30B annually in OPEX, boosting margins to 35%-40%. ~Scalability for Agentic AI: Supports "Helios" rack-scale platforms (up to 128 GPUs + EPYC hosts), delivering 3.58x FP6 perf for distributed inference. AWS can run 700K+ more tokens/sec in 1,000-node clusters (via EPYC 9575F boosts), enabling real-time apps like personalized search or fraud detection at enterprise scale. ~Adoption Catalysts: Early partners like Oracle signal broad uptake; AWS's existing AMD instances G4ad with Radeon GPUs) pave the way. By 2026, EPYC could power 40%+ of AWS AI infra, outpacing Nvidia's GPU lock-in via open standards (ROCm 8 software). Lastly, Amazon’s trajectory toward a $320 stock price is not a speculative leap but a grounded projection rooted in its unmatched fundamentals and strategic AI leadership. With Amazon Web Services poised to surpass $100 billion in annual revenue by 2026, driven by explosive AI inference demand, Amazon is redefining cloud computing’s future. The adoption of AMD’s 2026 EPYC processors with "Secret" architecture is a game-changer, slashing costs by up to 50% and boosting inference throughput 3x, enabling AWS to dominate enterprise AI workloads with unmatched efficiency. This technological edge, combined with Amazon’s e-commerce dominance and high-margin advertising growth, supports a valuation rerating to 22x EV/EBITDA, and it is still a discount to historical highs. Trading at $222, $AMZN is undervalued for its 15–20% revenue CAGR and 25%+ EPS growth through 2030.
0:49

$AMD $AMZN partnership will 🚀 in 2026 🔥 Amazon/AMD partnership is hidden among hot headlines from OpenAI $NVDA $ORCL... TLDR: Amazon refused to bid up the overpriced $NVDA chips among other hyperscalers, and decided to work closely with $AMD. Amazon is expected to spend up to $10-$20B a year on 2026 EPYC breakthrough Gen and Future Gen. Dr. Su confirmed "we have plenty for other large customers". For its 2026 EPYC "Venice" processors, AMD is using a multi-node manufacturing strategy: the CPU core complex dies (CCDs) are built on TSMC's 2 nm-class node (N2), while the I/O die (IOD) uses the N3P (3 nm) process. Context: Andy Jassy Amazon Web Services has been working with AMD on EPYC processors since November 2018. With this "secret weapon" breakthrough(patented), this long time partnership has expanded to New breakthrough 2026 EPYC Gen. AMD's 6th Gen EPYC "Venice" processors, slated for 2026, introduce New Chiplet design breakthrough. a revolutionary chiplet interconnect fabric that redefines server scalability for AI. This isn't just faster silicon; it's a paradigm shift for AWS, enabling hyper-efficient, rack-scale AI inference that slashes costs and latency while boosting throughput. AMD to benefit AWS's $100B+ AI opportunity along with $ORCL $MSFT $GOOGL $META Saudi, UAE ,38+ countries and startups. In early October, Amazon/AWS announced the new EC2 M8a instances as their latest-generation, general-purpose compute instances now powered by AMD EPYC 9005 "Turin" processors. Amazon announced the M8a as having up to 30% higher performance and up to 19% better price performance over M7a. With my testing of both at 32 vCPUs, the new AMD EPYC Turin instance provided 1.59x the performance over the prior-generation EPYC Genoa instance! How will this impact AWS AI Inference? ~Cost Efficiency: Inference is 80%+ of AI workloads and latency-sensitive (e.g., chatbots need <1s responses). "Secret weapon" enables 35x better inference perf (per AMD's CDNA roadmap tie-in), cutting AWS's energy use by 50%+ in clusters. With $118B 2025 capex, this could save $20–$30B annually in OPEX, boosting margins to 35%-40%. ~Scalability for Agentic AI: Supports "Helios" rack-scale platforms (up to 128 GPUs + EPYC hosts), delivering 3.58x FP6 perf for distributed inference. AWS can run 700K+ more tokens/sec in 1,000-node clusters (via EPYC 9575F boosts), enabling real-time apps like personalized search or fraud detection at enterprise scale. ~Adoption Catalysts: Early partners like Oracle signal broad uptake; AWS's existing AMD instances G4ad with Radeon GPUs) pave the way. By 2026, EPYC could power 40%+ of AWS AI infra, outpacing Nvidia's GPU lock-in via open standards (ROCm 8 software). Lastly, Amazon’s trajectory toward a $320 stock price is not a speculative leap but a grounded projection rooted in its unmatched fundamentals and strategic AI leadership. With Amazon Web Services poised to surpass $100 billion in annual revenue by 2026, driven by explosive AI inference demand, Amazon is redefining cloud computing’s future. The adoption of AMD’s 2026 EPYC processors with "Secret" architecture is a game-changer, slashing costs by up to 50% and boosting inference throughput 3x, enabling AWS to dominate enterprise AI workloads with unmatched efficiency. This technological edge, combined with Amazon’s e-commerce dominance and high-margin advertising growth, supports a valuation rerating to 22x EV/EBITDA, and it is still a discount to historical highs. Trading at $222, $AMZN is undervalued for its 15–20% revenue CAGR and 25%+ EPS growth through 2030.

Mike

511,082 次观看 • 9 个月前