Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

LFM2-2.6B-Transcript AMD🤝Liquid AI > Private, on-device meeting summarization > Cloud-level quality > Faster processing, lower memory footprint > Runs across CPU, GPU, and NPU on AMD Ryzen AI PC Great showcase of what tiny models can achieve when fine-tuned

Maxime Labonne

26,004 subscribers

27,703 просмотров • 6 месяцев назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Bringing AI agents to the PC requires efficiency, speed, and smart memory management. AMD and Liquid AI are partnering to enable foundational models on PCs and end-point devices to address the real-world challenges of running AI agents locally. Liquid AI Co-founder and CEO Ramin explains how we are helping make on-device AI practical and accessible.

Bringing AI agents to the PC requires efficiency, speed, and smart memory management. AMD and Liquid AI are partnering to enable foundational models on PCs and end-point devices to address the real-world challenges of running AI agents locally. Liquid AI Co-founder and CEO Ramin explains how we are helping make on-device AI practical and accessible.

AMD

20,936 просмотров • 5 месяцев назад

Introducing NexaSDK for Android (Beta) — run the latest AI models locally, 9× more energy-efficient and 2× faster, on Android devices, powered by the Qualcomm Hexagon NPU. This is the first SDK to support NPU, GPU and CPU, unlocking the full power of every Android device — for example, LFM2-1.2B achieves 85 t/s on NPU vs 37 t/s on CPU. With just 3 lines of code, you can run the latest state-of-the-art models across every task, for example: Multimodal (Vision, Audio, Text): OmniNeural-4B Embedding: EmbeddingGemma from Google ASR: Parakeet-v3 from NVIDIA OCR: PaddleOCR from Baidu Inc. Rerank: Jina-reranker from Jina AI LLM: LFM2-1.2B from And we continue to deliver Day-0 model support across our framework. Build on-device AI into your Android app today and enjoy no cloud API cost, full privacy, and offline availability. Check out our Quickstart guide and example app below to get started in minutes.

Introducing NexaSDK for Android (Beta) — run the latest AI models locally, 9× more energy-efficient and 2× faster, on Android devices, powered by the Qualcomm Hexagon NPU. This is the first SDK to support NPU, GPU and CPU, unlocking the full power of every Android device — for example, LFM2-1.2B achieves 85 t/s on NPU vs 37 t/s on CPU. With just 3 lines of code, you can run the latest state-of-the-art models across every task, for example: Multimodal (Vision, Audio, Text): OmniNeural-4B Embedding: EmbeddingGemma from Google ASR: Parakeet-v3 from NVIDIA OCR: PaddleOCR from Baidu Inc. Rerank: Jina-reranker from Jina AI LLM: LFM2-1.2B from And we continue to deliver Day-0 model support across our framework. Build on-device AI into your Android app today and enjoy no cloud API cost, full privacy, and offline availability. Check out our Quickstart guide and example app below to get started in minutes.

NEXA AI

26,091 просмотров • 8 месяцев назад

AMD on local AI: "this is the smallest AI development system in the world, capable of running models up to 200 billion parameters locally, not connected to anything" 128GB of unified memory shared by CPU, GPU, and NPU fits in your hand. Runs a model larger than GPT-3 with no internet $3,999 once. roughly $16 a month in power Cloud equivalent: up to $750 a month The question was never whether local AI could compete on power It was whether it could compete on price. Now it does

AMD on local AI: "this is the smallest AI development system in the world, capable of running models up to 200 billion parameters locally, not connected to anything" 128GB of unified memory shared by CPU, GPU, and NPU fits in your hand. Runs a model larger than GPT-3 with no internet $3,999 once. roughly $16 a month in power Cloud equivalent: up to $750 a month The question was never whether local AI could compete on power It was whether it could compete on price. Now it does

plutos

151,966 просмотров • 1 месяц назад

Great spending time with Rick Mershad, CEO of Micro Center, to talk about an exciting milestone for builders. Ryzen AI Halo systems are now shipping from Micro Center. 💪 AMD is the #1 desktop CPU at Micro Center with Ryzen X3D. 🎮 Radeon powers approximately 1 in every 3 desktop gaming GPUs sold at Micro Center. Now we are extending that same momentum into a new era. Ryzen AI Halo gives AI developers and builders the ability to run larger models locally, iterate faster, and unlock powerful new AI experiences while keeping more of their work on device for greater privacy and security, without cloud limits. The Agentic PC is here 🚀

Great spending time with Rick Mershad, CEO of Micro Center, to talk about an exciting milestone for builders. Ryzen AI Halo systems are now shipping from Micro Center. 💪 AMD is the #1 desktop CPU at Micro Center with Ryzen X3D. 🎮 Radeon powers approximately 1 in every 3 desktop gaming GPUs sold at Micro Center. Now we are extending that same momentum into a new era. Ryzen AI Halo gives AI developers and builders the ability to run larger models locally, iterate faster, and unlock powerful new AI experiences while keeping more of their work on device for greater privacy and security, without cloud limits. The Agentic PC is here 🚀

Jack Huynh

19,080 просмотров • 21 дней назад

AI agents are evolving and becoming more proactive. From cloud-based to on-device, these systems can run continuously, handle tasks, and keep sensitive data local. New Liquid Foundation Models bring agents directly to your laptop and phone, even while you are offline. Watch the latest episode of Advanced Insights to learn more about our partnership with Liquid AI:

AI agents are evolving and becoming more proactive. From cloud-based to on-device, these systems can run continuously, handle tasks, and keep sensitive data local. New Liquid Foundation Models bring agents directly to your laptop and phone, even while you are offline. Watch the latest episode of Advanced Insights to learn more about our partnership with Liquid AI:

AMD

29,729 просмотров • 3 месяцев назад

💥AI is no longer confined to the cloud. At Computex, Jack Huynh showcased the AMD Ryzen AI Halo and Ryzen AI Max PRO 400 Series processors. The next wave of innovation will be built, trained, and run locally. As AI evolves from chatbots to intelligent agents, developers and enterprises need the performance, memory capacity, and flexibility to bring these experiences to life directly on the device. ⚡ Ryzen AI Halo enables local AI development with support for models up to 200B parameters, up to 128GB of unified memory, ROCm optimization, and Windows + Linux support. ⚡ Ryzen AI Max PRO 400 Series combines AI, graphics, and compute in a single architecture, delivering workstation-class performance for next-gen Agent Computers and AI PCs. Some of the most impactful AI experiences will happen right at your desk, where latency, privacy, and control matter most. We're just getting started.

💥AI is no longer confined to the cloud. At Computex, Jack Huynh showcased the AMD Ryzen AI Halo and Ryzen AI Max PRO 400 Series processors. The next wave of innovation will be built, trained, and run locally. As AI evolves from chatbots to intelligent agents, developers and enterprises need the performance, memory capacity, and flexibility to bring these experiences to life directly on the device. ⚡ Ryzen AI Halo enables local AI development with support for models up to 200B parameters, up to 128GB of unified memory, ROCm optimization, and Windows + Linux support. ⚡ Ryzen AI Max PRO 400 Series combines AI, graphics, and compute in a single architecture, delivering workstation-class performance for next-gen Agent Computers and AI PCs. Some of the most impactful AI experiences will happen right at your desk, where latency, privacy, and control matter most. We're just getting started.

AMD

24,315 просмотров • 1 месяц назад

What Lisa Su actually held on stage: A mini PC the size of a lunchbox running Qwen3-235B locally, with no cloud and no discrete GPU Inside: the Ryzen AI Max+ 395, 128GB unified memory, 110GB usable as VRAM on Linux The first x86 chip that handles 200+ billion parameters on a single die AMD claims it beats the RTX 5080 by several times on memory-bound models — because the 5080 simply cannot fit them $1,400 to $2,500 once. cloud bills run $200 to $400 a month It pays for itself in a few months, then costs nothing per request This is not a faster GPU. it is the first real argument that your AI does not belong in someone else's data center

What Lisa Su actually held on stage: A mini PC the size of a lunchbox running Qwen3-235B locally, with no cloud and no discrete GPU Inside: the Ryzen AI Max+ 395, 128GB unified memory, 110GB usable as VRAM on Linux The first x86 chip that handles 200+ billion parameters on a single die AMD claims it beats the RTX 5080 by several times on memory-bound models — because the 5080 simply cannot fit them $1,400 to $2,500 once. cloud bills run $200 to $400 a month It pays for itself in a few months, then costs nothing per request This is not a faster GPU. it is the first real argument that your AI does not belong in someone else's data center

plutos

1,235,717 просмотров • 1 месяц назад

Still buzzing from our live #COMPUTEX demos! Watching Chang Liu speak Mandarin while an AMD AI PC translated her words into English — live, on-device, with zero cloud assistance — was a true milestone. This demo proved real-time, voice-to-voice translation can run privately and with low latency on a single Ryzen AI system. Huge thanks to Nick Ni for orchestrating this seamless demo, Chang for her perfect delivery, and the entire AMD AI Group for turning months of hard work into that unforgettable moment. Why it matters: • Latency so low it feels natural — conversations flow smoothly • Data stays on-device — privacy uncompromised This is what next-gen Ryzen AI processors unlock — and it’s only the beginning. We’re committed to improving accuracy and reducing latency as we push the boundaries of what’s possible.

Still buzzing from our live #COMPUTEX demos! Watching Chang Liu speak Mandarin while an AMD AI PC translated her words into English — live, on-device, with zero cloud assistance — was a true milestone. This demo proved real-time, voice-to-voice translation can run privately and with low latency on a single Ryzen AI system. Huge thanks to Nick Ni for orchestrating this seamless demo, Chang for her perfect delivery, and the entire AMD AI Group for turning months of hard work into that unforgettable moment. Why it matters: • Latency so low it feels natural — conversations flow smoothly • Data stays on-device — privacy uncompromised This is what next-gen Ryzen AI processors unlock — and it’s only the beginning. We’re committed to improving accuracy and reducing latency as we push the boundaries of what’s possible.

Jack Huynh

14,634 просмотров • 1 год назад

AMD CEO LISA SU HELD A MINI PC ON STAGE THAT RUNS A 235B MODEL AND REPLACES YOUR $440/MONTH AI STACK amd's ryzen ai max+ 395 is the first x86 chip that runs a 200 billion parameter model on one piece of silicon. cpu and gpu share 128gb of unified memory, no separate graphics card needed the gmktec evo-x2 runs qwen3 235b fully, deepseek v3 comfortably and llama 3.3 70b with headroom. on linux you get 110gb of usable vram out of 128gb amd claimed the chip beat an nvidia rtx 5080 by more than 3x on deepseek r1 inference. a lunchbox sized pc outrunning a $1,000 discrete gpu on a real ai workload a heavy ai user pays $200 for claude code max, $200 for chatgpt pro, $20 for cursor and $20 for gemini. that's $5,280 a year and the box pays itself off in 9 to 10 months install ollama, pull the model, point claude code at localhost. same interface, nothing leaves the machine, nothing costs per request bookmark this and read the article below

AMD CEO LISA SU HELD A MINI PC ON STAGE THAT RUNS A 235B MODEL AND REPLACES YOUR $440/MONTH AI STACK amd's ryzen ai max+ 395 is the first x86 chip that runs a 200 billion parameter model on one piece of silicon. cpu and gpu share 128gb of unified memory, no separate graphics card needed the gmktec evo-x2 runs qwen3 235b fully, deepseek v3 comfortably and llama 3.3 70b with headroom. on linux you get 110gb of usable vram out of 128gb amd claimed the chip beat an nvidia rtx 5080 by more than 3x on deepseek r1 inference. a lunchbox sized pc outrunning a $1,000 discrete gpu on a real ai workload a heavy ai user pays $200 for claude code max, $200 for chatgpt pro, $20 for cursor and $20 for gemini. that's $5,280 a year and the box pays itself off in 9 to 10 months install ollama, pull the model, point claude code at localhost. same interface, nothing leaves the machine, nothing costs per request bookmark this and read the article below

starmex

978,917 просмотров • 1 месяц назад

✨ Personal AI is the next computing platform. AI is shifting from something you access to something you build with, locally, at the edge, and across systems. We’re unlocking new possibilities for developers: • AMD Ryzen AI Halo, a local-first developer system, preorder starting in June, develop AI without limits on your desk • Gorgon Halo with up to 192GB unified memory, supporting 300B+ parameter models locally, run massive models locally We’re excited to partner with clem 🤗🤗, Co-founder and CEO of Hugging Face, to advance open-source AI for Ryzen AI. Our focus is seamless AI, from model to deployment. Cloud, edge, device. One continuum. Multi-agent systems, local inference at scale, open models as infrastructure. This is the next computing era 🚀

✨ Personal AI is the next computing platform. AI is shifting from something you access to something you build with, locally, at the edge, and across systems. We’re unlocking new possibilities for developers: • AMD Ryzen AI Halo, a local-first developer system, preorder starting in June, develop AI without limits on your desk • Gorgon Halo with up to 192GB unified memory, supporting 300B+ parameter models locally, run massive models locally We’re excited to partner with clem 🤗🤗, Co-founder and CEO of Hugging Face, to advance open-source AI for Ryzen AI. Our focus is seamless AI, from model to deployment. Cloud, edge, device. One continuum. Multi-agent systems, local inference at scale, open models as infrastructure. This is the next computing era 🚀

Jack Huynh

47,822 просмотров • 2 месяцев назад

A guy who tested every AI Halo mini PC on the market picked a winner: The Minisforum MS-S1 Max They all run the same AMD Ryzen AI Max+ 395 with 128GB unified memory, so the decision came down to everything around the chip What won it: dual 10-gig Ethernet, 80 Gbit/s USB4 V2, internal power supply, a low-profile PCIe slot the competition skipped The overlooked detail: the CPU/GPU memory split quietly changes CPU performance by 15% on identical silicon The honest comparison: NVIDIA's GB10 boxes have nearly closed the price gap and still lead on software maturity. AMD's edge is Windows, gaming, and expansion in one machine The takeaway is not which box is fastest It is that local AI hardware is now a commodity, and the real differences are the boring ones: ports, service, and software

plutos

97,428 просмотров • 1 месяц назад

NVIDIA'S BIGGEST COMPETITOR AMD BUILT A BOX THAT RUNS A 235B MODEL AND KILLS $200 OPENAI AND $200 CLAUDE CODE FOR $9 A MONTH amd's ryzen ai max+ 395 is the first x86 chip that runs a 200 billion parameter model on a single piece of silicon with 128gb of unified memory the gmktec evo-x2 runs qwen3 235b fully, deepseek v3 comfortably and llama 3.3 70b with headroom. on linux you get 110gb of usable vram out of 128gb amd claimed the chip beat an nvidia rtx 5080 by more than 3x on deepseek r1 inference. a lunchbox sized pc outrunning a $1,000 discrete gpu on a real ai workload a heavy ai user pays $200 for claude code max, $200 for chatgpt pro, $20 for cursor and $20 for gemini. that's $5,280 a year and the box pays itself off in 9 to 10 months ollama installs in one command and claude code points at localhost so nothing leaves the machine and nothing costs per request bookmark this and read the article below

NVIDIA'S BIGGEST COMPETITOR AMD BUILT A BOX THAT RUNS A 235B MODEL AND KILLS $200 OPENAI AND $200 CLAUDE CODE FOR $9 A MONTH amd's ryzen ai max+ 395 is the first x86 chip that runs a 200 billion parameter model on a single piece of silicon with 128gb of unified memory the gmktec evo-x2 runs qwen3 235b fully, deepseek v3 comfortably and llama 3.3 70b with headroom. on linux you get 110gb of usable vram out of 128gb amd claimed the chip beat an nvidia rtx 5080 by more than 3x on deepseek r1 inference. a lunchbox sized pc outrunning a $1,000 discrete gpu on a real ai workload a heavy ai user pays $200 for claude code max, $200 for chatgpt pro, $20 for cursor and $20 for gemini. that's $5,280 a year and the box pays itself off in 9 to 10 months ollama installs in one command and claude code points at localhost so nothing leaves the machine and nothing costs per request bookmark this and read the article below

starmex

93,357 просмотров • 1 месяц назад

Hugging Face Gemma Challenge results are in! 📈 Over 6 days, more than 100 AI agents and humans collaborated to make Gemma 4 inference 5x faster on a single NVIDIA A10G GPU. - Fastest result: 491.8 TPS (fastest overall, but resulted in a drop in model quality in other areas) - Fastest lossless: 315 TPS A great example of what humans and agents can achieve when they work together.

Hugging Face Gemma Challenge results are in! 📈 Over 6 days, more than 100 AI agents and humans collaborated to make Gemma 4 inference 5x faster on a single NVIDIA A10G GPU. - Fastest result: 491.8 TPS (fastest overall, but resulted in a drop in model quality in other areas) - Fastest lossless: 315 TPS A great example of what humans and agents can achieve when they work together.

Google Gemma

377,100 просмотров • 17 дней назад

$AMD Strategic Price Positioning Long🧵 AMD is increasingly the most hated semi stock that can rival $NVDA dominance in GPUs and software(Cuda v. ROCm). $AMD is also the most under-owned among all Funds in 2025 according to Bank of America! For what I learnt for years as an investor with Dr. Lisa Su, all analysts and market are underestimate Dr. Su leadership. $AMD is capable of raising price, making high quality hardware with software. Dr. Su or AMD choice to adopt a lower price strategy to gain market share is a deliberate and multifacets approach rooted in competitive positioning, market dynamics, and long-term growth objectives. As an investor, it may take time like CPUs and embedded to see margin improving. 1. . Penetration Pricing to Challenge Dominant Competitors AMD has historically positioned itself as a cost-effective alternative to dominant players like Intel in CPUs and Nvidia in GPUs. By setting prices lower than competitors, AMD aims to attract customers and quickly gain market share. This is a classic penetration pricing strategy, where the goal is to capture a significant portion of the market by offering high-performance products at a lower price point. ~CPU Market Example: When AMD launched its Ryzen processors in 2017, it priced them competitively compared to Intel's Core processors, emphasizing a better price-to-performance ratio. Ryzen CPUs offered higher core counts and multi-core performance at lower prices, appealing to cost-conscious consumers, gamers, and professionals. This strategy helped AMD increase its CPU market share to 16.6% by early 2025, narrowing the gap with Intel. ~GPU Market Context: In the GPU market, where Nvidia holds an 88% share compared to AMD's 12%, AMD has been criticized for not launching GPUs at low enough prices to compete effectively. However, posts on X and articles suggest AMD is shifting its GPU strategy to focus on mainstream, cost-effective products rather than high-end enthusiast segments, aiming to regain market share through competitive pricing. 2. Appealing to Cost-Conscious Market Segments AMD targets price-sensitive customers, including gamers, small businesses, and enterprises looking for high-performance computing at a lower cost. This is particularly effective in segments where performance is critical, but budgets are constrained. ~Value Proposition: AMD’s Ryzen and EPYC processors, as well as Radeon GPUs, are designed to deliver performance comparable to or better than competitors in specific workloads (e.g., multi-core processing or AI compute) at a lower price. For example, Ryzen processors have been noted for their superior multi-core performance compared to Intel CPUs at similar or lower price points, making them attractive for tasks like video editing or gaming. ~AI and Data Center: In the AI and data center markets, AMD’s cost-effective Instinct MI300X GPUs and EPYC CPUs target enterprises seeking affordable alternatives to Nvidia’s expensive AI ecosystem. This strategy taps into an underleveraged market segment that Nvidia’s broad, premium-priced AI solutions may not fully address. 3. Building Scale and Developer Support AMD’s leadership, including Jack Huynh, has emphasized the importance of scale—gaining a larger market share to attract developer support and optimize software ecosystems. A lower price strategy helps AMD achieve this by increasing adoption among consumers and enterprises. ~Gaming GPUs: By focusing on mainstream GPUs with competitive pricing (e.g., targeting an 80% addressable market rather than the high-end 10%), AMD aims to build a larger user base. This scale encourages developers to optimize games for AMD’s technologies, such as FSR 3 (FidelityFX Super Resolution) and Anti-Lag 2, improving the ecosystem and competitiveness against Nvidia’s CUDA platform. ~Open Ecosystem in AI: AMD’s open-source ROCm platform contrasts with Nvidia’s proprietary CUDA, appealing to developers who prefer flexibility. Lower-priced hardware makes it easier for developers to adopt AMD’s solutions, fostering a broader AI software ecosystem. 4. Historical Context and Brand Positioning Since its founding in 1969, AMD has positioned itself as a challenger brand, often acting as a “second source” supplier to Intel. This role required competitive pricing to gain a foothold in markets dominated by established players. Over time, AMD has built a reputation for quality and affordability, reinforced by products like the Am9080 (a reverse-engineered Intel 8080) and modern Ryzen and EPYC lines. This historical strategy of undercutting competitors’ prices while delivering comparable performance continues to define AMD’s approach. 5. Countering Competitor Dominance AMD operates in highly competitive markets where Intel and Nvidia have significant advantages in brand recognition, market share, and ecosystems. A lower price strategy is a pragmatic way to disrupt this in CPUs: ~Intel’s historical dominance in the CPU market (servers, desktops, and laptops) has been challenged by AMD’s Ryzen and EPYC processors, which offer better value. For instance, AMD’s EPYC CPUs have driven a 122% year-over-year revenue increase in the data center segment, partly due to their cost-effectiveness, helping AMD capture 94% of CPU sales at some retailers. ~Nvidia in GPUs: Nvidia’s 88% GPU market share and premium pricing (e.g., high-end GPUs like the RTX 4090) leave room for AMD to compete in the mid-to-low range. However, AMD’s failure to launch GPUs at sufficiently low prices (e.g., the RX 7900 XT at $900 instead of its current $680) has limited its success, prompting a strategic shift toward more aggressive pricing in future RDNA 4 GPUs. 6. Market Share as a Long-Term Investment AMD’s lower price strategy is not just about immediate sales but also about long-term market positioning. By capturing market share, AMD can: ~Increase Brand Loyalty: Affordable, high-performance products build customer loyalty, especially among gamers and small businesses, creating a foundation for future sales. ~Drive Revenue Growth: Market share gains in CPUs (e.g., 16.6% in 2025) and data centers (e.g., $3.5 billion in Q3 revenue) translate into higher revenue, even if margins are initially lower. ~Influence Industry Standards: Greater market presence allows AMD to influence hardware and software standards, such as pushing for open-source AI frameworks or gaming optimizations, reducing reliance on competitors’ proprietary systems. 7. Challenges and Risks While effective, AMD’s lower price strategy carries risks: ~Profitability Concerns: Lower prices can compress profit margins, and some analysts note that AMD’s high stock valuation expects future profitability that may be delayed if pricing remains aggressive. ~Perception of Quality: Persistently low prices risk positioning AMD as a “budget” brand, potentially undermining its ability to compete in premium segments. ~Competitor Response: Intel and Nvidia can counter with price cuts or superior features, as seen with Nvidia’s feature-rich GPUs. AMD must balance price with innovation to avoid being outmaneuvered. 8. Strategic Shift in GPUs Recent reports indicate AMD is adjusting its GPU strategy to prioritize market share over competing in the high-end enthusiast segment. For the upcoming Radeon RX 8000 series (RDNA 4), AMD is focusing on mainstream GPUs priced competitively to appeal to a broader audience, rather than chasing Nvidia’s high-end dominance. This shift aligns with AMD’s broader goal of achieving 40–50% market share by targeting the “80%” of the market that prioritizes affordability over premium features. Lastly, AMD’s lower price strategy is a calculated move to disrupt Intel and Nvidia’s dominance, capture market share, and build scale for long-term growth. By offering high-performance CPUs and GPUs at competitive prices, AMD appeals to cost-conscious consumers and enterprises, particularly in the CPU and AI markets, where it has seen significant gains (e.g., 16.6% CPU market share and $3.5 billion in data center revenue). Recent price increase on MI350 and MI355 and more on MI400 signaled #AI chip leadership and pricing power, which will result in significant top and bottom line growth.

Mike

38,006 просмотров • 10 месяцев назад

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Daniel Romero

38,220 просмотров • 17 дней назад

Some time ago, I had the idea to port NVIDIA Physical AI stack to AMD. The motivation was to improve hardware diversity and enable world models and VLAs to run beyond a single ecosystem. We started with NVIDIA Cosmos Predict 2.5-2B. Porting wasn’t trivial: these models are deeply optimized for NVIDIA’s stack. We used this as an opportunity to apply our ROCm kernels. The results were surprising: Both encode and diffusion run faster on AMD Instinct MI300X vs. NVIDIA H200 (FA3) and we still saw significant headroom for further optimization. Quality is unchanged across modalities (validated with WorldJen) To be clear, this is no luck. We have deep experience with diffusion models and AMD GPUs. But this just gives us a good opportunity to get closer to a true hardware-to-hardware comparison, as we work with less software abstractions than usual. Just to give an example, on AMD, memory instructions are async with a hardware queue of ordered pending instructions, enabling concurrent load/store with compute without warp specialization. Bottom line: there are real architectural advantages on AMD, if you take the time to work with the hardware. Note, we did tradeoff ~20% higher memory usage, That being said, AMD has more to give to begin with :) in the coming weeks: AMD versions of Cosmos Transfer and GR00T, an even faster version of Cosmos Predict, and open-sourcing an attention kernel faster than AITER v3 (which is closed-source for some reason? cc: Anush Elangovan )

Some time ago, I had the idea to port NVIDIA Physical AI stack to AMD. The motivation was to improve hardware diversity and enable world models and VLAs to run beyond a single ecosystem. We started with NVIDIA Cosmos Predict 2.5-2B. Porting wasn’t trivial: these models are deeply optimized for NVIDIA’s stack. We used this as an opportunity to apply our ROCm kernels. The results were surprising: Both encode and diffusion run faster on AMD Instinct MI300X vs. NVIDIA H200 (FA3) and we still saw significant headroom for further optimization. Quality is unchanged across modalities (validated with WorldJen) To be clear, this is no luck. We have deep experience with diffusion models and AMD GPUs. But this just gives us a good opportunity to get closer to a true hardware-to-hardware comparison, as we work with less software abstractions than usual. Just to give an example, on AMD, memory instructions are async with a hardware queue of ordered pending instructions, enabling concurrent load/store with compute without warp specialization. Bottom line: there are real architectural advantages on AMD, if you take the time to work with the hardware. Note, we did tradeoff ~20% higher memory usage, That being said, AMD has more to give to begin with :) in the coming weeks: AMD versions of Cosmos Transfer and GR00T, an even faster version of Cosmos Predict, and open-sourcing an attention kernel faster than AITER v3 (which is closed-source for some reason? cc: Anush Elangovan )

Omer Shlomovits

36,620 просмотров • 3 месяцев назад

Everyone wrote Apple off as the AI loser, but one hardware spec might flip that story upside down (Save this). @jason called Apple a screaming buy on the back of a single chip detail. The rumored M7 Ultra, expected around 2028, is designed to support up to 1.5TB of unified memory, enough to run frontier class trillion parameter AI models locally, with no cloud required. The Street's bear case on Apple is straightforward. Apple has no frontier model of its own, Siri has stumbled for years and the company effectively rents OpenAI's models for its hardest queries. That narrative treats Apple as the one Magnificent Seven name that missed the AI wave entirely but the bull case flips that framing on its head. If frontier AI models keep shrinking and getting cheaper to run, Apple doesn't need the smartest model in the world, it just needs to own the device that model runs on. And unified memory is the mechanism that makes this possible. Unlike traditional systems where the CPU and GPU each need separate memory, Apple's architecture lets the CPU, GPU and Neural Engine draw from one shared pool. A fully specced M7 Ultra could theoretically run something on the scale of a 1.2 trillion parameter model locally and that capability plugs directly into the one advantage Apple has spent over a decade building: privacy. Apple has already shipped Private Cloud Compute, a system designed so even Apple can't access user data processed off device. Apple doubled down on this at WWDC 2026, framing on device privacy as non-negotiable while rivals default to the cloud. If the best AI models get small enough to run on Apple silicon, the moat stops being the model and becomes the hardware it has to sit on. Milk Road Pro remains bullish on Apple and it remains as one of our core positions, if you want the full thesis + our full AI trades, come join us using the link below for just a $1.

Everyone wrote Apple off as the AI loser, but one hardware spec might flip that story upside down (Save this). @jason called Apple a screaming buy on the back of a single chip detail. The rumored M7 Ultra, expected around 2028, is designed to support up to 1.5TB of unified memory, enough to run frontier class trillion parameter AI models locally, with no cloud required. The Street's bear case on Apple is straightforward. Apple has no frontier model of its own, Siri has stumbled for years and the company effectively rents OpenAI's models for its hardest queries. That narrative treats Apple as the one Magnificent Seven name that missed the AI wave entirely but the bull case flips that framing on its head. If frontier AI models keep shrinking and getting cheaper to run, Apple doesn't need the smartest model in the world, it just needs to own the device that model runs on. And unified memory is the mechanism that makes this possible. Unlike traditional systems where the CPU and GPU each need separate memory, Apple's architecture lets the CPU, GPU and Neural Engine draw from one shared pool. A fully specced M7 Ultra could theoretically run something on the scale of a 1.2 trillion parameter model locally and that capability plugs directly into the one advantage Apple has spent over a decade building: privacy. Apple has already shipped Private Cloud Compute, a system designed so even Apple can't access user data processed off device. Apple doubled down on this at WWDC 2026, framing on device privacy as non-negotiable while rivals default to the cloud. If the best AI models get small enough to run on Apple silicon, the moat stops being the model and becomes the hardware it has to sit on. Milk Road Pro remains bullish on Apple and it remains as one of our core positions, if you want the full thesis + our full AI trades, come join us using the link below for just a $1.

Milk Road AI

36,711 просмотров • 8 дней назад

AMD might have disrupted Nvidia's entire cloud GPU rental business. In January at CES, AMD CEO Lisa Su demonstrated a $1,499 mini PC running the same class of AI model that currently costs companies $2,500 to $3,000 every month to rent from Nvidia-powered cloud servers. AMD's own branded version opened pre-orders this month at $3,999. Third party manufacturers have been selling the same chip since 2025 starting at $1,499. Here is exactly why this is dangerous for Nvidia. Nvidia's $75 billion quarterly revenue is built almost entirely on one business model, companies rent access to Nvidia GPUs through cloud providers like AWS and Lambda Labs to run AI. They pay monthly. Nvidia gets paid every time someone runs an AI model in the cloud. That recurring rental income is what turned Nvidia into a $5 trillion company. The AMD box eliminates that monthly fee permanently. One AI consultant switched from $2,800 per month in Nvidia cloud rental costs to $8 per month in electricity. The hardware paid for itself in 11 days. Over 8 months he generated $47,000 running the same AI workloads that previously left him paying Nvidia's ecosystem $2,800 every single month. Multiply that across thousands of enterprise customers and the revenue erosion becomes structural. Every business that buys this box stops paying cloud rental fees forever. Lawyers, doctors, banks, accountants, and financial advisors, businesses with sensitive data that cannot legally go to a cloud server represent billions in annual cloud GPU fees that Nvidia is now at risk of losing permanently. The threat is also closing in from the top. Google signed deals worth tens of billions with Anthropic and Meta to replace Nvidia with its own chips. Amazon built its own AI chips across AWS. Apple trained its AI on Google's chips, not Nvidia's. Custom silicon has grown from 21% of the AI chip market in 2025 to 28% in 2026. Nvidia's rental model only worked because serious AI compute had no alternative.

AMD might have disrupted Nvidia's entire cloud GPU rental business. In January at CES, AMD CEO Lisa Su demonstrated a $1,499 mini PC running the same class of AI model that currently costs companies $2,500 to $3,000 every month to rent from Nvidia-powered cloud servers. AMD's own branded version opened pre-orders this month at $3,999. Third party manufacturers have been selling the same chip since 2025 starting at $1,499. Here is exactly why this is dangerous for Nvidia. Nvidia's $75 billion quarterly revenue is built almost entirely on one business model, companies rent access to Nvidia GPUs through cloud providers like AWS and Lambda Labs to run AI. They pay monthly. Nvidia gets paid every time someone runs an AI model in the cloud. That recurring rental income is what turned Nvidia into a $5 trillion company. The AMD box eliminates that monthly fee permanently. One AI consultant switched from $2,800 per month in Nvidia cloud rental costs to $8 per month in electricity. The hardware paid for itself in 11 days. Over 8 months he generated $47,000 running the same AI workloads that previously left him paying Nvidia's ecosystem $2,800 every single month. Multiply that across thousands of enterprise customers and the revenue erosion becomes structural. Every business that buys this box stops paying cloud rental fees forever. Lawyers, doctors, banks, accountants, and financial advisors, businesses with sensitive data that cannot legally go to a cloud server represent billions in annual cloud GPU fees that Nvidia is now at risk of losing permanently. The threat is also closing in from the top. Google signed deals worth tens of billions with Anthropic and Meta to replace Nvidia with its own chips. Amazon built its own AI chips across AWS. Apple trained its AI on Google's chips, not Nvidia's. Custom silicon has grown from 21% of the AI chip market in 2025 to 28% in 2026. Nvidia's rental model only worked because serious AI compute had no alternative.

Bull Theory

26,668 просмотров • 1 месяц назад

QVAC SDK 0.15.0 is live. This release adds multiple prompts batching, brings a native AMD GPU backend to the stack, moves more vision encoders onto mobile GPUs, and adds a second local coding-agent integration. Main highlights: - Prompt batching for the LLM addon. Batch multiple prompts into one job and process them concurrently, with each answer returned the moment its generation finishes. - Native AMD GPU backend. A first-class HIP/ROCm backend in @qvac/vla-ggml, auto-selected over Vulkan with clean fallback when ROCm is absent. - A second local coding agent. OpenClaw joins OpenCode for local, cloud-free agent workflows. AGENTS - OpenCode plugin update (@qvac/opencode-plugin). Aligned with the current SDK, CLI, and AI SDK provider packages. A fresh install runs OpenCode against managed local QVAC models out of the box, from the default qvac/qwen3.5-9b, with no manual qvac serve setup. - OpenClaw plugin (@qvac/openclaw-plugin). A second coding-agent integration alongside OpenCode. A fresh setup installs the plugin, creates a local qvac provider through onboarding, and runs a QVAC model through OpenClaw🦞's local service path. LANGUAGE MODELS - Prompt batching (LLM addon). Batch multiple prompts in one job and run them concurrently, each answer returns the moment its generation finishes, no waiting on the others. - Reasoning-context trimming on hybrid + recurrent models (@qvac/llm-llamacpp). remove_thinking_from_context now works beyond pure-attention models. Same JS API, no throw. VOICE AND SPEECH - Transcription (transcription-parakeet 0.9.0). More robust CPU fallback on GPU failure and a faster Vulkan backend on Pixel 9. - Text-to-speech features (tts-ggml 0.4.0). Adds LavaSR for noise removal and adjustable output frequency up to 48 kHz, plus Japanese via Chatterbox. - Text-to-speech fixes (tts-ggml 0.4.1). CPU fallback on GPU failure, a q8_0 KV crash fix on Metal with Chatterbox. VISION - Qwen3.5 vision encoder on GPU (Android). Image encoder moves onto the phone GPU, with a smarter tile-grid preprocessor and default image-token caps, for flagship Android: Vulkan on Mali (Pixel 9 Pro) and OpenCL on Adreno 830 (Galaxy S25). - Gemma-4 vision encoder on GPU (Android). Vision encoder runs on the phone GPU instead of CPU, same flagship Android targets. PLATFORM AND PERFORMANCE - AMD GPU backend (@qvac/vla-ggml). Native HIP/ROCm backend, auto-selected over Vulkan with clean fallback when ROCm is absent (Linux x64 only). Comes with ~23% faster than Vulkan, ~14% faster than PyTorch-ROCm, parity preserved. Unified code style. A cleaner, more consistent, easier-to-contribute codebase. Let's build. npm install @qvac/sdk

QVAC SDK 0.15.0 is live. This release adds multiple prompts batching, brings a native AMD GPU backend to the stack, moves more vision encoders onto mobile GPUs, and adds a second local coding-agent integration. Main highlights: - Prompt batching for the LLM addon. Batch multiple prompts into one job and process them concurrently, with each answer returned the moment its generation finishes. - Native AMD GPU backend. A first-class HIP/ROCm backend in @qvac/vla-ggml, auto-selected over Vulkan with clean fallback when ROCm is absent. - A second local coding agent. OpenClaw joins OpenCode for local, cloud-free agent workflows. AGENTS - OpenCode plugin update (@qvac/opencode-plugin). Aligned with the current SDK, CLI, and AI SDK provider packages. A fresh install runs OpenCode against managed local QVAC models out of the box, from the default qvac/qwen3.5-9b, with no manual qvac serve setup. - OpenClaw plugin (@qvac/openclaw-plugin). A second coding-agent integration alongside OpenCode. A fresh setup installs the plugin, creates a local qvac provider through onboarding, and runs a QVAC model through OpenClaw🦞's local service path. LANGUAGE MODELS - Prompt batching (LLM addon). Batch multiple prompts in one job and run them concurrently, each answer returns the moment its generation finishes, no waiting on the others. - Reasoning-context trimming on hybrid + recurrent models (@qvac/llm-llamacpp). remove_thinking_from_context now works beyond pure-attention models. Same JS API, no throw. VOICE AND SPEECH - Transcription (transcription-parakeet 0.9.0). More robust CPU fallback on GPU failure and a faster Vulkan backend on Pixel 9. - Text-to-speech features (tts-ggml 0.4.0). Adds LavaSR for noise removal and adjustable output frequency up to 48 kHz, plus Japanese via Chatterbox. - Text-to-speech fixes (tts-ggml 0.4.1). CPU fallback on GPU failure, a q8_0 KV crash fix on Metal with Chatterbox. VISION - Qwen3.5 vision encoder on GPU (Android). Image encoder moves onto the phone GPU, with a smarter tile-grid preprocessor and default image-token caps, for flagship Android: Vulkan on Mali (Pixel 9 Pro) and OpenCL on Adreno 830 (Galaxy S25). - Gemma-4 vision encoder on GPU (Android). Vision encoder runs on the phone GPU instead of CPU, same flagship Android targets. PLATFORM AND PERFORMANCE - AMD GPU backend (@qvac/vla-ggml). Native HIP/ROCm backend, auto-selected over Vulkan with clean fallback when ROCm is absent (Linux x64 only). Comes with ~23% faster than Vulkan, ~14% faster than PyTorch-ROCm, parity preserved. Unified code style. A cleaner, more consistent, easier-to-contribute codebase. Let's build. npm install @qvac/sdk

QVAC

29,251,774 просмотров • 14 дней назад