Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Why rent GPUs on Hyperbolic? Get started in under a minute. Check out the full step-by-step tutorial and this week's prices: > H200 - $2.25/gpu/hr > H100 SXM - $1.50/gpu/hr > RTX 4090 - $0.40/gpu/hr > RTX 3080 - $0.20/gpu/hr > RTX 3070 - $0.16/gpu/hr

Hyperbolic

50,324 subscribers

19,659 просмотров • 1 год назад •via X (Twitter)

Финансы Наука и технологии Образование

Anya Rossi• Live Now

Private livecam show

Комментарии: 11

Фото профиля Hyperbolic

Hyperbolic1 год назад

Become GPU rich by visiting Hyperbolic at

Фото профиля Quant Data

Quant Data1 год назад

🚨 $NVDA is down over 16% 🚨 Yesterday evening, we tweeted about the $25M in Puts being purchased on $NVDA. DeepSeek AI news came out and NVDA dropped $24 today. Want to see trades like this? Try our 7 day free trial at:

Фото профиля wincy.eth

wincy.eth1 год назад

this is cheaper than my morning coffee and does more for productivity! ppl need to think about it)

Фото профиля Hyperbolic

Hyperbolic1 год назад

💯💯

Фото профиля Hyperbolic

Hyperbolic1 год назад

@jesriley12 we appreciate you

Фото профиля Dip 𐤊∆₹m∆𐤊∆₹

Dip 𐤊∆₹m∆𐤊∆₹1 год назад

On a mission to make AI accessible for everyone. Lets make us GPU rich.

Фото профиля Hyperbolic

Hyperbolic1 год назад

GPU rich!

Фото профиля RONIN🀄️

RONIN🀄️1 год назад

This is real and important use case Nice work @hyperbolic_labs

Фото профиля Hyperbolic

Hyperbolic1 год назад

thank you

Фото профиля MD YOUSUF ALI 🌊

MD YOUSUF ALI 🌊1 год назад

Yeah why not ✅

Фото профиля Hyperbolic

Hyperbolic1 год назад

$1 free credit to use inference when signing up

Похожие видео

on-demand H100 for $0.99/hr, 4090 for $0.20/hr at Hyperbolic likely the cheapest GPUs around tell me what you're building, and I'll spot you free credits for an 8xH100 node for at least a few hours to start.

on-demand H100 for $0.99/hr, 4090 for $0.20/hr at Hyperbolic likely the cheapest GPUs around tell me what you're building, and I'll spot you free credits for an 8xH100 node for at least a few hours to start.

Yuchen Jin

190,663 просмотров • 1 год назад

We just launched On-Demand GPU Cloud at Hyperbolic! - H100s at $1.49/hr - reliable nodes with persistent storage - spin up a cluster of GPU nodes with Infiniband in just a few clicks Tell me what you're building. I'm giving out free credits to cool projects.

We just launched On-Demand GPU Cloud at Hyperbolic! - H100s at $1.49/hr - reliable nodes with persistent storage - spin up a cluster of GPU nodes with Infiniband in just a few clicks Tell me what you're building. I'm giving out free credits to cool projects.

Yuchen Jin

53,409 просмотров • 1 год назад

GPU GIVEAWAY Buy a GPU × GTC 2026 = Give a GPU > RTX PRO 6000 Blackwell > 96GB VRAM • NVFP4 > ~$15K value > Brand new If this does well I’ll ask NVIDIA for more GPUs next time… maybe even DGX Sparks How to enter? Short clip here Full clip in the replies GO GO GO

Ahmad

36,697 просмотров • 4 месяцев назад

STAGE 2.5 STARTED! I'm now officially a Prover My RTX 4090 is active and producing real-time proof in stage 2.5 of the Succinct testnet. Detailed information will be shared by Succinct soon. PLEASE do not rent a GPU before these announcements. You can check the document to understand the process and for more details: Thank you crashout

STAGE 2.5 STARTED! I'm now officially a Prover My RTX 4090 is active and producing real-time proof in stage 2.5 of the Succinct testnet. Detailed information will be shared by Succinct soon. PLEASE do not rent a GPU before these announcements. You can check the document to understand the process and for more details: Thank you crashout

CryptoMeii

14,887 просмотров • 1 год назад

GPU tradeoff series: A100 is not much more powerful than 4090 🫠 GPU Perf and Price: - 4090: 330 fp16 TFLOPs, $1,749 - A100 (80GB): 312 fp16 TFLOPs, $20,000 > A100 is 11.4X more pricy Training speed for GPT-2(124M) with llm.c: - 4090: 153K tokens/s - A100 (80GB): 195K tokens/s > A100 is only 1.3X faster (both trained using a single card, A100 llm.c training is shown in the video, 4090 video is in the quoted tweet) Conclusion: 4090 has a much better cost vs performance ratio Why: As in the H100 vs. 4090 comparison, the biggest difference between A100 and 4090 is their GPU memory size/bandwidth and cross-GPU communication bandwidth, which does not matter too much if your model can fit into a single 4090. Specs: 4090: - GPU memory size: 24GB - memory bandwidth: 1 TB/s - communication bandwidth: 64 GB/s A100: - GPU memory size: 80GB - memory bandwidth: 2 TB/s - communication bandwidth: 900 GB/s Nvidia killed off NVLink (a high-speed communication link that connects GPUs) on 4090. (Jensen Huang smiling face) If multiple 4090s could be interconnected via NVLink, their performance would be closer to datacenter-grade A100 GPUs, even for training larger models. Additionally, 4090 isn't allowed in datacenters, that's how Nvidia makes 💰💰💰

GPU tradeoff series: A100 is not much more powerful than 4090 🫠 GPU Perf and Price: - 4090: 330 fp16 TFLOPs, $1,749 - A100 (80GB): 312 fp16 TFLOPs, $20,000 > A100 is 11.4X more pricy Training speed for GPT-2(124M) with llm.c: - 4090: 153K tokens/s - A100 (80GB): 195K tokens/s > A100 is only 1.3X faster (both trained using a single card, A100 llm.c training is shown in the video, 4090 video is in the quoted tweet) Conclusion: 4090 has a much better cost vs performance ratio Why: As in the H100 vs. 4090 comparison, the biggest difference between A100 and 4090 is their GPU memory size/bandwidth and cross-GPU communication bandwidth, which does not matter too much if your model can fit into a single 4090. Specs: 4090: - GPU memory size: 24GB - memory bandwidth: 1 TB/s - communication bandwidth: 64 GB/s A100: - GPU memory size: 80GB - memory bandwidth: 2 TB/s - communication bandwidth: 900 GB/s Nvidia killed off NVLink (a high-speed communication link that connects GPUs) on 4090. (Jensen Huang smiling face) If multiple 4090s could be interconnected via NVLink, their performance would be closer to datacenter-grade A100 GPUs, even for training larger models. Additionally, 4090 isn't allowed in datacenters, that's how Nvidia makes 💰💰💰

Yuchen Jin

234,951 просмотров • 1 год назад

Whoa, this is Temporal Gaussian Hierarchy (TGH). This AI can real-time render long volumetric videos in 1080p at 450 fps on a single RTX 4090 GPU. And 18,000 frames (around 10 minutes) long 🤯 7 wild demos: 1. Real-time demos on MobileStage dataset

Whoa, this is Temporal Gaussian Hierarchy (TGH). This AI can real-time render long volumetric videos in 1080p at 450 fps on a single RTX 4090 GPU. And 18,000 frames (around 10 minutes) long 🤯 7 wild demos: 1. Real-time demos on MobileStage dataset

Min Choi

464,076 просмотров • 1 год назад

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,622 просмотров • 2 лет назад

#Battlefield6 NVIDIA DLSS 4 Reveal Trailer 📽️ The game includes support for DLSS 4 with Multi Frame Generation, DLSS Frame Generation, DLSS Super Resolution, DLAA, and NVIDIA Reflex. Desktop GPUs Performance At 4K, Ultra settings, DLSS 4 with Multi Frame Generation and DLSS Super Resolution multiply Battlefield 6's GeForce RTX 50 Series frame rates by an average of 3.8X. ▪️ GeForce RTX 5090 performance rockets to over 470 FPS. ▪️ GeForce RTX 5080 exceeds 330 FPS. ▪️ GeForce RTX 5070 Ti is in touching distance of 300 FPS. ▪️ GeForce RTX 5070 surpasses 230 FPS. At 2560x1440, Ultra settings, DLSS 4 with Multi Frame Generation and DLSS Super Resolution increase Battlefield 6 frame rates by an average of 3X. ▪️ GeForce RTX 5090 runs at almost 600 FPS. ▪️ GeForce RTX 5060 Ti over 240 FPS. At 1920x1080, Ultra settings, the combination of DLSS 4 with Multi Frame Generation and DLSS Super Resolution boost Battlefield 6’s GeForce RTX 50 Series frame rates by an average of 2.9X. ▪️ Allowing every GPU in NVIDIA's line-up to play at over 230 FPS, maxing out at over 740 FPS on the GeForce RTX 5090. Laptop GPUs Performance GeForce RTX 50 Series Laptop GPUs benefit similarly from DLSS 4 with Multi Frame Generation and DLSS Super Resolution, with Battlefield 6 frame rates multiplied by 3X on average at 2560x1600, with Ultra settings, enabling performance of up to 360 FPS. At 1920x1080, Ultra settings, GeForce RTX 50 Series Laptop GPU performance increases by 2.8X on average, enabling Battlefield 6 frame rates to surpass 480 FPS, and the range to run at over 200 frames per second.

#Battlefield6 NVIDIA DLSS 4 Reveal Trailer 📽️ The game includes support for DLSS 4 with Multi Frame Generation, DLSS Frame Generation, DLSS Super Resolution, DLAA, and NVIDIA Reflex. Desktop GPUs Performance At 4K, Ultra settings, DLSS 4 with Multi Frame Generation and DLSS Super Resolution multiply Battlefield 6's GeForce RTX 50 Series frame rates by an average of 3.8X. ▪️ GeForce RTX 5090 performance rockets to over 470 FPS. ▪️ GeForce RTX 5080 exceeds 330 FPS. ▪️ GeForce RTX 5070 Ti is in touching distance of 300 FPS. ▪️ GeForce RTX 5070 surpasses 230 FPS. At 2560x1440, Ultra settings, DLSS 4 with Multi Frame Generation and DLSS Super Resolution increase Battlefield 6 frame rates by an average of 3X. ▪️ GeForce RTX 5090 runs at almost 600 FPS. ▪️ GeForce RTX 5060 Ti over 240 FPS. At 1920x1080, Ultra settings, the combination of DLSS 4 with Multi Frame Generation and DLSS Super Resolution boost Battlefield 6’s GeForce RTX 50 Series frame rates by an average of 2.9X. ▪️ Allowing every GPU in NVIDIA's line-up to play at over 230 FPS, maxing out at over 740 FPS on the GeForce RTX 5090. Laptop GPUs Performance GeForce RTX 50 Series Laptop GPUs benefit similarly from DLSS 4 with Multi Frame Generation and DLSS Super Resolution, with Battlefield 6 frame rates multiplied by 3X on average at 2560x1600, with Ultra settings, enabling performance of up to 360 FPS. At 1920x1080, Ultra settings, GeForce RTX 50 Series Laptop GPU performance increases by 2.8X on average, enabling Battlefield 6 frame rates to surpass 480 FPS, and the range to run at over 200 frames per second.

Battlefield Bulletin

30,559 просмотров • 9 месяцев назад

Revolutionizing Access to #GPU Power - One Node at a Time.⚡️ Our 10th Utility is coming out on June 20.🚨 BIG GPU Nodes Platform isn’t just another name in the crowded GPU cloud services space, we’re a game-changer. Our platform is purpose-built to bridge the gap between server owners and those hungry for raw, high-performance computing power. 🖥️ We specialize in connecting suppliers from individual server owners to enterprise-scale GPU rig operators with consumers who demand serious horsepower for AI, machine learning, data science, and other compute-heavy applications. Whether you’re offering a sleek single-GPU setup or a massive mining-style array with multiple GPUs, BIG GPU Nodes is your launchpad. We make it simple to rent out underutilized GPU resources and just as easy for users to find exactly what they need — fast, affordable, and tailored to their workload.⛓️‍💥 This is more than cloud computing. This is decentralized GPU power for the future of innovation. ♾️ This product will generate revenue and we will give back to our loyal $CNCT holders. ✨ We’re proud to have MESSIER | M87 as our trusted partner and advisor, backing us with this vision. #M87 🤝 $CNCT From the very beginning, we promised you ONE BIG ECOSYSTEM, and a seamless space where everything you need comes together. Now, that vision is becoming reality. Millions of users will soon be connected, working, creating, and thriving within it. 🌍

Revolutionizing Access to #GPU Power - One Node at a Time.⚡️ Our 10th Utility is coming out on June 20.🚨 BIG GPU Nodes Platform isn’t just another name in the crowded GPU cloud services space, we’re a game-changer. Our platform is purpose-built to bridge the gap between server owners and those hungry for raw, high-performance computing power. 🖥️ We specialize in connecting suppliers from individual server owners to enterprise-scale GPU rig operators with consumers who demand serious horsepower for AI, machine learning, data science, and other compute-heavy applications. Whether you’re offering a sleek single-GPU setup or a massive mining-style array with multiple GPUs, BIG GPU Nodes is your launchpad. We make it simple to rent out underutilized GPU resources and just as easy for users to find exactly what they need — fast, affordable, and tailored to their workload.⛓️‍💥 This is more than cloud computing. This is decentralized GPU power for the future of innovation. ♾️ This product will generate revenue and we will give back to our loyal $CNCT holders. ✨ We’re proud to have MESSIER | M87 as our trusted partner and advisor, backing us with this vision. #M87 🤝 $CNCT From the very beginning, we promised you ONE BIG ECOSYSTEM, and a seamless space where everything you need comes together. Now, that vision is becoming reality. Millions of users will soon be connected, working, creating, and thriving within it. 🌍

Fixera AI

26,669 просмотров • 1 год назад

GLM 5.2 in pi running at almost 100 tokens/sec on 8x H200. I really like this model. It feels a frontier-level model you can actually self-host. Next step: squeeze more out of it by benchmarking different quants, tuning the stack, and figuring out the minimum GPU setup people can realistically run it on. Expect more on this soon.

GLM 5.2 in pi running at almost 100 tokens/sec on 8x H200. I really like this model. It feels a frontier-level model you can actually self-host. Next step: squeeze more out of it by benchmarking different quants, tuning the stack, and figuring out the minimum GPU setup people can realistically run it on. Expect more on this soon.

Atharva Ingle

26,408 просмотров • 11 дней назад

Kimi K2.5 is now live on GPU-accelerated endpoints for free prototyping, so you can quickly start building with a frontier-scale multimodal model instead of just reading about it. Want to get started? We’ve got you: • Step-by-step tutorial • Ready-to-run GitHub notebook • First inference in minutes, not hours 📖 All available in the technical blog →

Kimi K2.5 is now live on GPU-accelerated endpoints for free prototyping, so you can quickly start building with a frontier-scale multimodal model instead of just reading about it. Want to get started? We’ve got you: • Step-by-step tutorial • Ready-to-run GitHub notebook • First inference in minutes, not hours 📖 All available in the technical blog →

NVIDIA AI Developer

108,637 просмотров • 5 месяцев назад

$0.05 PER KWH IN PARAGUAY VS $0.15 IN THE US, A 5,000 GPU FARM NEAR ASUNCION CLEARS $4.7M/MO ON VAST AI WHILE US OPERATORS NET $1.8M ON THE SAME RACK the warehouse in the clip is somewhere outside asuncion, multi story racks running floor to ceiling, the operator built it next to a hydroelectric dam where electricity costs 5 cents per kilowatt hour a US warehouse running the same 5,000 GPU rack burns through $1.6M a month in electricity at $0.15/kWh, the same rack in paraguay burns $530K, the gap is $1M of pure margin every 30 days the racks host RTX 4090s, A100s and H100s on vast ai and clore ai, indie AI startups and image generation services rent them by the hour because aws is rationing GPU access to fortune 500 a 4090 nets the US operator $500 to $1,000 a month, the paraguay operator clears $850 to $1,400 on the same card because power eats half what it does north of the equator bitfarms, riot and core scientific have all expanded into paraguay since 2024, the next 10,000 GPU AI farm coming online in 2026 is more likely to sit near a paraguayan dam than in texas bookmark this and read the article below

$0.05 PER KWH IN PARAGUAY VS $0.15 IN THE US, A 5,000 GPU FARM NEAR ASUNCION CLEARS $4.7M/MO ON VAST AI WHILE US OPERATORS NET $1.8M ON THE SAME RACK the warehouse in the clip is somewhere outside asuncion, multi story racks running floor to ceiling, the operator built it next to a hydroelectric dam where electricity costs 5 cents per kilowatt hour a US warehouse running the same 5,000 GPU rack burns through $1.6M a month in electricity at $0.15/kWh, the same rack in paraguay burns $530K, the gap is $1M of pure margin every 30 days the racks host RTX 4090s, A100s and H100s on vast ai and clore ai, indie AI startups and image generation services rent them by the hour because aws is rationing GPU access to fortune 500 a 4090 nets the US operator $500 to $1,000 a month, the paraguay operator clears $850 to $1,400 on the same card because power eats half what it does north of the equator bitfarms, riot and core scientific have all expanded into paraguay since 2024, the next 10,000 GPU AI farm coming online in 2026 is more likely to sit near a paraguayan dam than in texas bookmark this and read the article below

starmex

26,916 просмотров • 20 дней назад

Qwen3.6 35B-A3B dropped yesterday, so I ran it on 4 GPUs to see how it performs: 🟣 RTX 3090 — 49.78 tok/s, TTFT 852ms 🟡 RTX 4090 — 118.93 tok/s, TTFT 686ms 🟢 RTX 5090 — 160.37 tok/s, TTFT 409ms 🔵 DGX Spark — 59.98 tok/s, TTFT 228ms I went with ollama as the backend because honestly, it's the easiest way for most people to get started. One command, model pulled, done. I used Q4_K_M (24GB) across all four cards. The reason is the 3090 and 4090 don't support NVFP4 (only the 5090 and DGX Spark could use it). Keeping the same quant everywhere felt like the fairest way to compare. And yes, you can absolutely squeeze more performance out of every card with vLLM, SGLang, or TensorRT-LLM. But that's not what this test is about. This is just the out-of-the-box experience for folks who own a GPU and want to try the new model tonight.

Qwen3.6 35B-A3B dropped yesterday, so I ran it on 4 GPUs to see how it performs: 🟣 RTX 3090 — 49.78 tok/s, TTFT 852ms 🟡 RTX 4090 — 118.93 tok/s, TTFT 686ms 🟢 RTX 5090 — 160.37 tok/s, TTFT 409ms 🔵 DGX Spark — 59.98 tok/s, TTFT 228ms I went with ollama as the backend because honestly, it's the easiest way for most people to get started. One command, model pulled, done. I used Q4_K_M (24GB) across all four cards. The reason is the 3090 and 4090 don't support NVFP4 (only the 5090 and DGX Spark could use it). Keeping the same quant everywhere felt like the fairest way to compare. And yes, you can absolutely squeeze more performance out of every card with vLLM, SGLang, or TensorRT-LLM. But that's not what this test is about. This is just the out-of-the-box experience for folks who own a GPU and want to try the new model tonight.

stevibe

398,628 просмотров • 2 месяцев назад

MiniMax M2.7 is 230B params. Can you actually run it at home? I tested Unsloth's UD-IQ3_XXS (80GB) on 4 different rigs: 🟠 4x RTX 4090 (96GB): 71.52 tok/s, TTFT 1045ms 🟢 4x RTX 5090 (128GB): 120.54 tok/s, TTFT 725ms 🟡 1x RTX PRO 6000 (96GB): 118.74 tok/s, TTFT 765ms 🟣 DGX Spark (128GB) — 24.41 tok/s, TTFT 741ms Backend: llama.cpp. Context: 32k. Max tokens: 4096. I went with IQ3_XXS because it's the biggest quant that fits in 96GB VRAM while still leaving safe headroom for 32k context. Same quant across all four rigs, fairest comparison I could run. Now look at rough peak GPU power draw: 🟠 4x4090 → 1,800W peak (450W × 4) 🟢 4x5090 → 2,300W peak (575W × 4) 🟡 RTX PRO 6000 → 600W peak 🟣 DGX Spark → 240W peak (whole system) The RTX PRO 6000 is the quiet winner. One card, 96GB, matching a 4x5090 rig at roughly a quarter of the power and zero multi-GPU headaches. Best tokens-per-watt by a wide margin. DGX Spark is slow on generation but pulls the least power of any rig here, around 240W for the whole system. Prefill-friendly, memory-rich, wall-socket-friendly. And yes, plenty of people cap their cards. Even then, 4x 4090 or 4x 5090 still pulls well over 1,200W from the GPUs alone.

MiniMax M2.7 is 230B params. Can you actually run it at home? I tested Unsloth's UD-IQ3_XXS (80GB) on 4 different rigs: 🟠 4x RTX 4090 (96GB): 71.52 tok/s, TTFT 1045ms 🟢 4x RTX 5090 (128GB): 120.54 tok/s, TTFT 725ms 🟡 1x RTX PRO 6000 (96GB): 118.74 tok/s, TTFT 765ms 🟣 DGX Spark (128GB) — 24.41 tok/s, TTFT 741ms Backend: llama.cpp. Context: 32k. Max tokens: 4096. I went with IQ3_XXS because it's the biggest quant that fits in 96GB VRAM while still leaving safe headroom for 32k context. Same quant across all four rigs, fairest comparison I could run. Now look at rough peak GPU power draw: 🟠 4x4090 → 1,800W peak (450W × 4) 🟢 4x5090 → 2,300W peak (575W × 4) 🟡 RTX PRO 6000 → 600W peak 🟣 DGX Spark → 240W peak (whole system) The RTX PRO 6000 is the quiet winner. One card, 96GB, matching a 4x5090 rig at roughly a quarter of the power and zero multi-GPU headaches. Best tokens-per-watt by a wide margin. DGX Spark is slow on generation but pulls the least power of any rig here, around 240W for the whole system. Prefill-friendly, memory-rich, wall-socket-friendly. And yes, plenty of people cap their cards. Even then, 4x 4090 or 4x 5090 still pulls well over 1,200W from the GPUs alone.

stevibe

191,728 просмотров • 2 месяцев назад

AMD has the opportunity to make the price to performance king. RX 9070 XT will be miles better than RTX 5080 if priced appropriately. We know by now that FSR 4 is miles better than FSR 3.1 as shown in this video here (credit: Hardware Unboxed) With the right marketing and highlighting the performance and quality jump that these new GPUs have, I could see AMD gaining a lot of GPU market share.

AMD has the opportunity to make the price to performance king. RX 9070 XT will be miles better than RTX 5080 if priced appropriately. We know by now that FSR 4 is miles better than FSR 3.1 as shown in this video here (credit: Hardware Unboxed) With the right marketing and highlighting the performance and quality jump that these new GPUs have, I could see AMD gaining a lot of GPU market share.

NikTek

61,166 просмотров • 1 год назад

six months ago this wasn't happening on 8gb vram. running unsloth's Q4_K_XL quant of gemma 4 26b-a4b-it-qat, a sparse MoE model with only 4b active params on a single rtx 4060 laptop gpu, 8gb vram, 20+ tok/s decode. no cloud, no api, no offload hacks. just a gaming laptop on battery. what makes it fit: google's QAT (quantization aware training), plus MTP (multi token prediction) support in the latest llama.cpp builds. that combo is the single biggest unlock for local inference on low vram. rtx 3060, rtx 3070, gtx 1070, gtx 1080, rtx 4050, rtx 4060, rtx 5050, rtx 5060 — any 6-8gb consumer gpu, old or new — this model runs on it. world cup season, so i told it to build a soccer themed flappy bird clone. one shot, zero iteration, fully playable. six months ago an 8gb model could barely clone vanilla flappy bird. now it's shipping a themed game from a sparse MoE model running locally on a laptop battery. inference benchmarks: - decode throughput: 30 tok/s - context: 64k. this is the real unlock. 64k ctx is what makes a hermes agent loop viable locally on this model, not just single-turn chat. llama.cpp flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 64000 -cmoe --port 8080 game's deployed on my own site, built and shipped end to end with open source llm, zero closed source api dependency in the pipeline. link in the description. gguf weights on huggingface, link in the comments. pull it down, run it on whatever 8gb card is sitting in your rig. try the game and tell me your score and what you want in v2. local llms on consumer gpus stopped being a meme.

six months ago this wasn't happening on 8gb vram. running unsloth's Q4_K_XL quant of gemma 4 26b-a4b-it-qat, a sparse MoE model with only 4b active params on a single rtx 4060 laptop gpu, 8gb vram, 20+ tok/s decode. no cloud, no api, no offload hacks. just a gaming laptop on battery. what makes it fit: google's QAT (quantization aware training), plus MTP (multi token prediction) support in the latest llama.cpp builds. that combo is the single biggest unlock for local inference on low vram. rtx 3060, rtx 3070, gtx 1070, gtx 1080, rtx 4050, rtx 4060, rtx 5050, rtx 5060 — any 6-8gb consumer gpu, old or new — this model runs on it. world cup season, so i told it to build a soccer themed flappy bird clone. one shot, zero iteration, fully playable. six months ago an 8gb model could barely clone vanilla flappy bird. now it's shipping a themed game from a sparse MoE model running locally on a laptop battery. inference benchmarks: - decode throughput: 30 tok/s - context: 64k. this is the real unlock. 64k ctx is what makes a hermes agent loop viable locally on this model, not just single-turn chat. llama.cpp flags: -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf -c 64000 -cmoe --port 8080 game's deployed on my own site, built and shipped end to end with open source llm, zero closed source api dependency in the pipeline. link in the description. gguf weights on huggingface, link in the comments. pull it down, run it on whatever 8gb card is sitting in your rig. try the game and tell me your score and what you want in v2. local llms on consumer gpus stopped being a meme.

Alok

59,908 просмотров • 23 дней назад

a new 8GB VRAM GPU dense Local LLM leader was born yesterday runs on: RTX 4060 / RTX 3070 / RTX 2080. any 8GB card Qwen 3.5 9B (dense) was the go to for 6-8GB VRAM builds. Gemma 4 12B QAT (dense) just changed that. same llama.cpp + cuda 13.2. i7 12700H. 16GB RAM. same -ngl 99 flags. same 48k context. unsloth gemma-4-12b-it-Q4_K_M.gguf → 15 tok/sec @ 48k ctx unsloth gemma-4-12B-it-qat-UD-Q4_K_XL.gguf → 32 tok/sec @ 48k ctx → 26 tok/sec @ 64k ctx 64k context is a big deal. Hermes 3 agent requires 64k minimum to run. you're now getting full hermes compatible context on a budget consumer GPU at 26 tok/sec locally. 2.1x faster on identical hardware. and here's the part that breaks your brain: the QAT-UD-Q4_K_XL is actually SMALLER than the Q4_K_M "XL" why? QAT = Quantization Aware Training Google didn't train the model first and compress it later they trained it to be quantized from day one the weights already know how to survive low precision that's why you get more quality per byte llamacpp flags: -m gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -cnv -ngl 99 -c 48000 -v fits in 8GB VRAM clean. no API. no cloud. no subscription. and this isn't even the MTP variant yet Gemma-4-E2B QAT runs on 3GB RAM, E4B on 5GB, 12B on 7GB, 26-A4B on 15GB and 31B on 18GB. I have benchmarked the 26b and 31b qat as well on a single RTX 4090, checkout the comments for details. If you have a 6GB or 8GB VRAM GPU, post your numbers. more benchmarks and configs coming soon

a new 8GB VRAM GPU dense Local LLM leader was born yesterday runs on: RTX 4060 / RTX 3070 / RTX 2080. any 8GB card Qwen 3.5 9B (dense) was the go to for 6-8GB VRAM builds. Gemma 4 12B QAT (dense) just changed that. same llama.cpp + cuda 13.2. i7 12700H. 16GB RAM. same -ngl 99 flags. same 48k context. unsloth gemma-4-12b-it-Q4_K_M.gguf → 15 tok/sec @ 48k ctx unsloth gemma-4-12B-it-qat-UD-Q4_K_XL.gguf → 32 tok/sec @ 48k ctx → 26 tok/sec @ 64k ctx 64k context is a big deal. Hermes 3 agent requires 64k minimum to run. you're now getting full hermes compatible context on a budget consumer GPU at 26 tok/sec locally. 2.1x faster on identical hardware. and here's the part that breaks your brain: the QAT-UD-Q4_K_XL is actually SMALLER than the Q4_K_M "XL" why? QAT = Quantization Aware Training Google didn't train the model first and compress it later they trained it to be quantized from day one the weights already know how to survive low precision that's why you get more quality per byte llamacpp flags: -m gemma-4-12B-it-qat-UD-Q4_K_XL.gguf -cnv -ngl 99 -c 48000 -v fits in 8GB VRAM clean. no API. no cloud. no subscription. and this isn't even the MTP variant yet Gemma-4-E2B QAT runs on 3GB RAM, E4B on 5GB, 12B on 7GB, 26-A4B on 15GB and 31B on 18GB. I have benchmarked the 26b and 31b qat as well on a single RTX 4090, checkout the comments for details. If you have a 6GB or 8GB VRAM GPU, post your numbers. more benchmarks and configs coming soon

Alok

259,993 просмотров • 1 месяц назад

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Daniel Romero

38,220 просмотров • 3 дней назад

Nvidia announces the new RTX Spark, a new platform powered by the NX1 CPU, and shows off Spark laptops running 007 First Light and Forza 6. The CPU has 20 ARM based cores and a Blackwell RTX GPU with 6144 CUDA Cores. This is the same core count as a 5070, but with 128GB of unified LPDDR5X RAM memory sitting in the same package as the CPU and GPU. The entire Nvidia software stack is available, particularly CUDA, vital for AI. Nvidia's new laptops will likely be ideal for running local LLMs be cause the unified memory means you can load models up to 120-180B parameters (quantized). These laptops are expected to ship later this year and could become strong competitors to high-end MacBooks and even Mac Studios for local AI workloads, thanks to CUDA support and unified memory. Price is unannounced.

Nvidia announces the new RTX Spark, a new platform powered by the NX1 CPU, and shows off Spark laptops running 007 First Light and Forza 6. The CPU has 20 ARM based cores and a Blackwell RTX GPU with 6144 CUDA Cores. This is the same core count as a 5070, but with 128GB of unified LPDDR5X RAM memory sitting in the same package as the CPU and GPU. The entire Nvidia software stack is available, particularly CUDA, vital for AI. Nvidia's new laptops will likely be ideal for running local LLMs be cause the unified memory means you can load models up to 120-180B parameters (quantized). These laptops are expected to ship later this year and could become strong competitors to high-end MacBooks and even Mac Studios for local AI workloads, thanks to CUDA support and unified memory. Price is unannounced.

Grummz

30,573 просмотров • 1 месяц назад

$Access codes will go out tomorrow via welcome emails. API status updates and full API access will follow on Friday. HdeAPoHivsm9MZfeY5tW7apJEprc8Fs594bWmnzfpump The $ICPX website has been rebuilt from the ground up and it reflects exactly where this protocol is headed. The new site is the gateway to decentralised compute powered by Nvidia and Google GPU cloud partner programs, meaning whenever you rent a GPU for a job, it is always available. No downtime, no waitlists. Enterprise grade H100s, H200s and RTX 4090s. Accessible to anyone, at a fraction of what centralised providers charge. Pay with $ICPX tokens and get immediate access to serious compute power. Stake $ICPX to unlock deeper discounts and watch your costs drop over time the more committed you are to the network. This is what pay-per-job decentralised compute actually looks like in practice. Full control over training and running your models, with a lean 0.25% protocol fee managed by multisig. Open-source, immutable, and formally verified with Kani. $$

Access codes will go out tomorrow via welcome emails. API status updates and full API access will follow on Friday. HdeAPoHivsm9MZfeY5tW7apJEprc8Fs594bWmnzfpump The $ICPX website has been rebuilt from the ground up and it reflects exactly where this protocol is headed. The new site is the gateway to decentralised compute powered by Nvidia and Google GPU cloud partner programs, meaning whenever you rent a GPU for a job, it is always available. No downtime, no waitlists. Enterprise grade H100s, H200s and RTX 4090s. Accessible to anyone, at a fraction of what centralised providers charge. Pay with $ICPX tokens and get immediate access to serious compute power. Stake $ICPX to unlock deeper discounts and watch your costs drop over time the more committed you are to the network. This is what pay-per-job decentralised compute actually looks like in practice. Full control over training and running your models, with a lean 0.25% protocol fee managed by multisig. Open-source, immutable, and formally verified with Kani. $

ICPX Protocol

32,161 просмотров • 26 дней назад