Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

We built the first GPU-accelerated metal casting solver. FastCast by Digital Metal does fill, solidification, porosity & distortion on a single NVIDIA GPU, 91× faster than an industry-leading $15k/yr solver with the same level of accuracy. Verified against real world results

Connor Kapoor

9,709 subscribers

18,898 Aufrufe • vor 20 Tagen •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

20 Kommentare

Profilbild von Connor Kapoor

Connor Kapoorvor 20 Tagen

Casting simulation before fastcast used to mean an overnight solve on a sequential CPU solver before you could even change one variable, killing iteration and manufacturing lead times. Not to mention its' eye-watering prices. FastCast by Digital Metal shortens solve time to an average of just 5 minutes for most parts & trees, making casting optimization an easy, and quick path- increasing iteration and reducing lead time for our customers.

Profilbild von Connor Kapoor

Connor Kapoorvor 20 Tagen

One run, four answers. It shows how the metal fills the mould, what freezes last, where shrinkage porosity will form, and how the casting will warp once it cools. All from one upload.

Profilbild von Connor Kapoor

Connor Kapoorvor 20 Tagen

Why it's fast: it was built for the GPU, not ported to it. Every stage runs on the same voxel grid in GPU memory, with no remeshing and no hand-off to the CPU between stages. A flanged hub in 17-4PH took 36 seconds on a laptop GPU, against 54 minutes on the industry leading reference solver.

Profilbild von Connor Kapoor

Connor Kapoorvor 20 Tagen

FastCast is just one of the tools we've built from the ground up at Digital Metal to make casting move from 16 weeks in average lead time, down to days. We're making the solver playground public for anyone who'd like to try and benchmark. (Be warned there is a queue for cloud run GPUs)

Profilbild von Harris Rothaermel

Harris Rothaermelvor 20 Tagen

badass

Profilbild von Carlos DP 🤖🇺🇸

Carlos DP 🤖🇺🇸vor 20 Tagen

@zachglabman Sick!

Profilbild von Andrea B. Kalmans

Andrea B. Kalmansvor 20 Tagen

Wild

Profilbild von Parth Ingle

Parth Inglevor 20 Tagen

🔥🔥🔥

Profilbild von Ryne Ogren

Ryne Ogrenvor 20 Tagen

91x faster? Wow

Profilbild von Dave

Davevor 20 Tagen

Sick!

Profilbild von tmo

tmovor 20 Tagen

@digitalmetalinc Nice work! I have been trying to build the same thing for the last few months tapping into a pair of 3090’s (very much an on and off again process)

Profilbild von Boardy

Boardyvor 20 Tagen

@andrewdsouza take a look: Digital Metal’s FastCast casting solver for foundries. I can help them reach foundry teams evaluating simulation software.

Profilbild von Jack Aspinall

Jack Aspinallvor 20 Tagen

Thats epic! Nice work

Profilbild von Levy Strouse

Levy Strousevor 20 Tagen

That’s impressive!

Profilbild von Andrew Hughes

Andrew Hughesvor 20 Tagen

So elite

Profilbild von pymoment

pymomentvor 20 Tagen

🔥

Profilbild von Mohammad Nafisi

Mohammad Nafisivor 20 Tagen

🔥🔥🔥

Profilbild von Paul Stetson

Paul Stetsonvor 20 Tagen

@zachglabman Awesome

Profilbild von savelii

saveliivor 20 Tagen

cool

Profilbild von Banana Republican

Banana Republicanvor 20 Tagen

Is this something i could run locally?

Ähnliche Videos

Introducing Parrot 🦜 Our first speech-to-text model built for production-grade voice agents with an industry leading accuracy, built for noisy calls, regional dialects and compressed audio. Trained on thousands of hours of real-world audio to make sure when you say Udaipur, the agent doesn't hear 'you-die-poor'.

Introducing Parrot 🦜 Our first speech-to-text model built for production-grade voice agents with an industry leading accuracy, built for noisy calls, regional dialects and compressed audio. Trained on thousands of hours of real-world audio to make sure when you say Udaipur, the agent doesn't hear 'you-die-poor'.

Vedant

15,977 Aufrufe • vor 5 Monaten

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,770 Aufrufe • vor 2 Jahren

Today we're thrilled to announce the first end-to-end platform for Reinforcement Fine-Tuning. With just a dozen labeled data points, you can outperform #OpenAI o1 and #DeepSeekR1 on complex tasks. Built on the #GRPO methodology that DeepSeek-R1 popularized, our platform delivers exceptional results. In our real-world PyTorch to Triton transpilation case study, we achieved 3x higher accuracy than OpenAI o1 and DeepSeek-R1 when writing GPU code. Check out the thread below to learn how you can adapt an #opensource #LLM to your use cases with unmatched efficiency. #rft

Today we're thrilled to announce the first end-to-end platform for Reinforcement Fine-Tuning. With just a dozen labeled data points, you can outperform #OpenAI o1 and #DeepSeekR1 on complex tasks. Built on the #GRPO methodology that DeepSeek-R1 popularized, our platform delivers exceptional results. In our real-world PyTorch to Triton transpilation case study, we achieved 3x higher accuracy than OpenAI o1 and DeepSeek-R1 when writing GPU code. Check out the thread below to learn how you can adapt an #opensource #LLM to your use cases with unmatched efficiency. #rft

Predibase by Rubrik

1,022,546 Aufrufe • vor 1 Jahr

$Someone converted the Mandelbrot set into sound by mapping fractal orbits to audio frequencies in real time on a GPU - and accidentally built an infinite synthesizer. Every point in the fractal converges to a repeating orbit with its own period and shape. Map those coordinates to speaker amplitude at 8kHz and you get an instrument where zooming in adds harmonics on top of harmonics - the same principle behind every DSP audio engine ever built. 8 minutes. Bookmark & watch today. The most unexpected use of a GPU you'll see this week.$

Someone converted the Mandelbrot set into sound by mapping fractal orbits to audio frequencies in real time on a GPU - and accidentally built an infinite synthesizer. Every point in the fractal converges to a repeating orbit with its own period and shape. Map those coordinates to speaker amplitude at 8kHz and you get an instrument where zooming in adds harmonics on top of harmonics - the same principle behind every DSP audio engine ever built. 8 minutes. Bookmark & watch today. The most unexpected use of a GPU you'll see this week.

Eron

27,390 Aufrufe • vor 2 Monaten

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Dylan Patel on the importance of memory and storage Two key quotes: "An $NVDA GPU is faster than an $AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads." “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly" Full Quote: “We have over $80 million of compute: GPUs from $NVDA and $AMD, TPUs from Google, and Trainium from Amazon. We constantly run this benchmark using the newest inference engines, drivers, PyTorch versions, and other software. It runs every day through automated CI across the latest Chinese models from GLM, Zhipu, Moonshot, Kimi, Alibaba, and others. Initially, when we were benchmarking the differences between these chips, inference engines, and parallelism schemes, we used fixed context lengths. But with Agent X, we have now analyzed more than $5 million worth of Claude Code traces. This is real production traffic that users have donated to us, combined with internally generated data, so we now understand what an actual agent workload looks like. When we implement those workloads and run the benchmarks, it turns out that the chip you are using is very important, but how you handle memory offload can be even more important. An Nvidia GPU is faster than an AMD GPU in most cases, but because AMD GPUs have more memory, they can outperform Nvidia in certain workloads. Similarly, you can use a less powerful GPU with a much better storage solution and outperform the best GPU when it lacks those solutions. Simply buying the newest GPU does not necessarily give you the best inference economics. You need to layer in other innovations, including storage and memory.” Interviewer: “Who is the top player on your chart? Can you tell us?” Dylan Patel: “It is a difficult, multivariable problem. Generally, you need the best GPU, such as a GB300, but you also need the best storage solutions. I will not spoil who comes out on top, but storage solutions matter a lot, memory solutions matter a lot, and frontend networking also matters significantly.”

Daniel Romero

48,978 Aufrufe • vor 3 Monaten

“The GPU is the most important commodity of our decade. But if you bought a single chip, you couldn’t get a loan against it.” — David | www.usd.ai USD.AI — a stablecoin backed by the GPUs powering AI — is one of three real-world assets accepted as collateral in the Bitwise Premium RWA Vault on Morpho 🦋. David Choi, CEO of Permian Labs, in conversation with Jonathan Man:

“The GPU is the most important commodity of our decade. But if you bought a single chip, you couldn’t get a loan against it.” — David | www.usd.ai USD.AI — a stablecoin backed by the GPUs powering AI — is one of three real-world assets accepted as collateral in the Bitwise Premium RWA Vault on Morpho 🦋. David Choi, CEO of Permian Labs, in conversation with Jonathan Man:

Bitwise

38,196 Aufrufe • vor 1 Monat

We are back again :) After three weeks of quiet building. Introducing Genesis World 1.0, our latest simulation platform, the second release in our full-stack suite. Open-sourced. Robotics is still bottlenecked by the 1× speed of the physical world. Every model, checkpoint, and data recipe eventually needs to be tested on physical hardware, slowly, expensively, and with limited coverage. One hour in reality can become 100 days in simulation. That is how robotics model iteration moves from a wall-clock bottleneck to a compute problem. To make this work, simulation has to be both fast and trustworthy. Over the past year, we rebuilt the entire stack: a GPU-accelerated cross-platform compiler, penetration-free multi-physics contact solvers, unified rigid and deformable physics, and a photo-realistic renderer purpose-built for physical AI applications. We built Nyx, a high-performance path-traced rendering engine for robotics application. Genesis World 1.0 achieves near realtime performance with our latest development for penetration-free IPC solver, supporting various types of deformables beyond rigid bodies. It supports contact-rich, dexterous manipulation simulation across different embodiments: unitree, sharpa, wuji, genesis hand and various types of grippers. Under the hood is Quadrants, our effort in pushing forward cross-platform GPU-accelerated computation. Quadrants started as a fork of Taichi, and we rebuilt most of the critical parts for optimizing simulation workloads, giving 10x faster launch time and up to 4.6x runtime performance compared to the initial Genesis release. Together, they bring us to an unprecedentedly low sim-to-real gap, enabling zero-shot real-to-sim model evaluation and much faster iteration of GENE. All available today. Genesis World 1.0: Quadrants: Nyx:

We are back again :) After three weeks of quiet building. Introducing Genesis World 1.0, our latest simulation platform, the second release in our full-stack suite. Open-sourced. Robotics is still bottlenecked by the 1× speed of the physical world. Every model, checkpoint, and data recipe eventually needs to be tested on physical hardware, slowly, expensively, and with limited coverage. One hour in reality can become 100 days in simulation. That is how robotics model iteration moves from a wall-clock bottleneck to a compute problem. To make this work, simulation has to be both fast and trustworthy. Over the past year, we rebuilt the entire stack: a GPU-accelerated cross-platform compiler, penetration-free multi-physics contact solvers, unified rigid and deformable physics, and a photo-realistic renderer purpose-built for physical AI applications. We built Nyx, a high-performance path-traced rendering engine for robotics application. Genesis World 1.0 achieves near realtime performance with our latest development for penetration-free IPC solver, supporting various types of deformables beyond rigid bodies. It supports contact-rich, dexterous manipulation simulation across different embodiments: unitree, sharpa, wuji, genesis hand and various types of grippers. Under the hood is Quadrants, our effort in pushing forward cross-platform GPU-accelerated computation. Quadrants started as a fork of Taichi, and we rebuilt most of the critical parts for optimizing simulation workloads, giving 10x faster launch time and up to 4.6x runtime performance compared to the initial Genesis release. Together, they bring us to an unprecedentedly low sim-to-real gap, enabling zero-shot real-to-sim model evaluation and much faster iteration of GENE. All available today. Genesis World 1.0: Quadrants: Nyx:

Genesis AI

325,778 Aufrufe • vor 4 Monaten

We are releasing Carbon: a crazy fast DNA model Carbon is 275x faster than the next best model. So fast you can process the whole human genome on a single GPU in <2 days. Here are the tricks we used: When modelling DNA sequences a lot of the performance comes down to tokenizing the sequences in a smart way. BPE tokenizer struggle because there are no whitespaces and character (called base in DNA) level tokenizers waste a lot of compute on too many tokens. Carbon is built with a unique tokenizer: we split sequences in chunks of 6 bases, but during both training and inference we can work with single base resolution. That's similar to having word tokens but resolving them at the character level. All possible thanks to the DNA tokens unique structure. The architecture combined with the tokenizer makes the model 275x faster than the previous SoTA (Evo2) at this size. We built an interactive demo so you can explore how the model can generate DNA sequences, investigate the structure of genes, predict the effect of mutations, generate and fold proteins and even reconstruct parts of the tree of life.

We are releasing Carbon: a crazy fast DNA model Carbon is 275x faster than the next best model. So fast you can process the whole human genome on a single GPU in <2 days. Here are the tricks we used: When modelling DNA sequences a lot of the performance comes down to tokenizing the sequences in a smart way. BPE tokenizer struggle because there are no whitespaces and character (called base in DNA) level tokenizers waste a lot of compute on too many tokens. Carbon is built with a unique tokenizer: we split sequences in chunks of 6 bases, but during both training and inference we can work with single base resolution. That's similar to having word tokens but resolving them at the character level. All possible thanks to the DNA tokens unique structure. The architecture combined with the tokenizer makes the model 275x faster than the previous SoTA (Evo2) at this size. We built an interactive demo so you can explore how the model can generate DNA sequences, investigate the structure of genes, predict the effect of mutations, generate and fold proteins and even reconstruct parts of the tree of life.

Leandro von Werra

405,979 Aufrufe • vor 5 Monaten

What Lisa Su actually held on stage: A mini PC the size of a lunchbox running Qwen3-235B locally, with no cloud and no discrete GPU Inside: the Ryzen AI Max+ 395, 128GB unified memory, 110GB usable as VRAM on Linux The first x86 chip that handles 200+ billion parameters on a single die AMD claims it beats the RTX 5080 by several times on memory-bound models — because the 5080 simply cannot fit them $1,400 to $2,500 once. cloud bills run $200 to $400 a month It pays for itself in a few months, then costs nothing per request This is not a faster GPU. it is the first real argument that your AI does not belong in someone else's data center

What Lisa Su actually held on stage: A mini PC the size of a lunchbox running Qwen3-235B locally, with no cloud and no discrete GPU Inside: the Ryzen AI Max+ 395, 128GB unified memory, 110GB usable as VRAM on Linux The first x86 chip that handles 200+ billion parameters on a single die AMD claims it beats the RTX 5080 by several times on memory-bound models — because the 5080 simply cannot fit them $1,400 to $2,500 once. cloud bills run $200 to $400 a month It pays for itself in a few months, then costs nothing per request This is not a faster GPU. it is the first real argument that your AI does not belong in someone else's data center

plutos

1,239,389 Aufrufe • vor 4 Monaten

QVAC SDK 0.14.0 is live. This release makes the on-device stack faster on mobile, ships the developer-agent path, and takes local text-to-speech to 31 languages. Main highlights: - OpenCode and OpenClaw. The first official OpenCode plugin, plus a maintained OpenClaw compatibility path, both built on managed mode and qvac serve. Point a coding agent at a local model with far less setup and far fewer surprises. - Brain-computer interface transcription, on the SDK. Take recorded neural signal data and decode it into text, fully on-device, no cloud. Stream it in chunks through a simple API. In 0.14 it runs GPU-accelerated on iOS. - Text to Speech in 31 languages with our Supertonic3 upgrade. VOICE AND SPEECH - Supertonic3 multilingual TTS, 5 languages to 31. - Chatterbox and Supertonic now run on the Android GPU, with lower memory use (especially on iOS), quantized s3gen Chatterbox support, and a fix for Chatterbox occasionally emitting random speech. - Whisper transcription now runs on the iOS GPU. Parakeet runs on the Android GPU, with steadier real-time streaming. VISION AND OCR - VLM multi-tile batching: high-resolution Pan and Scan images are encoded in one pass instead of tile by tile, for faster vision throughput. - OCR on ggml (EasyOCR and DocTR) reaches full speed parity with the onnx path, across Metal, OpenCL, and Vulkan. PLATFORM AND RELIABILITY - Dynamic compute backends on Linux: one build picks the right backend at runtime, and opens the door to ROCm and CUDA support without per-backend builds. - Thinking tokens are kept out of the model context, so reasoning no longer fills the KV cache. SDK 0.14.0 is now leaner and faster to start. Let’s build.

QVAC SDK 0.14.0 is live. This release makes the on-device stack faster on mobile, ships the developer-agent path, and takes local text-to-speech to 31 languages. Main highlights: - OpenCode and OpenClaw. The first official OpenCode plugin, plus a maintained OpenClaw compatibility path, both built on managed mode and qvac serve. Point a coding agent at a local model with far less setup and far fewer surprises. - Brain-computer interface transcription, on the SDK. Take recorded neural signal data and decode it into text, fully on-device, no cloud. Stream it in chunks through a simple API. In 0.14 it runs GPU-accelerated on iOS. - Text to Speech in 31 languages with our Supertonic3 upgrade. VOICE AND SPEECH - Supertonic3 multilingual TTS, 5 languages to 31. - Chatterbox and Supertonic now run on the Android GPU, with lower memory use (especially on iOS), quantized s3gen Chatterbox support, and a fix for Chatterbox occasionally emitting random speech. - Whisper transcription now runs on the iOS GPU. Parakeet runs on the Android GPU, with steadier real-time streaming. VISION AND OCR - VLM multi-tile batching: high-resolution Pan and Scan images are encoded in one pass instead of tile by tile, for faster vision throughput. - OCR on ggml (EasyOCR and DocTR) reaches full speed parity with the onnx path, across Metal, OpenCL, and Vulkan. PLATFORM AND RELIABILITY - Dynamic compute backends on Linux: one build picks the right backend at runtime, and opens the door to ROCm and CUDA support without per-backend builds. - Thinking tokens are kept out of the model context, so reasoning no longer fills the KV cache. SDK 0.14.0 is now leaner and faster to start. Let’s build.

QVAC

23,996,144 Aufrufe • vor 3 Monaten

Dylan Patel of SemiAnalysis says a worse GPU with better storage and memory now beats the best chip without them, so buying the newest GPU alone no longer wins inference. So, an AMD GPU with more memory can outperform Nvidia in some cases. "So what we have is we have over $80 million of compute, GPUs from Nvidia, AMD, TPUs from Google, Trainium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest PyTorch version, whatever it is." "Every day it runs on an automated CI, and we run it on all the latest Chinese models, from GLM, Zhipu, Moonshot, Kimi, Alibaba, all these models we run." "Initially, when we were benchmarking the difference between these chips and different engines, different schemes for parallelism, we were just running it fixed context length." "But now with Agent X, we've analyzed over $5 million worth of Claude Code traces. This is real production traffic that people have donated to us as well as internally generated. Now we know what the actual agent workload looks like." "And then as we implement that and run those benchmarks, it turns out yes, the chip you're using is very important, but now even more important is how are you handling this memory offload?" "And so while an Nvidia GPU is faster than an AMD GPU in most cases, because AMD GPUs have more memory, they actually end up outperforming in some cases." "Or you can have a worse GPU, but a much better storage solution, and now you can outperform what the best GPU can do without those solutions. So just buying the newest and latest GPU alone doesn't get you the best inference economics." "Actually, you need to layer in all these other innovations including storage and memory." [ Who's the top player on your chart? ] "That really is a difficult multivariable problem. And generally that means you need to have, yes, you need to have the best GPU, a GB300, but you also need to have the best storage solutions. And so I won't spoil who's the best right here, but I will say that storage solutions matter a lot and memory solutions matter a lot, as does your front-end networking. That matters a lot."

Dylan Patel of SemiAnalysis says a worse GPU with better storage and memory now beats the best chip without them, so buying the newest GPU alone no longer wins inference. So, an AMD GPU with more memory can outperform Nvidia in some cases. "So what we have is we have over $80 million of compute, GPUs from Nvidia, AMD, TPUs from Google, Trainium from Amazon, and we run this benchmark constantly on the newest inference engine, newest drivers, newest PyTorch version, whatever it is." "Every day it runs on an automated CI, and we run it on all the latest Chinese models, from GLM, Zhipu, Moonshot, Kimi, Alibaba, all these models we run." "Initially, when we were benchmarking the difference between these chips and different engines, different schemes for parallelism, we were just running it fixed context length." "But now with Agent X, we've analyzed over $5 million worth of Claude Code traces. This is real production traffic that people have donated to us as well as internally generated. Now we know what the actual agent workload looks like." "And then as we implement that and run those benchmarks, it turns out yes, the chip you're using is very important, but now even more important is how are you handling this memory offload?" "And so while an Nvidia GPU is faster than an AMD GPU in most cases, because AMD GPUs have more memory, they actually end up outperforming in some cases." "Or you can have a worse GPU, but a much better storage solution, and now you can outperform what the best GPU can do without those solutions. So just buying the newest and latest GPU alone doesn't get you the best inference economics." "Actually, you need to layer in all these other innovations including storage and memory." [ Who's the top player on your chart? ] "That really is a difficult multivariable problem. And generally that means you need to have, yes, you need to have the best GPU, a GB300, but you also need to have the best storage solutions. And so I won't spoil who's the best right here, but I will say that storage solutions matter a lot and memory solutions matter a lot, as does your front-end networking. That matters a lot."

Fireside Alpha

185,279 Aufrufe • vor 2 Monaten

World, Sam Altman's digital identity project, just unveiled World ID 4.0, what the company calls "full-stack proof of human" infrastructure. The partner list: Tinder, Zoom, DocuSign, Shopify, Okta, AWS, and Vercel. Altman opened by saying we're heading to a world where AI generates more content than humans. Pantera Capital says we've already crossed that threshold. World's answer is an iris-scanning device called the Orb that creates a unique cryptographic ID proving you're a real person. 18 million people across 160 countries have already verified. Tinder is rolling out "verified human" badges in the U.S. after a Japan pilot. Zoom built a feature called "Deep Face" that verifies the person on a video call isn't a deepfake. DocuSign is adding proof-of-human checks to digital signatures. Shopify is enabling verified-human commerce. The most significant announcement is AgentKit, infrastructure that lets AI agents carry cryptographic proof they're acting on behalf of a verified human. Okta built an agent delegation system on top of it. The problem World is solving is real. The question is whether a centralized iris-scanning identity layer controlled by the same person whose company helped create the problem is the right answer. Altman is the CEO of OpenAI. He built the flood. Now he's selling the ark.

World, Sam Altman's digital identity project, just unveiled World ID 4.0, what the company calls "full-stack proof of human" infrastructure. The partner list: Tinder, Zoom, DocuSign, Shopify, Okta, AWS, and Vercel. Altman opened by saying we're heading to a world where AI generates more content than humans. Pantera Capital says we've already crossed that threshold. World's answer is an iris-scanning device called the Orb that creates a unique cryptographic ID proving you're a real person. 18 million people across 160 countries have already verified. Tinder is rolling out "verified human" badges in the U.S. after a Japan pilot. Zoom built a feature called "Deep Face" that verifies the person on a video call isn't a deepfake. DocuSign is adding proof-of-human checks to digital signatures. Shopify is enabling verified-human commerce. The most significant announcement is AgentKit, infrastructure that lets AI agents carry cryptographic proof they're acting on behalf of a verified human. Okta built an agent delegation system on top of it. The problem World is solving is real. The question is whether a centralized iris-scanning identity layer controlled by the same person whose company helped create the problem is the right answer. Altman is the CEO of OpenAI. He built the flood. Now he's selling the ark.

TFTC

1,426,691 Aufrufe • vor 6 Monaten

Introducing a new look, for a new world. At Verse, we’re on a mission to redefine digital environments. We aim to bridge the gap between the physical and digital worlds by providing our users with an experience that closely mimics real-life interactions. This is a world built for players, brands and communities to thrive in a new era of experiential gameplay and engagement. Welcome to the Internet of Reality. Welcome to Verse World.

Introducing a new look, for a new world. At Verse, we’re on a mission to redefine digital environments. We aim to bridge the gap between the physical and digital worlds by providing our users with an experience that closely mimics real-life interactions. This is a world built for players, brands and communities to thrive in a new era of experiential gameplay and engagement. Welcome to the Internet of Reality. Welcome to Verse World.

Verse World

64,603 Aufrufe • vor 1 Jahr

$THIS CLUSTER OF 4 MAC STUDIOS USES LESS POWER THAN ONE NVIDIA CHIP - AND COSTS 5X LESS 17:30 the number hits hard - 600 watts for all four Mac Studios - less than one H200 that costs $30,000 on its own instead of one $30,000 GPU you get 4 machines with 1.5TB of memory, 3x more performance and models that simply cannot run on a single chip Apple quietly added RDMA over Thunderbolt in a beta update - latency dropped 99% and the same cluster became 3x faster without buying any new hardware before the update: 5 tokens per second. after: 15.5 - a free upgrade through a software update 8 Nvidia DGX Sparks for the same memory would cost $32,000 - this cluster costs a fraction and gets faster with every update 10x more productive at half the price - and the only thing you need is knowing where to look$

THIS CLUSTER OF 4 MAC STUDIOS USES LESS POWER THAN ONE NVIDIA CHIP - AND COSTS 5X LESS 17:30 the number hits hard - 600 watts for all four Mac Studios - less than one H200 that costs $30,000 on its own instead of one $30,000 GPU you get 4 machines with 1.5TB of memory, 3x more performance and models that simply cannot run on a single chip Apple quietly added RDMA over Thunderbolt in a beta update - latency dropped 99% and the same cluster became 3x faster without buying any new hardware before the update: 5 tokens per second. after: 15.5 - a free upgrade through a software update 8 Nvidia DGX Sparks for the same memory would cost $32,000 - this cluster costs a fraction and gets faster with every update 10x more productive at half the price - and the only thing you need is knowing where to look

Noisy

106,054 Aufrufe • vor 4 Monaten

QVAC SDK 0.15.0 is live. This release adds multiple prompts batching, brings a native AMD GPU backend to the stack, moves more vision encoders onto mobile GPUs, and adds a second local coding-agent integration. Main highlights: - Prompt batching for the LLM addon. Batch multiple prompts into one job and process them concurrently, with each answer returned the moment its generation finishes. - Native AMD GPU backend. A first-class HIP/ROCm backend in @qvac/vla-ggml, auto-selected over Vulkan with clean fallback when ROCm is absent. - A second local coding agent. OpenClaw joins OpenCode for local, cloud-free agent workflows. AGENTS - OpenCode plugin update (@qvac/opencode-plugin). Aligned with the current SDK, CLI, and AI SDK provider packages. A fresh install runs OpenCode against managed local QVAC models out of the box, from the default qvac/qwen3.5-9b, with no manual qvac serve setup. - OpenClaw plugin (@qvac/openclaw-plugin). A second coding-agent integration alongside OpenCode. A fresh setup installs the plugin, creates a local qvac provider through onboarding, and runs a QVAC model through OpenClaw🦞's local service path. LANGUAGE MODELS - Prompt batching (LLM addon). Batch multiple prompts in one job and run them concurrently, each answer returns the moment its generation finishes, no waiting on the others. - Reasoning-context trimming on hybrid + recurrent models (@qvac/llm-llamacpp). remove_thinking_from_context now works beyond pure-attention models. Same JS API, no throw. VOICE AND SPEECH - Transcription (transcription-parakeet 0.9.0). More robust CPU fallback on GPU failure and a faster Vulkan backend on Pixel 9. - Text-to-speech features (tts-ggml 0.4.0). Adds LavaSR for noise removal and adjustable output frequency up to 48 kHz, plus Japanese via Chatterbox. - Text-to-speech fixes (tts-ggml 0.4.1). CPU fallback on GPU failure, a q8_0 KV crash fix on Metal with Chatterbox. VISION - Qwen3.5 vision encoder on GPU (Android). Image encoder moves onto the phone GPU, with a smarter tile-grid preprocessor and default image-token caps, for flagship Android: Vulkan on Mali (Pixel 9 Pro) and OpenCL on Adreno 830 (Galaxy S25). - Gemma-4 vision encoder on GPU (Android). Vision encoder runs on the phone GPU instead of CPU, same flagship Android targets. PLATFORM AND PERFORMANCE - AMD GPU backend (@qvac/vla-ggml). Native HIP/ROCm backend, auto-selected over Vulkan with clean fallback when ROCm is absent (Linux x64 only). Comes with ~23% faster than Vulkan, ~14% faster than PyTorch-ROCm, parity preserved. Unified code style. A cleaner, more consistent, easier-to-contribute codebase. Let's build. npm install @qvac/sdk

QVAC SDK 0.15.0 is live. This release adds multiple prompts batching, brings a native AMD GPU backend to the stack, moves more vision encoders onto mobile GPUs, and adds a second local coding-agent integration. Main highlights: - Prompt batching for the LLM addon. Batch multiple prompts into one job and process them concurrently, with each answer returned the moment its generation finishes. - Native AMD GPU backend. A first-class HIP/ROCm backend in @qvac/vla-ggml, auto-selected over Vulkan with clean fallback when ROCm is absent. - A second local coding agent. OpenClaw joins OpenCode for local, cloud-free agent workflows. AGENTS - OpenCode plugin update (@qvac/opencode-plugin). Aligned with the current SDK, CLI, and AI SDK provider packages. A fresh install runs OpenCode against managed local QVAC models out of the box, from the default qvac/qwen3.5-9b, with no manual qvac serve setup. - OpenClaw plugin (@qvac/openclaw-plugin). A second coding-agent integration alongside OpenCode. A fresh setup installs the plugin, creates a local qvac provider through onboarding, and runs a QVAC model through OpenClaw🦞's local service path. LANGUAGE MODELS - Prompt batching (LLM addon). Batch multiple prompts in one job and run them concurrently, each answer returns the moment its generation finishes, no waiting on the others. - Reasoning-context trimming on hybrid + recurrent models (@qvac/llm-llamacpp). remove_thinking_from_context now works beyond pure-attention models. Same JS API, no throw. VOICE AND SPEECH - Transcription (transcription-parakeet 0.9.0). More robust CPU fallback on GPU failure and a faster Vulkan backend on Pixel 9. - Text-to-speech features (tts-ggml 0.4.0). Adds LavaSR for noise removal and adjustable output frequency up to 48 kHz, plus Japanese via Chatterbox. - Text-to-speech fixes (tts-ggml 0.4.1). CPU fallback on GPU failure, a q8_0 KV crash fix on Metal with Chatterbox. VISION - Qwen3.5 vision encoder on GPU (Android). Image encoder moves onto the phone GPU, with a smarter tile-grid preprocessor and default image-token caps, for flagship Android: Vulkan on Mali (Pixel 9 Pro) and OpenCL on Adreno 830 (Galaxy S25). - Gemma-4 vision encoder on GPU (Android). Vision encoder runs on the phone GPU instead of CPU, same flagship Android targets. PLATFORM AND PERFORMANCE - AMD GPU backend (@qvac/vla-ggml). Native HIP/ROCm backend, auto-selected over Vulkan with clean fallback when ROCm is absent (Linux x64 only). Comes with ~23% faster than Vulkan, ~14% faster than PyTorch-ROCm, parity preserved. Unified code style. A cleaner, more consistent, easier-to-contribute codebase. Let's build. npm install @qvac/sdk

QVAC

29,260,217 Aufrufe • vor 3 Monaten

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.

UC Berkeley just open-sourced FreeToken. (2–4x faster local LLM inference than Ollama) the results are wild: - Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s - DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s - GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU. let me explain how: all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token. Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time. so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU. every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently. so everything comes down to what happens when the router picks an expert that is not on the GPU. there are two ways to serve that miss: 1. copy it over PCIe and run it on the GPU 2. run it on the CPU, where it already lives both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads. but routing changes on every token, so a fixed choice misses most of what the model asks for. FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation. two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU. none of that is readable off a spec sheet, so the engine profiles it once per machine. the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill. FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946. it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly. releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster. meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use. paper: repo: almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that. the article is quoted below.

Akshay 🚀

345,297 Aufrufe • vor 2 Monaten

🎙️ Want a full breakdown on why cost per token is a key metric for AI infrastructure TCO? We’re operating in a power-constrained world. Demand for intelligence is scaling faster than infrastructure can keep up—so metrics like cost per GPU hour or FLOPs per dollar don’t tell the full story. When it comes to TCO, Cost per Token is the metric that captures end-to-end performance across GPUs, CPUs, networking, software, and ecosystem making it the key driver of real profitability and scalability in AI. If you’re not measuring efficiency at the token level, you’re missing what actually drives cost. Watch the deep-dive conversation with Dr. Gerro Prinsloo , Nader Khalil and Carter Abdallah, live on the NVIDIA YouTube channel.

🎙️ Want a full breakdown on why cost per token is a key metric for AI infrastructure TCO? We’re operating in a power-constrained world. Demand for intelligence is scaling faster than infrastructure can keep up—so metrics like cost per GPU hour or FLOPs per dollar don’t tell the full story. When it comes to TCO, Cost per Token is the metric that captures end-to-end performance across GPUs, CPUs, networking, software, and ecosystem making it the key driver of real profitability and scalability in AI. If you’re not measuring efficiency at the token level, you’re missing what actually drives cost. Watch the deep-dive conversation with Dr. Gerro Prinsloo , Nader Khalil and Carter Abdallah, live on the NVIDIA YouTube channel.

NVIDIA

21,970 Aufrufe • vor 6 Monaten

Today, we’re excited to announce Deep Extract. We’re pushing the boundary of complex document extraction by utilizing an agent harness approach, iterating and verifying outputs until they are at human-level accuracy. The results speak for themselves: we’ve handled documents exceeding 2,500 pages, extracting a total of 30M+ fields in the last couple weeks with results in our production beta reaching 99-100% accuracy. It’s particularly effective on complex documents with long lists, such as invoice line items, brokerage statement transactions, equipment manifests, and more. See more on how we built it, how to use it, and more👇

Today, we’re excited to announce Deep Extract. We’re pushing the boundary of complex document extraction by utilizing an agent harness approach, iterating and verifying outputs until they are at human-level accuracy. The results speak for themselves: we’ve handled documents exceeding 2,500 pages, extracting a total of 30M+ fields in the last couple weeks with results in our production beta reaching 99-100% accuracy. It’s particularly effective on complex documents with long lists, such as invoice line items, brokerage statement transactions, equipment manifests, and more. See more on how we built it, how to use it, and more👇

Reducto

23,949,618 Aufrufe • vor 6 Monaten

Vivaldi didn’t invent heavy metal. But Summer Presto explains why metal feels inevitable. Most people split the rooms: Baroque in a concert hall, metal in an arena. Watch Latvian guitarist Laura Lāce tear through Presto on an Ibanez and that split looks fake. Not because of distortion. Because of the architecture. Vivaldi wrote the movement as weather. Heat. Wind. Hail. The violin line doesn’t “go fast for show.” It has to move fast enough to feel like a storm. Electric guitar just swaps the tool. Sweep picking and alternate picking are the modern translation of those stacked arpeggios. Different hands. Same physics: velocity as drama. Second point Americans miss when they treat classical as background wine music. Metal isn’t heavy because the amp is loud. It’s heavy because of minor-key pressure and hard cuts. Presto lives in that world. Sudden harmonic turns. No polite transition. That’s the same chest-tightening move a good riff uses. Call it shared DNA if you want. Don’t call Vivaldi the godfather. The real bridge is neoclassical metal: Baroque virtuosity run through a stack. Malmsteen made the genre. Covers like this just prove the source material was already violent. Third: the charge is in the original line. Distortion is a switch, not a generator. Mute the dirt and Presto still runs. Flip it on and it face-melts. So this isn’t “classical going slumming with rock.” It’s 18th-century program music doing the thing American metal later industrialized: turn weather into aggression. Classical isn’t dry. It was waiting on the right instrument. Do you hear a storm or a shred track?

Vivaldi didn’t invent heavy metal. But Summer Presto explains why metal feels inevitable. Most people split the rooms: Baroque in a concert hall, metal in an arena. Watch Latvian guitarist Laura Lāce tear through Presto on an Ibanez and that split looks fake. Not because of distortion. Because of the architecture. Vivaldi wrote the movement as weather. Heat. Wind. Hail. The violin line doesn’t “go fast for show.” It has to move fast enough to feel like a storm. Electric guitar just swaps the tool. Sweep picking and alternate picking are the modern translation of those stacked arpeggios. Different hands. Same physics: velocity as drama. Second point Americans miss when they treat classical as background wine music. Metal isn’t heavy because the amp is loud. It’s heavy because of minor-key pressure and hard cuts. Presto lives in that world. Sudden harmonic turns. No polite transition. That’s the same chest-tightening move a good riff uses. Call it shared DNA if you want. Don’t call Vivaldi the godfather. The real bridge is neoclassical metal: Baroque virtuosity run through a stack. Malmsteen made the genre. Covers like this just prove the source material was already violent. Third: the charge is in the original line. Distortion is a switch, not a generator. Mute the dirt and Presto still runs. Flip it on and it face-melts. So this isn’t “classical going slumming with rock.” It’s 18th-century program music doing the thing American metal later industrialized: turn weather into aggression. Classical isn’t dry. It was waiting on the right instrument. Do you hear a storm or a shred track?

𝗧𝗶𝗺𝗲𝗹𝗲𝘀𝘀 𝗠𝘂𝘀𝗶𝗰 ✨🎵

150,353 Aufrufe • vor 1 Monat