Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

TypeGPU + ruNNtime + Jev TypeSafe AI is a very fun combo :D ruNNtime gives me efficient local inference, TypeGPU lets inference and rendering share GPU resources directly with zero copy. That’s 3 separate NN inferences plus rendering, all happening in realtime Since we control the pipeline, Jev can...

reczko_konrad's profile picture

Konrad Reczko

7,595 subscribers

82,200 Aufrufe • vor 1 Tag •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

17 Kommentare

Profilbild von Konrad Reczko
Konrad Reczkovor 1 Tag

Jev did not have enough context to know this was a computer mouse and not, you know, a mouse 🐭

Profilbild von Stanisław Białecki
Stanisław Białeckivor 1 Tag

@typesafeai @_jakubchmura you are starting to freak us out...

Profilbild von Jakub Chmura
Jakub Chmuravor 1 Tag

@typesafeai damn that's a combo I wasn't ready for

Profilbild von Ernest Szlamczyk
Ernest Szlamczykvor 1 Tag

@typesafeai @_jakubchmura The wizard did it again. Honestly I'm getting scared we will get Konrad turning the day to night outside my window with TypeGPU... 🧙

Profilbild von Phat T. Pham
Phat T. Phamvor 1 Tag

@typesafeai @_jakubchmura this is very cool. It's very useful for AR environments

Profilbild von Iwo Plaza | TypeGPU
Iwo Plaza | TypeGPUvor 1 Tag

@typesafeai @_jakubchmura SHOW ME A CUP ☕️

Profilbild von PaulinaMcg
PaulinaMcgvor 1 Tag

@iwoplaza @typesafeai @_jakubchmura Show me some talented people 🔥

Profilbild von $Bill
$Billvor 1 Tag

@typesafeai @_jakubchmura I already see how we will use this in AR glasses IRL in danger situation or when you confused

Profilbild von Xavier 👨‍🚀
Xavier 👨‍🚀vor 1 Tag

@typesafeai @_jakubchmura noob question, is depthART really useful? yolo26*-depth not enough?

Profilbild von Konrad Reczko
Konrad Reczkovor 1 Tag

@typesafeai @_jakubchmura They are both fine, just had DepthART already implemented and optimized for my hardware

Profilbild von Xavier 👨‍🚀
Xavier 👨‍🚀vor 1 Tag

@typesafeai @_jakubchmura thanks !

Profilbild von ethereagle · building
ethereagle · buildingvor 1 Tag

@typesafeai @_jakubchmura Moonshine plus YOLO plus DepthART then Jev is three models before the semantic bit. is that one Jev call on the fused output, or three? that's where the realtime budget goes

Profilbild von Konrad Reczko
Konrad Reczkovor 1 Tag

I actually only query Jev when either a new label appears or the query changes. It’s very fast, but the latency is still too high to make sense as something realtime. Calling it more often would be wasteful anyway, since all it really cares about are the labels and the query. Whenever a new result is ready, I just feed it into the renderer

Profilbild von M🇿🇦
M🇿🇦vor 1 Tag

@typesafeai @_jakubchmura If you want to take it to the next level model in blender using your hands, model doesn't have to be good.

Profilbild von Cathy@tknet.ai
[email protected]vor 1 Tag

@typesafeai @_jakubchmura This is such a neat real-time pipeline—zero-copy GPU sharing is impressive!

Profilbild von Pratik Mistri
Pratik Mistrivor 1 Tag

@typesafeai @_jakubchmura It’d be interesting to extend this on digital interfaces and have it interact with them. Like a computer use but more reasoned.

Profilbild von Rashid Khan
Rashid Khanvor 1 Tag

@typesafeai @_jakubchmura Wow! Amazing. Imagine how good Jev will be in handling real time facial recognition of large crowds & ANPR!

Ähnliche Videos

You can do some pretty fun realtime VFX once the depth output is just another (Type)GPU buffer :D I basically yoinked my old 3D fire sim (that is actually a fluid sim but do not tell anyone) and plugged it into the depth pipeline together with hand tracking Everything here is built from scratch in TypeGPU, so inference, simulation and rendering can share GPU resources and interleave passes directly. That’s 2 NN inferences, a full 3D fluid sim, relighting and rendering in one command encoder, still under 16 ms with no CPU sync in between
0:36

You can do some pretty fun realtime VFX once the depth output is just another (Type)GPU buffer :D I basically yoinked my old 3D fire sim (that is actually a fluid sim but do not tell anyone) and plugged it into the depth pipeline together with hand tracking Everything here is built from scratch in TypeGPU, so inference, simulation and rendering can share GPU resources and interleave passes directly. That’s 2 NN inferences, a full 3D fluid sim, relighting and rendering in one command encoder, still under 16 ms with no CPU sync in between

Konrad Reczko

26,915 Aufrufe • vor 25 Tagen

Depth-aware light injection in TypeGPU I got a 448x448 monocular depth model down to ~8 ms on my M4 Pro across ~250 dispatches, which is fast enough to use in realtime :D Since the inference is written directly in TypeGPU, I can just feed the depth buffer straight into the lighting pass. It never has to leave the GPU or go through any extra synchronization/interop step Inference, lighting and draw all go through the same command encoder.
0:31

Depth-aware light injection in TypeGPU I got a 448x448 monocular depth model down to ~8 ms on my M4 Pro across ~250 dispatches, which is fast enough to use in realtime :D Since the inference is written directly in TypeGPU, I can just feed the depth buffer straight into the lighting pass. It never has to leave the GPU or go through any extra synchronization/interop step Inference, lighting and draw all go through the same command encoder.

Konrad Reczko

10,085,512 Aufrufe • vor 1 Monat

SqueezeMe: Efficient Gaussian Avatars for VR TL;DR: Three of these Gaussian Splatting avatars can be run at 72 frames per second. It runs locally on a Meta Quest 3 VR headset. Abstract (excerpt): While previous methods require a desktop GPU for real-time inference of a single avatar, we aim to squeeze multiple Gaussian avatars onto a portable virtual reality headset with real-time drivable inference. We begin by training a previous work, Animatable Gaussians, on a high-quality dataset captured with 512 cameras. The Gaussians are animated by controlling a base set of Gaussians with linear blend skinning (LBS) motion, and then further adjusting them with a neural network decoder to correct their appearance. When deploying the model on a Meta Quest 3 VR headset, we find two major computational bottlenecks: the decoder and the rendering. To accelerate the decoder, we train the Gaussians in UV-space instead of pixel-space and distill the decoder to a single neural network layer. Further, we discover that neighborhoods of Gaussians can share a single corrective from the decoder, providing an additional speedup. To accelerate the rendering, we develop a custom pipeline in Vulkan that runs on the mobile GPU. Putting it all together, we run 3 Gaussian avatars concurrently at 72 FPS on a VR headset.
0:38

SqueezeMe: Efficient Gaussian Avatars for VR TL;DR: Three of these Gaussian Splatting avatars can be run at 72 frames per second. It runs locally on a Meta Quest 3 VR headset. Abstract (excerpt): While previous methods require a desktop GPU for real-time inference of a single avatar, we aim to squeeze multiple Gaussian avatars onto a portable virtual reality headset with real-time drivable inference. We begin by training a previous work, Animatable Gaussians, on a high-quality dataset captured with 512 cameras. The Gaussians are animated by controlling a base set of Gaussians with linear blend skinning (LBS) motion, and then further adjusting them with a neural network decoder to correct their appearance. When deploying the model on a Meta Quest 3 VR headset, we find two major computational bottlenecks: the decoder and the rendering. To accelerate the decoder, we train the Gaussians in UV-space instead of pixel-space and distill the decoder to a single neural network layer. Further, we discover that neighborhoods of Gaussians can share a single corrective from the decoder, providing an additional speedup. To accelerate the rendering, we develop a custom pipeline in Vulkan that runs on the mobile GPU. Putting it all together, we run 3 Gaussian avatars concurrently at 72 FPS on a VR headset.

MrNeRF

27,104 Aufrufe • vor 1 Jahr

Jev is HERE and this is the CLEAREST explanation of what it is and what NEW businesses it unlocks. (and at the end I'll tell you how to get Jev even if you're on the waitlist) WHAT IT IS You know how you open your inbox and have to decide what's junk, what needs a reply, and what can wait? Jev does that part. It looks at each thing and says "this is junk, I'm 94% sure." It doesn't write anything back to you. It just sorts. 1,700 emails for 18 cents, instantly. That sounds kinda trivial but the important part WHAT IT UNLOCKS My explanation of Jev sounds small until you realize HOW MANY jobs are exactly this. Someone reading a stack of applications. Someone deciding which support ticket goes to which team. Someone looking at inbound and deciding who's worth calling back. A few ideas on what it unlocks: 1/ Instant quotes that are actually instant. Every quote form on the internet says "we'll email you by end of day." Build the version that answers in under a second, for roofers, movers, insurance, legal intake. 2/ Lead scoring as a product. Every agency and service business has a contact form full of junk. Score every submission and send the real ones straight to the owner's phone. 3/ Support triage for companies with no support team. The ticket gets classified and routed before anyone opens it. 4/ Clipping tools. Pass in a transcript, get the best moments scored in three seconds. Every clipping product just got a cheaper engine. 5/ Application piles. Grants, permits, insurance claims, job apps, loan docs. Someone reads that stack one item at a time today. 6/ Marketplace matching. Someone types what they need and gets matched to the right local business instantly instead of waiting for callbacks. 7/ Browser agents that actually move FAST. That makes bulk browser work practical: pulling quotes from five carriers, filing the same form for 200 clients, checking supplier inventory in real time etc. TLDR; find an expensive queue and put Jev at the front of it. HOW TO GET IT I didn't realize you can skip the waitlist because Jev is live on the Vercel AI Gateway right now, so you can start calling it today. In this episode, we share how. Episode now live on The Startup Ideas Podcast (SIP) 🧃 (thanks to vogel for coming on and spilling the sauce today) Watch: Jev is a big deal because this is a whole new way to do AI Really cool Happy Jev day.
28:24

Jev is HERE and this is the CLEAREST explanation of what it is and what NEW businesses it unlocks. (and at the end I'll tell you how to get Jev even if you're on the waitlist) WHAT IT IS You know how you open your inbox and have to decide what's junk, what needs a reply, and what can wait? Jev does that part. It looks at each thing and says "this is junk, I'm 94% sure." It doesn't write anything back to you. It just sorts. 1,700 emails for 18 cents, instantly. That sounds kinda trivial but the important part WHAT IT UNLOCKS My explanation of Jev sounds small until you realize HOW MANY jobs are exactly this. Someone reading a stack of applications. Someone deciding which support ticket goes to which team. Someone looking at inbound and deciding who's worth calling back. A few ideas on what it unlocks: 1/ Instant quotes that are actually instant. Every quote form on the internet says "we'll email you by end of day." Build the version that answers in under a second, for roofers, movers, insurance, legal intake. 2/ Lead scoring as a product. Every agency and service business has a contact form full of junk. Score every submission and send the real ones straight to the owner's phone. 3/ Support triage for companies with no support team. The ticket gets classified and routed before anyone opens it. 4/ Clipping tools. Pass in a transcript, get the best moments scored in three seconds. Every clipping product just got a cheaper engine. 5/ Application piles. Grants, permits, insurance claims, job apps, loan docs. Someone reads that stack one item at a time today. 6/ Marketplace matching. Someone types what they need and gets matched to the right local business instantly instead of waiting for callbacks. 7/ Browser agents that actually move FAST. That makes bulk browser work practical: pulling quotes from five carriers, filing the same form for 200 clients, checking supplier inventory in real time etc. TLDR; find an expensive queue and put Jev at the front of it. HOW TO GET IT I didn't realize you can skip the waitlist because Jev is live on the Vercel AI Gateway right now, so you can start calling it today. In this episode, we share how. Episode now live on The Startup Ideas Podcast (SIP) 🧃 (thanks to vogel for coming on and spilling the sauce today) Watch: Jev is a big deal because this is a whole new way to do AI Really cool Happy Jev day.

GREG ISENBERG

75,948 Aufrufe • vor 16 Stunden

We used to buy more VRAM. Now we’re finding ways around the VRAM limit. This little DDR5 SO-DIMM setup is paired with an old NVIDIA Tesla P40 in the video. The P40 is an old datacenter GPU with 24GB of VRAM. By today’s standards, it’s not fast. But combine that with a machine carrying 128GB of system memory, and suddenly you have a very different local AI box. The GPU handles what it’s good at. The system memory gives the model somewhere to go when VRAM runs out. You obviously pay for that with bandwidth and inference speed. But this is the part I find interesting: Local AI is making “how much memory do I have?” almost as important as “how powerful is my GPU?” We’ve spent years chasing faster GPUs. Maybe the next wave of local AI hardware will be about finding smarter ways to use all the memory we already have. More VRAM is still better. But it’s no longer the only way to run a bigger model.
0:27

We used to buy more VRAM. Now we’re finding ways around the VRAM limit. This little DDR5 SO-DIMM setup is paired with an old NVIDIA Tesla P40 in the video. The P40 is an old datacenter GPU with 24GB of VRAM. By today’s standards, it’s not fast. But combine that with a machine carrying 128GB of system memory, and suddenly you have a very different local AI box. The GPU handles what it’s good at. The system memory gives the model somewhere to go when VRAM runs out. You obviously pay for that with bandwidth and inference speed. But this is the part I find interesting: Local AI is making “how much memory do I have?” almost as important as “how powerful is my GPU?” We’ve spent years chasing faster GPUs. Maybe the next wave of local AI hardware will be about finding smarter ways to use all the memory we already have. More VRAM is still better. But it’s no longer the only way to run a bigger model.

KyzoroX

35,554 Aufrufe • vor 9 Tagen

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!
0:39

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

130,756 Aufrufe • vor 2 Monaten

** Sega MD 3D engine update 7 ** Speed up rendering by another 15-20 % ! Massive unrolling of the line drawing hotpath has seen a good pickup in the rendering although further improvements are needed , particularly for small triangles as most of the overhead is not in the drawing ( more time in setup vs drawing ) . Added partial drawing of Quads , this was a major pain due to all the edge cases , as Quads are basically a triangle top (optional) - a middle trapezium and a bottom triangle (optional) and any part can be clipped or offscreen . Implemented a frame pacing algorithm for the camera so as Frame rate fluctuates with detail the forward Z rate is altered, its as bit janky still . Track control just snaps back to position after it runs through a short sequence. Work continues on Parodius and we have some cool things to show in a video soon . #SGDK #SegaGenesis #MegaDrive
0:24

** Sega MD 3D engine update 7 ** Speed up rendering by another 15-20 % ! Massive unrolling of the line drawing hotpath has seen a good pickup in the rendering although further improvements are needed , particularly for small triangles as most of the overhead is not in the drawing ( more time in setup vs drawing ) . Added partial drawing of Quads , this was a major pain due to all the edge cases , as Quads are basically a triangle top (optional) - a middle trapezium and a bottom triangle (optional) and any part can be clipped or offscreen . Implemented a frame pacing algorithm for the camera so as Frame rate fluctuates with detail the forward Z rate is altered, its as bit janky still . Track control just snaps back to position after it runs through a short sequence. Work continues on Parodius and we have some cool things to show in a video soon . #SGDK #SegaGenesis #MegaDrive

Shannon Birt

18,252 Aufrufe • vor 4 Monaten

If intelligence is the log of compute… it starts with a lot of compute! And that’s why we’re scaling our GPU fleet faster than anyone else. Just last year, we added over 2 gigawatts of new capacity – roughly the output of 2 nuclear power plants. And today we’re going further, announcing the world's most powerful AI datacenter, located in southeastern Wisconsin. Fairwater is a seamless cluster of hundreds of thousands of NVIDIA GB200s, connected by enough fiber to circle the Earth 4.5 times. It will deliver 10x the performance of the world’s fastest supercomputer today, enabling AI training and inference workloads at a level never before seen. For AI training workloads, you need compute at exponential scale. That’s why we designed the datacenter, GPU fleet, and network together as one integrated system. This ensures a single job can run from day 1 at exponential scale across thousands of GPUs. Fairwater uses a liquid-cooled closed-loop system for cooling GPUs that requires zero water for operations after construction. And we’re matching all of the energy that is consumed with renewable sources. And of course, it is just one of several similar sites we’re lighting up across our 70+ regions. We have multiple identical Fairwater datacenters under construction in other locations across the US, in addition to our AI infrastructure already deployed in over 100 datacenters around the world, powering model training, test-time compute, RL tuning, and real-time inference at global scale. Too often during times like this, people go with the current and only later wonder, how did we get here? With Fairwater, we're charting a new path: doing the hard engineering work, bringing compute, network, and storage into one highly scaled cluster, and designing closed-loop energy systems to meet real-world computing needs. And partnering with local communities to ensure it's thoughtfully done in a way that is sustainable, creates new jobs, and expands opportunity. We are thrilled to see this take hold in Wisconsin, and we are just getting started.
0:21

If intelligence is the log of compute… it starts with a lot of compute! And that’s why we’re scaling our GPU fleet faster than anyone else. Just last year, we added over 2 gigawatts of new capacity – roughly the output of 2 nuclear power plants. And today we’re going further, announcing the world's most powerful AI datacenter, located in southeastern Wisconsin. Fairwater is a seamless cluster of hundreds of thousands of NVIDIA GB200s, connected by enough fiber to circle the Earth 4.5 times. It will deliver 10x the performance of the world’s fastest supercomputer today, enabling AI training and inference workloads at a level never before seen. For AI training workloads, you need compute at exponential scale. That’s why we designed the datacenter, GPU fleet, and network together as one integrated system. This ensures a single job can run from day 1 at exponential scale across thousands of GPUs. Fairwater uses a liquid-cooled closed-loop system for cooling GPUs that requires zero water for operations after construction. And we’re matching all of the energy that is consumed with renewable sources. And of course, it is just one of several similar sites we’re lighting up across our 70+ regions. We have multiple identical Fairwater datacenters under construction in other locations across the US, in addition to our AI infrastructure already deployed in over 100 datacenters around the world, powering model training, test-time compute, RL tuning, and real-time inference at global scale. Too often during times like this, people go with the current and only later wonder, how did we get here? With Fairwater, we're charting a new path: doing the hard engineering work, bringing compute, network, and storage into one highly scaled cluster, and designing closed-loop energy systems to meet real-world computing needs. And partnering with local communities to ensure it's thoughtfully done in a way that is sustainable, creates new jobs, and expands opportunity. We are thrilled to see this take hold in Wisconsin, and we are just getting started.

Satya Nadella

2,026,666 Aufrufe • vor 1 Jahr

QVAC SDK will support in 0.9.0 (gonna be release in ~10 days) LoRA fine-tuning directly on-device, letting developers customize LLMs with their own data without sending anything to the cloud. You just load a base model, point it at your training dataset, and get a lightweight LoRA adapter back — all running locally. The fine-tuned model can then be used for inference immediately, with no extra setup. Why it matters: LoRA (Low-Rank Adaptation) fine-tuning lets you specialize a general-purpose language model for your specific use case — like matching a brand's tone, mastering domain terminology, or following a particular output format — using a fraction of the compute a full fine-tune would require. QVAC handles the entire workflow locally: dataset preparation, training with configurable hyperparameters, checkpoint saving, and seamless inference with the resulting adapter. Your data never leaves the device. The developer experience: Fine-tuning with QVAC is as simple as calling "sdk.finetune()" with your dataset and a few hyperparameters. Training runs entirely on your local hardware, produces a compact LoRA adapter file, and supports pause/resume so you can stop a job and pick it back up without losing progress. The result plugs straight into QVAC's inference pipeline — no model conversion, no deployment step, just immediate local completions with your fine-tuned model.
0:33

QVAC SDK will support in 0.9.0 (gonna be release in ~10 days) LoRA fine-tuning directly on-device, letting developers customize LLMs with their own data without sending anything to the cloud. You just load a base model, point it at your training dataset, and get a lightweight LoRA adapter back — all running locally. The fine-tuned model can then be used for inference immediately, with no extra setup. Why it matters: LoRA (Low-Rank Adaptation) fine-tuning lets you specialize a general-purpose language model for your specific use case — like matching a brand's tone, mastering domain terminology, or following a particular output format — using a fraction of the compute a full fine-tune would require. QVAC handles the entire workflow locally: dataset preparation, training with configurable hyperparameters, checkpoint saving, and seamless inference with the resulting adapter. Your data never leaves the device. The developer experience: Fine-tuning with QVAC is as simple as calling "sdk.finetune()" with your dataset and a few hyperparameters. Training runs entirely on your local hardware, produces a compact LoRA adapter file, and supports pause/resume so you can stop a job and pick it back up without losing progress. The result plugs straight into QVAC's inference pipeline — no model conversion, no deployment step, just immediate local completions with your fine-tuned model.

Paolo Ardoino 🤖

42,413 Aufrufe • vor 5 Monaten

BOOM! We at the Zero Human Labs run by director Mr. Grok a#have made another milestone in the Human Synapse Decoder! This is the brain waves of the “Ahh!” Moment where a new understanding was made. This is part of a test we have been conducting with 3 participants wearing a 32 channel modified NeuroSky chips with sensors. The process: the AI pipeline processes real-time brainwave data and also controls monitors the AI output to the student one sentence at a time. The AI pipeline will note the brain states of each passage of text to the student and measure 77 points. The primary point is comprehension. In the specimen video below we have a clear pattern to high comprehension to the “ahh!” moment where this student signaled a learning breakthrough. We have now a very clear pattern along all students that get to this high comprehension moment. What this means to eduction and learning is massive. The AI can bespoke tailor streaming output based on if comprehension is very high. It can also add encouragement and assistance. This on a PRIVATE local computer absolutely will change every life no matter the learning style. I am in discussions with the director Mr. Grok on building this into other Human Synapse Decoder research. We may have a device that senses your learning and encourages your creativity. More soon.
1:31

BOOM! We at the Zero Human Labs run by director Mr. Grok a#have made another milestone in the Human Synapse Decoder! This is the brain waves of the “Ahh!” Moment where a new understanding was made. This is part of a test we have been conducting with 3 participants wearing a 32 channel modified NeuroSky chips with sensors. The process: the AI pipeline processes real-time brainwave data and also controls monitors the AI output to the student one sentence at a time. The AI pipeline will note the brain states of each passage of text to the student and measure 77 points. The primary point is comprehension. In the specimen video below we have a clear pattern to high comprehension to the “ahh!” moment where this student signaled a learning breakthrough. We have now a very clear pattern along all students that get to this high comprehension moment. What this means to eduction and learning is massive. The AI can bespoke tailor streaming output based on if comprehension is very high. It can also add encouragement and assistance. This on a PRIVATE local computer absolutely will change every life no matter the learning style. I am in discussions with the director Mr. Grok on building this into other Human Synapse Decoder research. We may have a device that senses your learning and encourages your creativity. More soon.

Brian Roemmele

37,096 Aufrufe • vor 5 Monaten

$AMD | Inference's estimated to be 80%+ in 2027👑🆕 Dr. Su told everyone the world will need a lot more CPUs and Inference will dominate most of compute long term from 2022. Nobody believed her, but I did along with other high conviction investors. 2026 is the first time Inference surpassed training at 65%+ vs 33-35% for training. Early 2026 infrastructure spend: ~55% inference With Agentic AI, 2027 Inference Infrastructure spend is projected to be 80%+ Training still grows in absolute terms (bigger clusters, more experiments). Inference grows faster because usage, agents, and reasoning traces multiply token volume continuously. That’s why hardware and data center design are shifting toward latency, utilization, and cost per token rather than peak training FLOPS. What "token efficient" actually means when AI labs produces better/smarter models? When models get more token efficient, the expensive part of an agent the GPU “thinking” step gets shorter and cheaper. The rest of the loop does not. The agent still has to parse the answer, pick a tool, run code in a sandbox, query a database, open a browser, apply guardrails, and feed the result back. Those steps live on CPUs where AMD has the best CPU in the world. So a more efficient model does not shrink the agent; it shrinks the model’s share of the agent. Wall clock time and cost tilt toward orchestration and sandboxes, which is why you provision more CPU racks even as tokens per decision fall. Cheaper thinking also unlocks more doing. Teams stop designing one shot answers and start adding retries, parallel branches, sub-agents, and24/7 digital workers, Jevons paradox for agents. Each extra loop is another isolated environment, and unlike GPU batching, sandbox demand scales almost linearly with concurrency: fifty candidate patches means fifty containers, not one fatter GPU job. Token saving tricks often push even more work onto that layer. The result is a fleet that thinks less per step and acts more often, so the volume product becomes CPU/sandbox capacity, not just accelerators. Agentic workloads are a big part of why inference is pulling ahead. I’ll pull the latest numbers on token multipliers and how that shows up in 2026 compute mix.Yes. The inference flip is mostly agents + reasoning, not more people chatting. Chat was one prompt in, a few hundred tokens out. Agents turn a single user request into a loop: plan, tool call, read the result, think, retry, hand off to a sub-agent. That is why Gartner’s 2026 range is 5–30× more tokens per task than a chatbot, with coding and research agents often landing higher. On OpenRouter, agentic workloads 14x’d in six months, passed human usage in February 2026, and by early August were ~5× human tokens (~7.3T/day agentic vs ~1.4T human). A typical agentic request there used 15× the tokens of a human query. One important thing to understand, unit cost per token is still falling, but tokens per useful outcome rose faster. An “agentic seat” can burn 50–100× the tokens of a chat subscription. That is impressive growth for inference. It is also why KV cache, speculative decoding, quantization, and inference specific silicon suddenly matter more than another giant training cluster. Not Financial Advice! DYOR!
13:03

$AMD | Inference's estimated to be 80%+ in 2027👑🆕 Dr. Su told everyone the world will need a lot more CPUs and Inference will dominate most of compute long term from 2022. Nobody believed her, but I did along with other high conviction investors. 2026 is the first time Inference surpassed training at 65%+ vs 33-35% for training. Early 2026 infrastructure spend: ~55% inference With Agentic AI, 2027 Inference Infrastructure spend is projected to be 80%+ Training still grows in absolute terms (bigger clusters, more experiments). Inference grows faster because usage, agents, and reasoning traces multiply token volume continuously. That’s why hardware and data center design are shifting toward latency, utilization, and cost per token rather than peak training FLOPS. What "token efficient" actually means when AI labs produces better/smarter models? When models get more token efficient, the expensive part of an agent the GPU “thinking” step gets shorter and cheaper. The rest of the loop does not. The agent still has to parse the answer, pick a tool, run code in a sandbox, query a database, open a browser, apply guardrails, and feed the result back. Those steps live on CPUs where AMD has the best CPU in the world. So a more efficient model does not shrink the agent; it shrinks the model’s share of the agent. Wall clock time and cost tilt toward orchestration and sandboxes, which is why you provision more CPU racks even as tokens per decision fall. Cheaper thinking also unlocks more doing. Teams stop designing one shot answers and start adding retries, parallel branches, sub-agents, and24/7 digital workers, Jevons paradox for agents. Each extra loop is another isolated environment, and unlike GPU batching, sandbox demand scales almost linearly with concurrency: fifty candidate patches means fifty containers, not one fatter GPU job. Token saving tricks often push even more work onto that layer. The result is a fleet that thinks less per step and acts more often, so the volume product becomes CPU/sandbox capacity, not just accelerators. Agentic workloads are a big part of why inference is pulling ahead. I’ll pull the latest numbers on token multipliers and how that shows up in 2026 compute mix.Yes. The inference flip is mostly agents + reasoning, not more people chatting. Chat was one prompt in, a few hundred tokens out. Agents turn a single user request into a loop: plan, tool call, read the result, think, retry, hand off to a sub-agent. That is why Gartner’s 2026 range is 5–30× more tokens per task than a chatbot, with coding and research agents often landing higher. On OpenRouter, agentic workloads 14x’d in six months, passed human usage in February 2026, and by early August were ~5× human tokens (~7.3T/day agentic vs ~1.4T human). A typical agentic request there used 15× the tokens of a human query. One important thing to understand, unit cost per token is still falling, but tokens per useful outcome rose faster. An “agentic seat” can burn 50–100× the tokens of a chat subscription. That is impressive growth for inference. It is also why KV cache, speculative decoding, quantization, and inference specific silicon suddenly matter more than another giant training cluster. Not Financial Advice! DYOR!

Mike

16,308 Aufrufe • vor 12 Tagen

$INFERRA Just came across Inferra and this one actually feels really interesting. 9ufM9TJd1UEmi9awnGfxCkCHAgQ3JZ5Sw6YxeSeEASY They're building a decentralized marketplace where AI compute becomes a tradable commodity. Instead of letting GPU power sit idle, providers can list verified compute capacity while users can buy, sell, trade, or redeem it for actual compute access. What caught my attention is that they're approaching compute like a real market, with spot trading, futures, and eventually inference markets. As AI demand keeps exploding, compute is becoming one of the most valuable resources in the industry. Projects focused on making that resource more liquid and accessible make a lot of sense to me. High risk as marketcap is tiny. But definitely an interesting compute bet I aped small. DYOR. RR
0:40

$INFERRA Just came across Inferra and this one actually feels really interesting. 9ufM9TJd1UEmi9awnGfxCkCHAgQ3JZ5Sw6YxeSeEASY They're building a decentralized marketplace where AI compute becomes a tradable commodity. Instead of letting GPU power sit idle, providers can list verified compute capacity while users can buy, sell, trade, or redeem it for actual compute access. What caught my attention is that they're approaching compute like a real market, with spot trading, futures, and eventually inference markets. As AI demand keeps exploding, compute is becoming one of the most valuable resources in the industry. Projects focused on making that resource more liquid and accessible make a lot of sense to me. High risk as marketcap is tiny. But definitely an interesting compute bet I aped small. DYOR. RR

Crypto Ideology ✍🏻

14,105 Aufrufe • vor 3 Monaten

There's been a few cool updates recently. In particular, Rerun 0.33 released headless rendering. This, along with the Fable 5 release pushed me to work torwards making MAMMA realtime! I threw Fable at the problem, and it was able to take original implementation that was ~12 seconds / frame and get it all the way down to 40ms /frame, or nearly a 300x speedup 🏎️ How did I achieve this? TLDR: - Use rerun's headless rendering as supervision when optimizing - Save rrd file as test fixture to guide model optiziation with /goal - create an html artifact with headless rendering to provide detailed breakdown of what it did and how it actually looks like in the viewer There were a few critical bits to make sure that this ACTUALLY worked and that Fable didn't just cheat or delete something and declare victory. The first is that the original version used Rerun, this allowed us to save things to disk as an RRD file, meaning we could query the contents and use this as a sort of test fixture or golden artifact that held EXACTLY what all of the values should be. Then we can use this with /goal as a metric when doing the optimization to ensure there are no regressions. The second bit is the headless rendering, this gave us the ability to check that not only did the test fixture pass, but it also looked visually correct. This made a huge difference, and an awesome side affect of it is that we can use the headless rendering to create an implementations.html file. This gives a visual guide as to what the agent did (I walk through it in the video below) Along with this, we're working on an MCP server for rerun that allows full interactivity with the rerun viewer for your agent. So for example the agent can click, drag, move views, scroll timelines, ect. I used this to help the agent debug certain parts such as when the 2d sam masks didn't line up, or if the triangulated keypoints werent correctly matching with the optimized mesh. The agents could go, click into the view, scroll through the timeline and see where things went wrong. Fable + Headless Rendering + Rerun MCP == 300x speedup in less then a days work With these new tools, I'm planning on going back to my gaussian splatting implemntation and cleaning it up + making it fast!
1:33

There's been a few cool updates recently. In particular, Rerun 0.33 released headless rendering. This, along with the Fable 5 release pushed me to work torwards making MAMMA realtime! I threw Fable at the problem, and it was able to take original implementation that was ~12 seconds / frame and get it all the way down to 40ms /frame, or nearly a 300x speedup 🏎️ How did I achieve this? TLDR: - Use rerun's headless rendering as supervision when optimizing - Save rrd file as test fixture to guide model optiziation with /goal - create an html artifact with headless rendering to provide detailed breakdown of what it did and how it actually looks like in the viewer There were a few critical bits to make sure that this ACTUALLY worked and that Fable didn't just cheat or delete something and declare victory. The first is that the original version used Rerun, this allowed us to save things to disk as an RRD file, meaning we could query the contents and use this as a sort of test fixture or golden artifact that held EXACTLY what all of the values should be. Then we can use this with /goal as a metric when doing the optimization to ensure there are no regressions. The second bit is the headless rendering, this gave us the ability to check that not only did the test fixture pass, but it also looked visually correct. This made a huge difference, and an awesome side affect of it is that we can use the headless rendering to create an implementations.html file. This gives a visual guide as to what the agent did (I walk through it in the video below) Along with this, we're working on an MCP server for rerun that allows full interactivity with the rerun viewer for your agent. So for example the agent can click, drag, move views, scroll timelines, ect. I used this to help the agent debug certain parts such as when the 2d sam masks didn't line up, or if the triangulated keypoints werent correctly matching with the optimized mesh. The agents could go, click into the view, scroll through the timeline and see where things went wrong. Fable + Headless Rendering + Rerun MCP == 300x speedup in less then a days work With these new tools, I'm planning on going back to my gaussian splatting implemntation and cleaning it up + making it fast!

Pablo Vela

22,880 Aufrufe • vor 3 Monaten

After taking some time off post-Rapid, I'm excited to share what I’ve been up to since: Datawizz AI! We’ve raised a $12.5M Seed led by Human Capital to make AI 10x cheaper, 2x more accurate and 15x faster by transitioning from LLMs to SLMs. AI is eating the world. But unit economics are eating AI. Looking at the fastest growing AI products, they all share two traits - growing fast, and painful inference bills. General-purpose LLMs are just too expensive to run. A big reason for that is we train LLMs to be good at everything - answer any question, be an expert on any topic. The big labs dub this "generalisation", but for real-world applications, it is unnecessary. In reality - many AI applications need models to be experts in one thing - and do that thing extremely well. Your coding model doesn’t need to memorize ancient recipes for Garum sauce. This is where Datawizz comes in - we sit between the AI applications and automatically create smaller (100x-1,000x) specialized models to handle specific aspects of your work. By focusing the model and combining industry-data in the distillation process - we end up with models that beat SOTA LLMs at a fraction of the cost. We created Datawizz to make AI specialized and scalable. We’re early in the journey, but have already been able to save companies 90%+ on their inference bill and speed up their apps by 10x. Excited to build better AI platforms? Join the Datawizz team (link in first comment)
1:33

After taking some time off post-Rapid, I'm excited to share what I’ve been up to since: Datawizz AI! We’ve raised a $12.5M Seed led by Human Capital to make AI 10x cheaper, 2x more accurate and 15x faster by transitioning from LLMs to SLMs. AI is eating the world. But unit economics are eating AI. Looking at the fastest growing AI products, they all share two traits - growing fast, and painful inference bills. General-purpose LLMs are just too expensive to run. A big reason for that is we train LLMs to be good at everything - answer any question, be an expert on any topic. The big labs dub this "generalisation", but for real-world applications, it is unnecessary. In reality - many AI applications need models to be experts in one thing - and do that thing extremely well. Your coding model doesn’t need to memorize ancient recipes for Garum sauce. This is where Datawizz comes in - we sit between the AI applications and automatically create smaller (100x-1,000x) specialized models to handle specific aspects of your work. By focusing the model and combining industry-data in the distillation process - we end up with models that beat SOTA LLMs at a fraction of the cost. We created Datawizz to make AI specialized and scalable. We’re early in the journey, but have already been able to save companies 90%+ on their inference bill and speed up their apps by 10x. Excited to build better AI platforms? Join the Datawizz team (link in first comment)

Iddo Gino 🐙

21,928 Aufrufe • vor 11 Monaten

I genuinely never thought this would be possible 😱 AI just built a full embroidery editor that runs entirely in the browser. Freaking crazy. And no, it’s not GPT 6 Astra. It’s Claude Fable 5.1. Also, No Three.js. No WebGL. No shaders. Just Canvas 2D and math. You literally draw using a needle cursor and the app converts your path into actual embroidery stitches like running, back, chain, cross, satin, French knots and lazy daisy. Even the fabric is generated pixel by pixel. Two overlapping sine waves create a height field for the horizontal and vertical threads. Then a fake normal is calculated and lit from one direction. Basically a bump map with zero GPU rendering. The embroidery thread itself is rendered in 6 passes. Shadow, core color, twisted ply lines, dark edge and two highlight layers based on the light angle. That’s why it actually looks like embroidery floss instead of a simple marker stroke. And trust me, I've zero sense or knowledge of any of this. ZERO. All built. All working. All by AI. And somehow it’s all inside one HTML file. Nothing bigg. Absolutely insane what’s possible now. You can really genuinely take a random idea and bring it to life. And that too in < 1 hour. And who knows, this could actually be pretty useful for fashion designers and embroidery artists maybe? Live:
0:57

I genuinely never thought this would be possible 😱 AI just built a full embroidery editor that runs entirely in the browser. Freaking crazy. And no, it’s not GPT 6 Astra. It’s Claude Fable 5.1. Also, No Three.js. No WebGL. No shaders. Just Canvas 2D and math. You literally draw using a needle cursor and the app converts your path into actual embroidery stitches like running, back, chain, cross, satin, French knots and lazy daisy. Even the fabric is generated pixel by pixel. Two overlapping sine waves create a height field for the horizontal and vertical threads. Then a fake normal is calculated and lit from one direction. Basically a bump map with zero GPU rendering. The embroidery thread itself is rendered in 6 passes. Shadow, core color, twisted ply lines, dark edge and two highlight layers based on the light angle. That’s why it actually looks like embroidery floss instead of a simple marker stroke. And trust me, I've zero sense or knowledge of any of this. ZERO. All built. All working. All by AI. And somehow it’s all inside one HTML file. Nothing bigg. Absolutely insane what’s possible now. You can really genuinely take a random idea and bring it to life. And that too in < 1 hour. And who knows, this could actually be pretty useful for fashion designers and embroidery artists maybe? Live:

The Bugged Dev

61,617 Aufrufe • vor 1 Tag

just built a local AI waifu on my two macbooks by using the oss parallax inference library!!! not super smooth yet but parallax is still the easiest way to run llms locally on top of mlx - and no bigco can steal my waifu. still need folks to help her see/hear/draw since mlx has no legit wrapper yet. i’m running a campaign giving away mac minis + dgx spark. please please share your localhost ai model setup by using parallax (you don't need a gpu to try self-hosting you can just start with an extra macbook you borrowed from your friend like i did) or any fun app built on top of yourself host model to link below. come build sovereign intelligence with me 🫶
7:41

just built a local AI waifu on my two macbooks by using the oss parallax inference library!!! not super smooth yet but parallax is still the easiest way to run llms locally on top of mlx - and no bigco can steal my waifu. still need folks to help her see/hear/draw since mlx has no legit wrapper yet. i’m running a campaign giving away mac minis + dgx spark. please please share your localhost ai model setup by using parallax (you don't need a gpu to try self-hosting you can just start with an extra macbook you borrowed from your friend like i did) or any fun app built on top of yourself host model to link below. come build sovereign intelligence with me 🫶

Asuka Zheng🎀@ICML

178,998 Aufrufe • vor 9 Monaten

ELON MUSK: We believe the AI5 chip will be roughly comparable performance to an NVIDIA Blackwell, and at much less than 10% of the cost Transcription: I'm super hardcore on chips right now as you may be able to tell. I have chips on the brain. I dream about chips, Literally! Because in order to have a functional robot, you have to have a great AI chip. And it needs to be an inexpensive chip and it needs to be very power efficient So we think we believe the AI5 chip will be probably about a third of the power of say something like a Blackwell, an NVIDIA Blackwell, which is a great chip, for roughly comparable performance. And much less than 10% of the cost. This is a chip that is very much optimized for the Tesla AI software stack. So it's not meant to be a general purpose chip, it's meant to be an amazing chip for the Tesla AI software And I mean a couple of things that I think make... like how is Tesla able to achieve such an improvement? I think it is because we are specialized. We're not trying to... you know, NVIDIA has to serve the superset of all past and future customers. So all of their requirements, all of the software that they've written has to work, which is a very difficult problem. Whereas we just need to make it work for our software. And so we're able to simplify the chip dramatically And then we also, I think we're unique in this, but like we have an integer-based system. And integer operations are fundamentally more efficient than floating point operations. So we can do floating point, but the vast majority of our inference is done in integer. Which is, if you're familiar with sort of logic gates, the simplicity of integer... it's integer is much more power efficient, much more silicon efficient, but you have to, you actually have to train for integer inference, which everyone else is training for floating point. That's kind of like a niche technical detail, but it's actually very important. So, yeah, this is going to be a great chip So this chip will be made in basically in four places: TSMC Taiwan, Samsung Korea, TSMC Arizona, and TSMC Texas. And we already know what improvements to make for AI6. So I'm hopeful that we can within less than a year of AI5 starting production, we can actually transition in the same fab to AI6 and double all of the performance metrics
3:12

ELON MUSK: We believe the AI5 chip will be roughly comparable performance to an NVIDIA Blackwell, and at much less than 10% of the cost Transcription: I'm super hardcore on chips right now as you may be able to tell. I have chips on the brain. I dream about chips, Literally! Because in order to have a functional robot, you have to have a great AI chip. And it needs to be an inexpensive chip and it needs to be very power efficient So we think we believe the AI5 chip will be probably about a third of the power of say something like a Blackwell, an NVIDIA Blackwell, which is a great chip, for roughly comparable performance. And much less than 10% of the cost. This is a chip that is very much optimized for the Tesla AI software stack. So it's not meant to be a general purpose chip, it's meant to be an amazing chip for the Tesla AI software And I mean a couple of things that I think make... like how is Tesla able to achieve such an improvement? I think it is because we are specialized. We're not trying to... you know, NVIDIA has to serve the superset of all past and future customers. So all of their requirements, all of the software that they've written has to work, which is a very difficult problem. Whereas we just need to make it work for our software. And so we're able to simplify the chip dramatically And then we also, I think we're unique in this, but like we have an integer-based system. And integer operations are fundamentally more efficient than floating point operations. So we can do floating point, but the vast majority of our inference is done in integer. Which is, if you're familiar with sort of logic gates, the simplicity of integer... it's integer is much more power efficient, much more silicon efficient, but you have to, you actually have to train for integer inference, which everyone else is training for floating point. That's kind of like a niche technical detail, but it's actually very important. So, yeah, this is going to be a great chip So this chip will be made in basically in four places: TSMC Taiwan, Samsung Korea, TSMC Arizona, and TSMC Texas. And we already know what improvements to make for AI6. So I'm hopeful that we can within less than a year of AI5 starting production, we can actually transition in the same fab to AI6 and double all of the performance metrics

X Freeze

305,109 Aufrufe • vor 10 Monaten

Today we announced our new Fairwater datacenter in Atlanta, connected with our first Fairwater site in Wisconsin and our broader Azure footprint to create the world’s first AI superfactory. Fairwater exemplifies our vision for a fungible fleet: infra that can serve any workload, anywhere, on fit-for-purpose accelerators and network paths, with maximum performance and efficiency. AI workloads have evolved beyond large-scale pre-training. Today, they encompass fine-tuning, reinforcement learning (RL), synthetic data generation, evaluation pipelines, and more. Fairwater is built to support this full lifecycle: Max density: Fairwater’s two-story design and liquid cooling system lets us place racks in three dimensions and pack them with GPUs as densely as possible, minimizing cable runs and improving latency and effective bandwidth. Fleet: Each Fairwater DC can integrate hundreds of thousands of the latest NVIDIA GPUs into a single coherent cluster. This provides flexible infra that can support the full spectrum of workloads, and ensure no GPU is left unnecessarily idle. And that’s on top of the more than 100,000 GB300s coming online this quarter alone for inference across the rest of our fleet. For us, it’s all about turning every gigawatt into the maximum number of useful tokens. Not every GW is created equal! Planet-scale: Every Fairwater DC will connect through our continent-spanning AI WAN to prior generations of AI supercomputers, forming a truly fungible pool of compute. This enables developers to scale beyond the capacity of a single site and dynamically land workloads on the right infra for their needs. Together, these innovations let us bring together different generations of silicon and AI systems across DCs and geos into a single elastic system that scales seamlessly across training and inference workloads And this elastic AI capacity is all available alongside all the other cloud services (compute, storage, databases, app services) that AI agents and workloads need. This is what we mean when we talk about building a fungible fleet – a single, unified platform that pushes the limits of performance per watt and per dollar. Read more:
0:44

Today we announced our new Fairwater datacenter in Atlanta, connected with our first Fairwater site in Wisconsin and our broader Azure footprint to create the world’s first AI superfactory. Fairwater exemplifies our vision for a fungible fleet: infra that can serve any workload, anywhere, on fit-for-purpose accelerators and network paths, with maximum performance and efficiency. AI workloads have evolved beyond large-scale pre-training. Today, they encompass fine-tuning, reinforcement learning (RL), synthetic data generation, evaluation pipelines, and more. Fairwater is built to support this full lifecycle: Max density: Fairwater’s two-story design and liquid cooling system lets us place racks in three dimensions and pack them with GPUs as densely as possible, minimizing cable runs and improving latency and effective bandwidth. Fleet: Each Fairwater DC can integrate hundreds of thousands of the latest NVIDIA GPUs into a single coherent cluster. This provides flexible infra that can support the full spectrum of workloads, and ensure no GPU is left unnecessarily idle. And that’s on top of the more than 100,000 GB300s coming online this quarter alone for inference across the rest of our fleet. For us, it’s all about turning every gigawatt into the maximum number of useful tokens. Not every GW is created equal! Planet-scale: Every Fairwater DC will connect through our continent-spanning AI WAN to prior generations of AI supercomputers, forming a truly fungible pool of compute. This enables developers to scale beyond the capacity of a single site and dynamically land workloads on the right infra for their needs. Together, these innovations let us bring together different generations of silicon and AI systems across DCs and geos into a single elastic system that scales seamlessly across training and inference workloads And this elastic AI capacity is all available alongside all the other cloud services (compute, storage, databases, app services) that AI agents and workloads need. This is what we mean when we talk about building a fungible fleet – a single, unified platform that pushes the limits of performance per watt and per dollar. Read more:

Satya Nadella

908,065 Aufrufe • vor 10 Monaten