Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

we made distributed inference verifiable with <1% overhead. verification is critical for any distributed system. in a trustless network, actors may swap your 70B model for a cheaper 8B one to cut costs. until now, maintaining inference integrity meant either doubling your cost (redundancy) or exploding your latency (zkp)....

tryParallax's profile picture

Parallax

1,307 subscribers

28,432 Aufrufe • vor 5 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Breaking: We can now prove that AI is correct. DeepProve-1 is the first production-ready zkML system to cryptographically verify a full LLM inference. Lagrange has successfully proven the inference of OpenAI’s GPT-2, moving verifiable AI from theory to production: 🧵
0:22

Breaking: We can now prove that AI is correct. DeepProve-1 is the first production-ready zkML system to cryptographically verify a full LLM inference. Lagrange has successfully proven the inference of OpenAI’s GPT-2, moving verifiable AI from theory to production: 🧵

LAGRANGE

249,292 Aufrufe • vor 10 Monaten

NVIDIA sent us 2 DGX Sparks. For a while we wondered what we would do with them. The memory bandwidth is 273GB/s making it 3x slower than an M3 Ultra (819GB/s) for batch_size=1 inference. But it has 4x more FLOPS (100 TFLOPS compared to 26 TFLOPS). So we thought, what if we could combine the DGX Spark & M3 Ultra, and make use of both the massive compute on the DGX Spark and the massive memory-bandwidth on the M3 Ultra. We came up with a way to split inference across both devices and achieve a speedup of up to 4x for long prompts compared to the M3 Ultra on its own. Full details in the blog post linked below.
1:13

NVIDIA sent us 2 DGX Sparks. For a while we wondered what we would do with them. The memory bandwidth is 273GB/s making it 3x slower than an M3 Ultra (819GB/s) for batch_size=1 inference. But it has 4x more FLOPS (100 TFLOPS compared to 26 TFLOPS). So we thought, what if we could combine the DGX Spark & M3 Ultra, and make use of both the massive compute on the DGX Spark and the massive memory-bandwidth on the M3 Ultra. We came up with a way to split inference across both devices and achieve a speedup of up to 4x for long prompts compared to the M3 Ultra on its own. Full details in the blog post linked below.

Alex Cheema

281,225 Aufrufe • vor 8 Monaten

In just one week, Binh Pham and I trained a full-body Unitree G1. Here's a recap: 1. Secured a Unitree G1 humanoid through a LinkedIn post 2. Deployed TWIST2 full-body teleoperation pipelines 3. Adapted TWIST2 for Zed stereo camera & collected full-body teleoperation samples (carried by Binh Pham ) 4. Adapted & fine-tuned NVIDIA Gr00T N1.5 VLA on the TWIST2 public datasets, which I fine-tuned on an 8xNVIDIA H100 Cluster. We picked Gr00T N1.5 as it was trained with Unitree G1 embodiment data. 5. Adapted the TWIST2 codebase to stream in the actions from Gr00T via ZMQ using a co-located NVIDIA H100 for ~200ms inference latency 6. Tested the model in sim, then deployed to the real-world Unitree G1. We streamed a training sample observation to the VLA (as we didn't want to break robot in case real observations were OOD) We were the first team in the world to deploy the full TWIST2 data collection pipeline to the unitree g1 :) Much more work ahead though, which I'll work on as a side-project over the next months: 1. Exploring the various types of 'world models': video backbones, dynamics models, v-jepa-2 models. I believe these will generalize better & train much more data-efficiently than VLM backbones 2. Speeding up inference - I believe low-latency robotics inference will be a big challenge. There are many works in video diffusion which I'd like to test (e.g. SageAttention, SparseAttention, Drifting Models). Perhaps also writing custom CUDA kernels. 3. Economics of inference scaling :) What will be the compute demands as we scale inference up to millions of humanoids? Will it run on edge or on distributed 'co-located' inference clusters? These are questions I'd like to answer. Adapted TWIST2 codebase: Adapted Gr00T-N1.5 codebase: The ETH Robotics Club are doing a cool GTC Golden ticket competition with NVIDIA , so this is my submission :) The DGX Spark compute will get me a long way with initial prototyping & especially working on inference optimization for next-gen Blackwell GPUs #NVIDIAGTC #GOLDENTICKET #ETHRC
0:45

In just one week, Binh Pham and I trained a full-body Unitree G1. Here's a recap: 1. Secured a Unitree G1 humanoid through a LinkedIn post 2. Deployed TWIST2 full-body teleoperation pipelines 3. Adapted TWIST2 for Zed stereo camera & collected full-body teleoperation samples (carried by Binh Pham ) 4. Adapted & fine-tuned NVIDIA Gr00T N1.5 VLA on the TWIST2 public datasets, which I fine-tuned on an 8xNVIDIA H100 Cluster. We picked Gr00T N1.5 as it was trained with Unitree G1 embodiment data. 5. Adapted the TWIST2 codebase to stream in the actions from Gr00T via ZMQ using a co-located NVIDIA H100 for ~200ms inference latency 6. Tested the model in sim, then deployed to the real-world Unitree G1. We streamed a training sample observation to the VLA (as we didn't want to break robot in case real observations were OOD) We were the first team in the world to deploy the full TWIST2 data collection pipeline to the unitree g1 :) Much more work ahead though, which I'll work on as a side-project over the next months: 1. Exploring the various types of 'world models': video backbones, dynamics models, v-jepa-2 models. I believe these will generalize better & train much more data-efficiently than VLM backbones 2. Speeding up inference - I believe low-latency robotics inference will be a big challenge. There are many works in video diffusion which I'd like to test (e.g. SageAttention, SparseAttention, Drifting Models). Perhaps also writing custom CUDA kernels. 3. Economics of inference scaling :) What will be the compute demands as we scale inference up to millions of humanoids? Will it run on edge or on distributed 'co-located' inference clusters? These are questions I'd like to answer. Adapted TWIST2 codebase: Adapted Gr00T-N1.5 codebase: The ETH Robotics Club are doing a cool GTC Golden ticket competition with NVIDIA , so this is my submission :) The DGX Spark compute will get me a long way with initial prototyping & especially working on inference optimization for next-gen Blackwell GPUs #NVIDIAGTC #GOLDENTICKET #ETHRC

Arnie Ramesh

14,815 Aufrufe • vor 4 Monaten

See how our Director of Growth and AI Partnerships ravidilse.eth walks you through the Proof of Uniqueness (PoU) process in 4 steps: Securing a Sentient Airdrop with PoU from Billions Step 1: Create Account Step 2: Check Eligibility Step 3: Verify PoU Step 4: Add Wallet Address You are not expected to download any software or install any application. The verification happens on your device. Within a few seconds, you are able to complete the verification process. Once completed, you will have the liveliness and uniqueness credential in your wallet. With a click of the "Verify" button, you can then generate a zero knowledge proof that can be shared with the verifier. The verifier can be Sentient or a third party like Clique. Note: No user data is being stored by Billions. Once the verification is done, you are eligible for the airdrop. This is where Sentient ensures that the treasury tokens are not being distributed to bot farmers - instead they are distributed to real, unique humans like you.
1:28

See how our Director of Growth and AI Partnerships ravidilse.eth walks you through the Proof of Uniqueness (PoU) process in 4 steps: Securing a Sentient Airdrop with PoU from Billions Step 1: Create Account Step 2: Check Eligibility Step 3: Verify PoU Step 4: Add Wallet Address You are not expected to download any software or install any application. The verification happens on your device. Within a few seconds, you are able to complete the verification process. Once completed, you will have the liveliness and uniqueness credential in your wallet. With a click of the "Verify" button, you can then generate a zero knowledge proof that can be shared with the verifier. The verifier can be Sentient or a third party like Clique. Note: No user data is being stored by Billions. Once the verification is done, you are eligible for the airdrop. This is where Sentient ensures that the treasury tokens are not being distributed to bot farmers - instead they are distributed to real, unique humans like you.

Billions

21,620 Aufrufe • vor 5 Monaten

2 1/2 days in of cooking up a verification system for the Ape Gang Community of Trainwreck . - Bot Protection - Captcha Control - 2 Step Verification from Twitch using their OAuth API to Kick verification using a generated code verified in the KickTools chatroom - Twitch OAuth only Login in for a secure connection (Twitch been around the block for awhile and has it figured out) - Admin access to search for users and cross check their platforms for verification - Duplicate field checks (For verifying Singleton BTC address and more) - Leaderboards and Point system coming soon (Watch time etc) Still more to come. But the ideal is Train viewers have to authenticate with both Twitch and Kick accounts, then submit BTC addresses or any field that the community would want for duplicate checks. This will allow more controlled giveaways with a verification system. Leaving the possibility of bots less likely. Plus some added community perks like the leaderboards (watch time etc) for comradery between the community Kick Twitch
1:52

2 1/2 days in of cooking up a verification system for the Ape Gang Community of Trainwreck . - Bot Protection - Captcha Control - 2 Step Verification from Twitch using their OAuth API to Kick verification using a generated code verified in the KickTools chatroom - Twitch OAuth only Login in for a secure connection (Twitch been around the block for awhile and has it figured out) - Admin access to search for users and cross check their platforms for verification - Duplicate field checks (For verifying Singleton BTC address and more) - Leaderboards and Point system coming soon (Watch time etc) Still more to come. But the ideal is Train viewers have to authenticate with both Twitch and Kick accounts, then submit BTC addresses or any field that the community would want for duplicate checks. This will allow more controlled giveaways with a verification system. Leaving the possibility of bots less likely. Plus some added community perks like the leaderboards (watch time etc) for comradery between the community Kick Twitch

Babz

10,332 Aufrufe • vor 1 Jahr

We are excited to share a preview of our peer-to-peer decentralized inference stack Engineered for consumer GPUs and high-latency networks — plus a research roadmap to scale it to a planetary-scale decentralized inference engine.
0:40

We are excited to share a preview of our peer-to-peer decentralized inference stack Engineered for consumer GPUs and high-latency networks — plus a research roadmap to scale it to a planetary-scale decentralized inference engine.

Prime Intellect

103,625 Aufrufe • vor 1 Jahr

We know our robotaxis look identical, so we created a feature to help you spot your Zoox on the street. Introducing Find My Zoox! Set your unique theme in the app, then easily locate your ride with a personalized light and sound sequence. 🌈👋🔊
0:29

We know our robotaxis look identical, so we created a feature to help you spot your Zoox on the street. Introducing Find My Zoox! Set your unique theme in the app, then easily locate your ride with a personalized light and sound sequence. 🌈👋🔊

Zoox

35,409 Aufrufe • vor 1 Monat

Wake the world's sleeping compute. Look at the Mac nearest to you. What's it doing? Probably nothing. There are 100M+ Macs with Apple Silicon out there. Apple quietly made them *really* good at inference. A $3k Mac runs a 60B model at 30 watts. Most sit idle most of the day. Meanwhile every AI API call passes through three layers of margin before reaching the hardware. We call this the Inference Tax. We got curious: what happens if you connect idle Macs directly to inference demand? This is Darkbloom. Private inference network for idle Macs. darkbloom [dot] dev -- paper + code open. Reply for invite + free credits ↓
2:58

Wake the world's sleeping compute. Look at the Mac nearest to you. What's it doing? Probably nothing. There are 100M+ Macs with Apple Silicon out there. Apple quietly made them *really* good at inference. A $3k Mac runs a 60B model at 30 watts. Most sit idle most of the day. Meanwhile every AI API call passes through three layers of margin before reaching the hardware. We call this the Inference Tax. We got curious: what happens if you connect idle Macs directly to inference demand? This is Darkbloom. Private inference network for idle Macs. darkbloom [dot] dev -- paper + code open. Reply for invite + free credits ↓

Gajesh

504,617 Aufrufe • vor 2 Monaten

An AI analyzed the scan and produced an answer. But can you prove what it actually saw? In high-stakes systems, trust requires verification. Verifiable inference changes how AI is trusted.
0:43

An AI analyzed the scan and produced an answer. But can you prove what it actually saw? In high-stakes systems, trust requires verification. Verifiable inference changes how AI is trusted.

OpenGradient (∇, ∇)

191,263 Aufrufe • vor 4 Monaten

$NBIS cofounder Roman Chernin describes how their recent acquisitions of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."
2:32

$NBIS cofounder Roman Chernin describes how their recent acquisitions of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."

TBPN

30,486 Aufrufe • vor 1 Monat

I'll let Noam Brown (OpenAI research) answer Yann's “told you so” again. There is nothing to add. There is no wall, there is no end in sight! "Now, we are in a world, like I said earlier, where the amount of compute that's going into pretraining for things large language models is very, very high. But the inference costs are very low. And there was a reasonable concern among various people that we were going to start seeing diminishing returns from AI progress because the costs and the amount of data that you need for pretraining would become so astronomical. And I think the really important takeaway from o1 is that that wall doesn't actually exist, that we can actually push this a lot further. Because, now, we can scale up inference compute. And there's so much room to scale up inference compute."
0:46

I'll let Noam Brown (OpenAI research) answer Yann's “told you so” again. There is nothing to add. There is no wall, there is no end in sight! "Now, we are in a world, like I said earlier, where the amount of compute that's going into pretraining for things large language models is very, very high. But the inference costs are very low. And there was a reasonable concern among various people that we were going to start seeing diminishing returns from AI progress because the costs and the amount of data that you need for pretraining would become so astronomical. And I think the really important takeaway from o1 is that that wall doesn't actually exist, that we can actually push this a lot further. Because, now, we can scale up inference compute. And there's so much room to scale up inference compute."

Chubby♨️

132,808 Aufrufe • vor 1 Jahr

IN NEWS: Baseten raises a $150M series D round. Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”
0:52

IN NEWS: Baseten raises a $150M series D round. Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”

TBPN

17,057 Aufrufe • vor 9 Monaten

Just launched #CES2026, the new open-source NVIDIA Nemotron Speech ASR model is here to solve latency drift and redundant compute. Its cache-aware streaming architecture eliminates the need for buffered inference, giving you stable, sub-100ms latency (24ms median T-T-F) and up to 3x more throughput on your GPU. 🤗 Read the technical blog with real-world results from Daily and Modal on Hugging Face:
1:24

Just launched #CES2026, the new open-source NVIDIA Nemotron Speech ASR model is here to solve latency drift and redundant compute. Its cache-aware streaming architecture eliminates the need for buffered inference, giving you stable, sub-100ms latency (24ms median T-T-F) and up to 3x more throughput on your GPU. 🤗 Read the technical blog with real-world results from Daily and Modal on Hugging Face:

NVIDIA AI Developer

138,370 Aufrufe • vor 5 Monaten

An OTP verification using Python steps: 1.First, create a 6-digit random number. 2.Then store the number in a variable 3. When sending an email, we need to use OTP as a message. 4. Finally, we need to request two user inputs. #syncintern #Python
0:45

An OTP verification using Python steps: 1.First, create a 6-digit random number. 2.Then store the number in a variable 3. When sending an email, we need to use OTP as a message. 4. Finally, we need to request two user inputs. #syncintern #Python

Regan.

35,259 Aufrufe • vor 2 Jahren

Real-time video captioning in your browser with @LiquidAI's LFM2-VL model on WebGPU. Sending every frame to a server was never going to be the answer. Imagine the bandwidth, latency and cost. Local inference. No server costs. Infinitely scalable. This is the way.
0:45

Real-time video captioning in your browser with @LiquidAI's LFM2-VL model on WebGPU. Sending every frame to a server was never going to be the answer. Imagine the bandwidth, latency and cost. Local inference. No server costs. Infinitely scalable. This is the way.

Xenova

48,682 Aufrufe • vor 3 Monaten

Introducing Parallax, the first fully distributed inference and serving engine for large language models. Try it now: 🧵
0:26

Introducing Parallax, the first fully distributed inference and serving engine for large language models. Try it now: 🧵

Gradient

160,973 Aufrufe • vor 1 Jahr

Rivian CEO RJ Scaringe in new interview on self-driving: "Tesla's got a huge fleet. So our approach is we have to catch up with them and compete. And so we're going to have more cameras with better dynamic range. We're going to supplement that with LiDAR that provides better safety for edge case conditions and allows us to train the vision models faster. And the incremental cost to do that is relatively small. And in fact any of the incremental costs that would have been there on its own is offset by the fact that we brought inference in house and reduced the cost of our inference platform so dramatically from what we have in your car. Your car uses an Nvidia inference platform. I say all this because in like the infinite long-term you know, you could make the case that once the models are very very robust you could have less cameras or you may be able to get away with less radar. It's not clear yet if that's the case for covering all these corner cases."
3:06

Rivian CEO RJ Scaringe in new interview on self-driving: "Tesla's got a huge fleet. So our approach is we have to catch up with them and compete. And so we're going to have more cameras with better dynamic range. We're going to supplement that with LiDAR that provides better safety for edge case conditions and allows us to train the vision models faster. And the incremental cost to do that is relatively small. And in fact any of the incremental costs that would have been there on its own is offset by the fact that we brought inference in house and reduced the cost of our inference platform so dramatically from what we have in your car. Your car uses an Nvidia inference platform. I say all this because in like the infinite long-term you know, you could make the case that once the models are very very robust you could have less cameras or you may be able to get away with less radar. It's not clear yet if that's the case for covering all these corner cases."

Sawyer Merritt

446,376 Aufrufe • vor 3 Monaten

1/2 Bring your characters to life like never before! We're officially launching Wan2.2-Animate, a unified model for high-fidelity character animation and replacement. The model weights and inference code are now open-source for the entire community! • Character Animation: Feed it a character image and a reference video, and it will animate your character by precisely replicating the source's expressions and movements. • Character Replacement: Seamlessly swap the animated character into the original video's scene. It automatically matches the lighting and color tone for perfect environmental integration.
1:19

1/2 Bring your characters to life like never before! We're officially launching Wan2.2-Animate, a unified model for high-fidelity character animation and replacement. The model weights and inference code are now open-source for the entire community! • Character Animation: Feed it a character image and a reference video, and it will animate your character by precisely replicating the source's expressions and movements. • Character Replacement: Seamlessly swap the animated character into the original video's scene. It automatically matches the lighting and color tone for perfect environmental integration.

Wan

4,522,618 Aufrufe • vor 9 Monaten