we made distributed inference verifiable with <1% overhead. verification... is critical for any distributed system. in a trustless network, actors may swap your 70B model for a cheaper 8B one to cut costs. until now, maintaining inference integrity meant either doubling your cost (redundancy) or exploding your latency (zkp). we created veri: an on-chain verification layer light enough for high-throughput frameworks like Parallax. it hits the economic sweet spot through architectural elegance: 1. commit-sample-verify we don't prove every step; we check a random slice using game theory. workers commit to their work before the audit. cheating becomes statistically irrational, allowing a 1% sample to secure the entire sequence. 2. simultaneous execution inference and verification happen simultaneously on the same worker pool. we don't need a separate "verifier set", so compute utilization stays high. find out more about the architecture and benchmarks: paper: blog:show more

Breaking: We can now prove that AI is correct.... DeepProve-1 is the first production-ready zkML system to cryptographically verify a full LLM inference. Lagrange has successfully proven the inference of OpenAI’s GPT-2, moving verifiable AI from theory to production: 🧵show more

NVIDIA sent us 2 DGX Sparks. For a while... we wondered what we would do with them. The memory bandwidth is 273GB/s making it 3x slower than an M3 Ultra (819GB/s) for batch_size=1 inference. But it has 4x more FLOPS (100 TFLOPS compared to 26 TFLOPS). So we thought, what if we could combine the DGX Spark & M3 Ultra, and make use of both the massive compute on the DGX Spark and the massive memory-bandwidth on the M3 Ultra. We came up with a way to split inference across both devices and achieve a speedup of up to 4x for long prompts compared to the M3 Ultra on its own. Full details in the blog post linked below.show more

In just one week, Binh Pham and I trained... a full-body Unitree G1. Here's a recap: 1. Secured a Unitree G1 humanoid through a LinkedIn post 2. Deployed TWIST2 full-body teleoperation pipelines 3. Adapted TWIST2 for Zed stereo camera & collected full-body teleoperation samples (carried by Binh Pham ) 4. Adapted & fine-tuned NVIDIA Gr00T N1.5 VLA on the TWIST2 public datasets, which I fine-tuned on an 8xNVIDIA H100 Cluster. We picked Gr00T N1.5 as it was trained with Unitree G1 embodiment data. 5. Adapted the TWIST2 codebase to stream in the actions from Gr00T via ZMQ using a co-located NVIDIA H100 for ~200ms inference latency 6. Tested the model in sim, then deployed to the real-world Unitree G1. We streamed a training sample observation to the VLA (as we didn't want to break robot in case real observations were OOD) We were the first team in the world to deploy the full TWIST2 data collection pipeline to the unitree g1 :) Much more work ahead though, which I'll work on as a side-project over the next months: 1. Exploring the various types of 'world models': video backbones, dynamics models, v-jepa-2 models. I believe these will generalize better & train much more data-efficiently than VLM backbones 2. Speeding up inference - I believe low-latency robotics inference will be a big challenge. There are many works in video diffusion which I'd like to test (e.g. SageAttention, SparseAttention, Drifting Models). Perhaps also writing custom CUDA kernels. 3. Economics of inference scaling :) What will be the compute demands as we scale inference up to millions of humanoids? Will it run on edge or on distributed 'co-located' inference clusters? These are questions I'd like to answer. Adapted TWIST2 codebase: Adapted Gr00T-N1.5 codebase: The ETH Robotics Club are doing a cool GTC Golden ticket competition with NVIDIA , so this is my submission :) The DGX Spark compute will get me a long way with initial prototyping & especially working on inference optimization for next-gen Blackwell GPUs #NVIDIAGTC #GOLDENTICKET #ETHRCshow more

AI inference is already live on UOMI Router. Access... frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:show more

See how our Director of Growth and AI Partnerships... ravidilse.eth walks you through the Proof of Uniqueness (PoU) process in 4 steps: Securing a Sentient Airdrop with PoU from Billions Step 1: Create Account Step 2: Check Eligibility Step 3: Verify PoU Step 4: Add Wallet Address You are not expected to download any software or install any application. The verification happens on your device. Within a few seconds, you are able to complete the verification process. Once completed, you will have the liveliness and uniqueness credential in your wallet. With a click of the "Verify" button, you can then generate a zero knowledge proof that can be shared with the verifier. The verifier can be Sentient or a third party like Clique. Note: No user data is being stored by Billions. Once the verification is done, you are eligible for the airdrop. This is where Sentient ensures that the treasury tokens are not being distributed to bot farmers - instead they are distributed to real, unique humans like you.show more

2 1/2 days in of cooking up a verification... system for the Ape Gang Community of Trainwreck . - Bot Protection - Captcha Control - 2 Step Verification from Twitch using their OAuth API to Kick verification using a generated code verified in the KickTools chatroom - Twitch OAuth only Login in for a secure connection (Twitch been around the block for awhile and has it figured out) - Admin access to search for users and cross check their platforms for verification - Duplicate field checks (For verifying Singleton BTC address and more) - Leaderboards and Point system coming soon (Watch time etc) Still more to come. But the ideal is Train viewers have to authenticate with both Twitch and Kick accounts, then submit BTC addresses or any field that the community would want for duplicate checks. This will allow more controlled giveaways with a verification system. Leaving the possibility of bots less likely. Plus some added community perks like the leaderboards (watch time etc) for comradery between the community Kick Twitchshow more

We are excited to share a preview of our... peer-to-peer decentralized inference stack Engineered for consumer GPUs and high-latency networks — plus a research roadmap to scale it to a planetary-scale decentralized inference engine.show more

We know our robotaxis look identical, so we created... a feature to help you spot your Zoox on the street. Introducing Find My Zoox! Set your unique theme in the app, then easily locate your ride with a personalized light and sound sequence. 🌈👋🔊show more

Wake the world's sleeping compute. Look at the Mac... nearest to you. What's it doing? Probably nothing. There are 100M+ Macs with Apple Silicon out there. Apple quietly made them *really* good at inference. A $3k Mac runs a 60B model at 30 watts. Most sit idle most of the day. Meanwhile every AI API call passes through three layers of margin before reaching the hardware. We call this the Inference Tax. We got curious: what happens if you connect idle Macs directly to inference demand? This is Darkbloom. Private inference network for idle Macs. darkbloom [dot] dev -- paper + code open. Reply for invite + free credits ↓show more

An AI analyzed the scan and produced an answer.... But can you prove what it actually saw? In high-stakes systems, trust requires verification. Verifiable inference changes how AI is trusted.show more

$NBIS cofounder Roman Chernin describes how their recent acquisitions... of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."show more

I'll let Noam Brown (OpenAI research) answer Yann's “told... you so” again. There is nothing to add. There is no wall, there is no end in sight! "Now, we are in a world, like I said earlier, where the amount of compute that's going into pretraining for things large language models is very, very high. But the inference costs are very low. And there was a reasonable concern among various people that we were going to start seeing diminishing returns from AI progress because the costs and the amount of data that you need for pretraining would become so astronomical. And I think the really important takeaway from o1 is that that wall doesn't actually exist, that we can actually push this a lot further. Because, now, we can scale up inference compute. And there's so much room to scale up inference compute."show more

Just launched #CES2026, the new open-source NVIDIA Nemotron Speech... ASR model is here to solve latency drift and redundant compute. Its cache-aware streaming architecture eliminates the need for buffered inference, giving you stable, sub-100ms latency (24ms median T-T-F) and up to 3x more throughput on your GPU. 🤗 Read the technical blog with real-world results from Daily and Modal on Hugging Face:show more

IN NEWS: Baseten raises a $150M series D round.... Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”show more

An OTP verification using Python steps: 1.First, create a... 6-digit random number. 2.Then store the number in a variable 3. When sending an email, we need to use OTP as a message. 4. Finally, we need to request two user inputs. #syncintern #Pythonshow more

Real-time video captioning in your browser with @LiquidAI's LFM2-VL... model on WebGPU. Sending every frame to a server was never going to be the answer. Imagine the bandwidth, latency and cost. Local inference. No server costs. Infinitely scalable. This is the way.show more

Introducing Parallax, the first fully distributed inference and serving... engine for large language models. Try it now: 🧵show more

Rivian CEO RJ Scaringe in new interview on self-driving:... "Tesla's got a huge fleet. So our approach is we have to catch up with them and compete. And so we're going to have more cameras with better dynamic range. We're going to supplement that with LiDAR that provides better safety for edge case conditions and allows us to train the vision models faster. And the incremental cost to do that is relatively small. And in fact any of the incremental costs that would have been there on its own is offset by the fact that we brought inference in house and reduced the cost of our inference platform so dramatically from what we have in your car. Your car uses an Nvidia inference platform. I say all this because in like the infinite long-term you know, you could make the case that once the models are very very robust you could have less cameras or you may be able to get away with less radar. It's not clear yet if that's the case for covering all these corner cases."show more