we made distributed inference verifiable with <1% overhead. verification... is critical for any distributed system. in a trustless network, actors may swap your 70B model for a cheaper 8B one to cut costs. until now, maintaining inference integrity meant either doubling your cost (redundancy) or exploding your latency (zkp). we created veri: an on-chain verification layer light enough for high-throughput frameworks like Parallax. it hits the economic sweet spot through architectural elegance: 1. commit-sample-verify we don't prove every step; we check a random slice using game theory. workers commit to their work before the audit. cheating becomes statistically irrational, allowing a 1% sample to secure the entire sequence. 2. simultaneous execution inference and verification happen simultaneously on the same worker pool. we don't need a separate "verifier set", so compute utilization stays high. find out more about the architecture and benchmarks: paper: blog:show more

Breaking: We can now prove that AI is correct.... DeepProve-1 is the first production-ready zkML system to cryptographically verify a full LLM inference. Lagrange has successfully proven the inference of OpenAI’s GPT-2, moving verifiable AI from theory to production: 🧵show more

$MDL is now listed on SafeTrade. A new step... for ModelOS as we continue building a Proof-of-Useful-Work Layer 1 for GPU mining, AI compute, dCloud, and future inference utility.show more

Hello WAG Team, We noticed that WAG's player verification... requires a one-time payment of 0.09 TON, plus an estimated 0.005 TON network fee. We understand that the stated purpose is to verify genuine human players, distinguish them from bots, and award 1,500 points. However, we would appreciate clarification on a few important questions: 1. Why is a payment required to verify that a player is a real person rather than a bot? 2. How will the 0.09 TON verification fee be used? Is it allocated to verification services, project operations, or another purpose? 3. Is the verification fee refundable, or is it a permanent, non-refundable payment? 4. Is there a free alternative for users who cannot afford the fee? 5. Does completing this verification guarantee eligibility for the $WAG airdrop, or does it only confirm the player's status and award 1,500 points? 6. Can the team provide official documentation explaining the verification process and payment? We are asking these questions to help the community understand the process before paying. This is not an accusation; we believe transparency helps build trust. Thank you for your clarification. WAG Join me on TG:show more

NVIDIA sent us 2 DGX Sparks. For a while... we wondered what we would do with them. The memory bandwidth is 273GB/s making it 3x slower than an M3 Ultra (819GB/s) for batch_size=1 inference. But it has 4x more FLOPS (100 TFLOPS compared to 26 TFLOPS). So we thought, what if we could combine the DGX Spark & M3 Ultra, and make use of both the massive compute on the DGX Spark and the massive memory-bandwidth on the M3 Ultra. We came up with a way to split inference across both devices and achieve a speedup of up to 4x for long prompts compared to the M3 Ultra on its own. Full details in the blog post linked below.show more

SGLang now supports DSpark, enabling confidence-driven, variable-length verification for... speculative decoding 🎉 DSpark addresses a key bottleneck under load: instead of verifying every draft token, it verifies only where the draft model is confident, so the gains hold even as batch size scales. We heavily optimized variable-length verification in SGLang. Across batch sizes 1 to 256, DSpark gives the best throughput/latency tradeoff on DeepSeek-V4-Flash, ahead of both MTP and non-spec. At high concurrency, dynamic scheduling provides up to ~20% higher throughput compared to a fixed budget, while maintaining high verification quality across workloads. With fused kernels and zero-overhead scheduling, DeepSeek-V4-Pro reaches 383.7 tok/s at B=1 on B300. DSpark is now available in SGLang with support for Qwen3 and DeepSeek-V4. Thanks DeepSeek for open-sourcing! Blog with full technical details and commands to run below 👇show more

In just one week, Binh and I trained a... full-body Unitree G1. Here's a recap: 1. Secured a Unitree G1 humanoid through a LinkedIn post 2. Deployed TWIST2 full-body teleoperation pipelines 3. Adapted TWIST2 for Zed stereo camera & collected full-body teleoperation samples (carried by Binh ) 4. Adapted & fine-tuned NVIDIA Gr00T N1.5 VLA on the TWIST2 public datasets, which I fine-tuned on an 8xNVIDIA H100 Cluster. We picked Gr00T N1.5 as it was trained with Unitree G1 embodiment data. 5. Adapted the TWIST2 codebase to stream in the actions from Gr00T via ZMQ using a co-located NVIDIA H100 for ~200ms inference latency 6. Tested the model in sim, then deployed to the real-world Unitree G1. We streamed a training sample observation to the VLA (as we didn't want to break robot in case real observations were OOD) We were the first team in the world to deploy the full TWIST2 data collection pipeline to the unitree g1 :) Much more work ahead though, which I'll work on as a side-project over the next months: 1. Exploring the various types of 'world models': video backbones, dynamics models, v-jepa-2 models. I believe these will generalize better & train much more data-efficiently than VLM backbones 2. Speeding up inference - I believe low-latency robotics inference will be a big challenge. There are many works in video diffusion which I'd like to test (e.g. SageAttention, SparseAttention, Drifting Models). Perhaps also writing custom CUDA kernels. 3. Economics of inference scaling :) What will be the compute demands as we scale inference up to millions of humanoids? Will it run on edge or on distributed 'co-located' inference clusters? These are questions I'd like to answer. Adapted TWIST2 codebase: Adapted Gr00T-N1.5 codebase: The ETH Robotics Club are doing a cool GTC Golden ticket competition with NVIDIA , so this is my submission :) The DGX Spark compute will get me a long way with initial prototyping & especially working on inference optimization for next-gen Blackwell GPUs #NVIDIAGTC #GOLDENTICKET #ETHRCshow more

AI inference is already live on UOMI Router. Access... frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:show more

See how our Director of Growth and AI Partnerships... ravidilse.eth walks you through the Proof of Uniqueness (PoU) process in 4 steps: Securing a Sentient Airdrop with PoU from Billions Step 1: Create Account Step 2: Check Eligibility Step 3: Verify PoU Step 4: Add Wallet Address You are not expected to download any software or install any application. The verification happens on your device. Within a few seconds, you are able to complete the verification process. Once completed, you will have the liveliness and uniqueness credential in your wallet. With a click of the "Verify" button, you can then generate a zero knowledge proof that can be shared with the verifier. The verifier can be Sentient or a third party like Clique. Note: No user data is being stored by Billions. Once the verification is done, you are eligible for the airdrop. This is where Sentient ensures that the treasury tokens are not being distributed to bot farmers - instead they are distributed to real, unique humans like you.show more

We are excited to share a preview of our... peer-to-peer decentralized inference stack Engineered for consumer GPUs and high-latency networks — plus a research roadmap to scale it to a planetary-scale decentralized inference engine.show more

DigitalOcean Inference Router is now cache-aware. 🔀 Before, switching... models mid-conversation meant losing your cache, and the savings and consistency it bought you. Now, Inference Router weighs that cost before switching. Set a model affinity header or routing budget for full control.show more

Sachin Katti says OpenAI had Astra optimize its own... inference on NVIDIA's Rubin chip, adding another 2x in 72 hours "It's a testament to NVIDIA's hardware that out of the box we got a 3x improvement in throughput. We literally took the same code that we had optimized for Blackwell, ran it on Rubin, got 3x improvement." "Then, can we actually make Astra run agents using Astra as the base model, go optimize this inference of Astra itself on Rubin? It's a recursive loop. The model is figuring out how to optimize its own inference serving on Rubin chips, and over 72 hours the model produced another 2x improvement on the throughput." "Things that would take weeks, if not months, to optimize in the past, we now are getting models that are very capable of actually producing over a weekend." _______ Link and more from Sachin Katti's conversation:show more

we achieved some progress in the implementation of state... inference inside octra , first we managed to implement continuous reasoning without clearing the circle memory, second we completed a pure resource model under which validators can verify and certify the execution flow of a set of useful work (in this case this is ML inference with text generation by input tokens fully onchain), a few months ago the answers of the test model were one-word and substantially limited, in the current version (which will be available for everyone) the reasoning flow is automatically extended, it will be possible to upload various models into circles or create tools for their use making them public or only private (full demo will be opensource as soon as we finish, it will be possible to deploy anything in 2-3 clicks) live use case, the economic model also converges here - wallet pays for inference in native coins, validators (with GPU support can maintain availability of their resources to quickly process such private calls, while the execution core remains private through HFHE)show more

Wake the world's sleeping compute. Look at the Mac... nearest to you. What's it doing? Probably nothing. There are 100M+ Macs with Apple Silicon out there. Apple quietly made them *really* good at inference. A $3k Mac runs a 60B model at 30 watts. Most sit idle most of the day. Meanwhile every AI API call passes through three layers of margin before reaching the hardware. We call this the Inference Tax. We got curious: what happens if you connect idle Macs directly to inference demand? This is Darkbloom. Private inference network for idle Macs. darkbloom [dot] dev -- paper + code open. Reply for invite + free credits ↓show more

Over the past month, we completely switched to frontier... LLMs to control a robot in a customer deployment. By doing this, we have a classic System 1/System 2 architecture, but on steroids. For the high-level reasoning layer, we use an agent based on Opus and Astra, which are many generations more capable than any VLA. For the fast execution layer, we use computer vision code written by the model, which is fast, editable, and reliable. As a bonus, we got in-context learning, which is just a basic feature of LLMs, and continual learning, since whenever a new situation arises, the large model steps in and adapts the pipeline. And it just feels better to be on the right side of the Bitter Lesson, benefiting from the billions being invested in frontier models around the world.show more

An AI analyzed the scan and produced an answer.... But can you prove what it actually saw? In high-stakes systems, trust requires verification. Verifiable inference changes how AI is trusted.show more

Today we are proposing a new Age Verification system... for Digital EU 🇪🇺 . No need for face or ID verification, with 100% accuracy. Nyach Linux using an advanced AI algorithms managed to create the most reliable and secure age predictor. My 0yr old nephew's test attached.show more

$NBIS cofounder Roman Chernin describes how their recent acquisitions... of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."show more

Introducing Contrastive Language Model (CLM): an ultra-fast System One... Model trained with a contrastive learning objective that connects states and actions. CLM-8B is pre-trained on internet-scale data and delivers up to 9× faster inference than Jev ⚡ while achieving comparable performance across computer-use, gaming, and tool-calling tasks. With lightweight fine-tuning, CLM-8B sets a new SOTA on challenging agentic coding benchmarks, such as DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%). In contrast, Jev fails to serve as an effective verifier for these long-horizon tasks. We also build an efficient training and serving infra for CLMs by disaggregating states and actions, allowing their embeddings to be cached and reused independently. This substantially reduces inference latency in settings where the state evolves continuously while the action set remains fixed. Finally, we establish scaling laws for CLMs and show that the test contrastive loss decreases predictably as a power law in training compute, model size, and dataset size. 📄 Blog: 💻 Code: 🗣️ Discord: 🤗 Data & Models: More details on CLM’s architecture, data recipe, and scaling laws in the thread below 🧵show more

Exciting to see Chamath and J-Cal discussing the power... of Targon distributed confidential compute on The All-In Podcast “I do think this idea of distributed inference has a real place in the American ecosystem.” We appreciate the mention, and are excited to watch the continued growth of the permissionless compute space.show more

Need faster LLM inference without sacrificing accuracy? Speculative decoding... can help. Choosing the right draft length and drafting method depends on your model, workload and hardware. We break down five practical guidelines for balancing throughput and latency.show more