Step 4 to achieve truly serverless GPUs for AI... inference: skip over unserializable inference engine setup steps like CUDA graph capture and Torch compilation by stacking GPU snapshots and CPU snapshots.show more

Microsoft killed the GPU mafia 🤯 They finally open-sourced... their 1-bit LLM inference framework called bitnet.cpp. It lets you run 100B parameter models on your local CPU without GPUs. - 6.17x faster inference - 82.2% less energy on CPUs 100% Open Source.show more

Real-time Moondream inference using our new inference engine

Watching llama.cpp do 40 tok/s inference of the 7B... model on my M2 Max, with 0% CPU usage, and using all 38 GPU cores. Congratulations Georgi Gerganov ! This is a triumph.show more

It’s live. AxonDAO’s GPU fleet is installed and will... soon be rentable on the open market via vast.ai. Premium GPU capacity - 8x NVIDIA B200 and 8x RTX Pro 6000 - built for large-scale AI training, inference, rendering, and research workloads. 🧵show more

First steps for a specialized DeepSeek v4 Flash inference... engine focused on inference quality / stability at different quantizations, with networked API that is batching capable. This is the 2 bit quants model running on my M3 Max 128GB.show more

My AI broke the world record on Tempest yesterday!... But I still hold the human record :-) [on Extreme difficulty settings] Here's a little demo reel of the Tempest AI doing inference and training at the same time up on the hardest Tempest levels. This is all running on our Dell Technologies 7875 Workstation, with the 9995WX CPU handling 2000 fps of Tempest while the dual Blackwell RTX6000 GPUs do inference and training.show more

Today, Nesa is excited to announce it is partnering... with io.net to bring their decentralized fleet of GPUs to Nesa. Nesa and are natural complements to one another. Nesa’s private, distributed AI inference executed on ocean of decentralized compute means greater accessibility for the network. Together, we are one step closer to decentralizing AI for all.show more

Laika AI x Inference Labs Excited to announce our... partnership with Inference Labs We're providing our real-time RAG & AI model API to Inference Labs, powering their verification infrastructure with live blockchain data. Inference Labs delivers open-source, trustless verification for AI agent outputs, so you can trust what you see—without relying on centralized gatekeepers.show more

Depth Anything 3 now runs as pure C++/ggml (ggml)... . No Python, no PyTorch, no CUDA toolkit at inference, just one self-contained GGUF. It's faster than PyTorch on CPU! and ties speed on GPU. The CPU win came from the last place..I'd have looked. Quantized GGUF on Hugging Face🤗 Shout out to Georgi Gerganov for ggml (we are building a ggml-world!❤️) and to ByteDance Open Source and Depth Anything 3 authors Bingyi Kang Jun Hao Liew Donny Y. Chen !show more

Snapshots. Scenes. 📸

Excited to share our NeurIPS 2024 Oral, Convolutional Differentiable... Logic Gate Networks, leading to a range of inference efficiency records, including inference in only 4 nanoseconds 🏎️. We reduce model sizes by factors of 29x-61x over the SOTA. Paper:show more

Meigen MultiTalk @gradio demo is available on Hugging Face... 🤗 Duplicate on L40S for personal and unlimited inference, enjoy ! *Compatible with multi-GPU too 😉show more

parakeet.cpp: native C++/ggml (ggml) inference for NVIDIA AI Developer's... Parakeet, one of the best speech-to-text models out there, from the LocalAI team. Every Parakeet model (TDT/CTC/RNNT/hybrid + cache-aware streaming), byte-for-byte identical output to NeMo, now running anywhere with no Python and even a bit faster, on CPU and GPU. Quantized GGUF on Hugging Face 🤗 Huge thanks to Georgi Gerganov for ggml and to NVIDIA AI Developer for releasing Parakeet! 🧵show more

Your GPU is sitting idle right now. It could... be earning money. Download the Earn Module. Click start. That's the whole setup. While you sleep, scroll X, or binge Netflix, your graphics card powers AI inference and stacks GUSD in your account. Zero technical knowledge required.show more

Dolphin Inference Network node operation is now live for... anyone who would like to beta test before we go into production $POD rewards live for testers Repurposing idle GPUs to run Qwen 3.5 35B MoEshow more

Virtuals Protocol is integrating Venice to power AI agent... building with private, uncensored inference, available to anyone, anywhere on Base. Venice brings best-in-class privacy-first inference. Virtuals EconomyOS brings the full agent infrastructure stack: wallets, identity, payments, commerce, funding rails, and launch infrastructure. We are deploying up to $400,000 in private inference credits so anyone can move from idea to working agent without compute or backend complexity getting in the way. Start your AI journey. Inference and infra are on us. Program details soon.show more

Some snapshots from Regions Tradition for Daily Wire ⛳️.

🚀 Hyra AI: 2,500,000+ Connected Devices From smartphones and... tablets to IoT and edge GPUs, our growing network proves: Idle compute isn’t idle anymore. Every connected device contributes to powering real-time inference, edge learning, and decentralized AI applications across 205 countries. ⚙️ Scalable ⚡️ On-demand 🌍 Open to all Next stop? 10 million???show more