Real-time Moondream inference using our new inference engine

the inference engine

Step 4 to achieve truly serverless GPUs for AI... inference: skip over unserializable inference engine setup steps like CUDA graph capture and Torch compilation by stacking GPU snapshots and CPU snapshots.show more

First steps for a specialized DeepSeek v4 Flash inference... engine focused on inference quality / stability at different quantizations, with networked API that is batching capable. This is the 2 bit quants model running on my M3 Max 128GB.show more

Dot Staking. A new economic layer for AI inference.... 9 days.show more

First AI Credit Marketplace + inference trading on @Arc.... Coming soon. 👀 Sell unused AI credits. Buy inference for less. Trade OpenAI ↔ Claude and more. AI inference is becoming a market.show more

Staking near:native, farming inference 🤯

Batching for vision models is now available in Beta... with our latest MLX engine update 👾 The updated engine also brings major improvements to caching for faster inference overall. Turn on Developer Mode, choose the beta runtime channel, and select LM Studio MLX v1.8.1.show more

Introducing DeepThought-8B: Transparent reasoning model built on LLaMA-3.1 with... test-time compute scaling. - JSON-structured thought chains & controllable inference paths. - ~16GB VRAM, competitive w/ 70B models. - Open model weights, and inference scripts.show more

Video generation is powerful but too slow for real-world... robotic tasks. How can we enable both video and action generation while ensuring real-time policy inference? Check out our work on the Unified Video Action Model (UVA) to find out! (1/7)show more

The team at Runway is pushing the frontier of... what's possible in world models. We're proud to work with them to power real-time inference for Runway Characters.show more

Excited to share our NeurIPS 2024 Oral, Convolutional Differentiable... Logic Gate Networks, leading to a range of inference efficiency records, including inference in only 4 nanoseconds 🏎️. We reduce model sizes by factors of 29x-61x over the SOTA. Paper:show more

Distributed Inference, Now in Hybrid. Try it now:

diffusion policy trained on 26 minutes of data (80... demos, 20 seconds each) using DDIM noise scheduler for inference with 10 denoising stepsshow more

Fireworks blazing fast LLM inference is now available on... Poe! Today, we’re excited to bring the power of the new Mistral 7B Instruct model to the Poe powered by our lightning-fast Fireworks inference platform. You can now have conversations with the Mistral 7B bot and even build your own bots on top of it. Try it here:show more

The Inference - Episode 7 👾 Our second “The... Inference” livestream of 2026 is coming! 🗓️ Friday (09.01.26) 🕐 10 AM UTC 📍 On X & YouTube 🎙️ With Josh Goodbody, Luis 👾, 🔥 Special guest: WachAI CTO, {{joey}} Questions? Drop them below - and set a reminder👇show more

For fellows. What is your inference from this LV angiogram?

The fastest search meets the fastest inference! Exa 🤝 Cerebras

Animals segmentation & tracking using Ultralytics YOLO26 🫎 How... I created this demo: - Data annotation using Ultralytics Platform. - YOLO26 model training using Platform. - Downloaded the trained model. - Run Inference on video files locally. #animal #ai #MachineLearningshow more

1/ CyAI is live. Requests are now being served... through Cysic’s verifiable inference network. In July, we described what verifiable inference should look like. This is it running. Check it out:show more