Real-time Moondream inference using our new inference engine

Laika AI x Inference Labs Excited to announce our... partnership with Inference Labs We're providing our real-time RAG & AI model API to Inference Labs, powering their verification infrastructure with live blockchain data. Inference Labs delivers open-source, trustless verification for AI agent outputs, so you can trust what you see—without relying on centralized gatekeepers.show more

First steps for a specialized DeepSeek v4 Flash inference... engine focused on inference quality / stability at different quantizations, with networked API that is batching capable. This is the 2 bit quants model running on my M3 Max 128GB.show more

Introducing DeepThought-8B: Transparent reasoning model built on LLaMA-3.1 with... test-time compute scaling. - JSON-structured thought chains & controllable inference paths. - ~16GB VRAM, competitive w/ 70B models. - Open model weights, and inference scripts.show more

Batching for vision models is now available in Beta... with our latest MLX engine update 👾 The updated engine also brings major improvements to caching for faster inference overall. Turn on Developer Mode, choose the beta runtime channel, and select LM Studio MLX v1.8.1.show more

Video generation is powerful but too slow for real-world... robotic tasks. How can we enable both video and action generation while ensuring real-time policy inference? Check out our work on the Unified Video Action Model (UVA) to find out! (1/7)show more

The team at Runway is pushing the frontier of... what's possible in world models. We're proud to work with them to power real-time inference for Runway Characters.show more

Excited to share our NeurIPS 2024 Oral, Convolutional Differentiable... Logic Gate Networks, leading to a range of inference efficiency records, including inference in only 4 nanoseconds 🏎️. We reduce model sizes by factors of 29x-61x over the SOTA. Paper:show more

Distributed Inference, Now in Hybrid. Try it now:

Fireworks blazing fast LLM inference is now available on... Poe! Today, we’re excited to bring the power of the new Mistral 7B Instruct model to the Poe powered by our lightning-fast Fireworks inference platform. You can now have conversations with the Mistral 7B bot and even build your own bots on top of it. Try it here:show more

For fellows. What is your inference from this LV angiogram?

The Inference - Episode 7 👾 Our second “The... Inference” livestream of 2026 is coming! 🗓️ Friday (09.01.26) 🕐 10 AM UTC 📍 On X & YouTube 🎙️ With Josh Goodbody, Luis 👾, 🔥 Special guest: WachAI CTO, {{joey}} Questions? Drop them below - and set a reminder👇show more

Grok 2 mini is now 2x faster than it... was yesterday. In the last three days Lianmin Zheng and Saeed Maleki rewrote our inference stack from scratch using SGLang ( This has also allowed us to serve the big Grok 2 model, which requires multi-host inference, at a reasonable speed. Both models didn’t just get faster, but also slightly more accurate. Stay tuned for further speed improvements!show more

Move from experimentation to real AI outcomes with secure... data, modern networking, and inference at the edge. Turn your ambition into reality with HPE.show more

Real-time distributed inference monitoring is live on exo intern... home AI cluster It comes with out of the box support for PrometheusMonitoring metrics and Grafana dashboards Helps diagnose performance bottlenecks in your exo clustershow more

We’re amazed: AMD now beats Nvidia in inference for... Image-to-Video generation. 20% faster and 35% cheaper when using our Higgsfield DoP model. AMD MI300X also has a very easy setup. Details and tests are below. 🧩 1/8show more

[ Comparison Video ] Truth & Inference Series Optimization... ▸ Photographer A-Tier Costume – "D.M." Source • •show more

PRIMA.CPP Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday... Home Clustersshow more

Virtuals Protocol is integrating Venice to power AI agent... building with private, uncensored inference, available to anyone, anywhere on Base. Venice brings best-in-class privacy-first inference. Virtuals EconomyOS brings the full agent infrastructure stack: wallets, identity, payments, commerce, funding rails, and launch infrastructure. We are deploying up to $400,000 in private inference credits so anyone can move from idea to working agent without compute or backend complexity getting in the way. Start your AI journey. Inference and infra are on us. Program details soon.show more

First came pre-training scaling; then came inference-time scaling. Now... comes judge-time scaling. Despite progress in AI through scaled inference-time compute, AI remains unreliable in open-ended, non-verifiable domains. The key limitation is not generation—it is evaluation. Therefore, the next big leap for AI comes from better judging. In service of this future, today we release Verdict, a library for scaling judge-time compute.show more

Have you missed us already? 😌 Last week we... were cooking so hard that we couldn’t make it to our traditional weekly Episodes of ‘The Inference’. But this time we’re back, and with a special guest! Tune in for Episode 4 of ‘The Inference’, featuring @Ungl1tched, Warden’s CTO, sharing key insights about the Studio and much more. 🚀 See you tomorrow at 1 PM UTC - with Josh Goodbody Luis 👾show more