Loading video...
Video Failed to Load
Introducing Atomic Local AI Server ⚡ Choose inference engine. Run local models. Connect your agents. • Multiple engines: llama.cpp, vLLM, SGLang, TensorRT, MLX, Prism and more. • Local AI server with an OpenAI-compatible API. • One-command engine installation matched to your GPU, CPU or NPU. • Pull from 3M... show more
40,325 views • 1 day ago •via X (Twitter)
21 Comments
Nice CLI. All engines in one place!
Most versatile tool for your inference factory imo
Mega release! Nearly any inference engine is supported now!
One-command engine install matched to my hardware, congrats 🔥
I love the cli
🔥🔥 when is strata coming?
working on it, stay tuned)
Nice, can we have a gui though? TUI's are annoying to use
Is this different than Atomic Chat? is there an app in the Apple store? Confused. You video runs too fast🥵
按硬件匹配引擎这一步挺吸引人。本地跑模型，能少花点时间处理安装和兼容，才更容易真的用起来。
An OpenAI-compatible endpoint on the LAN is an open door if it binds to 0.0.0.0 without a key. Does it listen on localhost only by default, or require an API key for remote clients?
For multi-engine setups the cleanest guardrail is a one-line wrapper: set ATOMIC_ENGINE=llama.cpp before the serve call so every request pins the same backend and you never get silent answer drift between llama.cpp and vLLM.
How does it differ from LM Studio?
switching engines on the same model is where i'd worry. llama.cpp and vLLM can give slightly different answers, so logging which engine ran each call would save a lot of debugging.
能随时切换推理引擎挺实用，关键看切过去后 Agent 的工具调用稳不稳。
You know what, that is clever!
Engine soup? Without q8_0 KV-cache, you’re just leaking VRAM. We cut footprint 43% on dual GPUs.
Run local models free
Letting people swap between llama.cpp, vLLM and MLX behind one OpenAI compatible API is handy for comparing real throughput on their own hardware. Does it keep the same model files across engines or does each one need its own format?
you will generally need different model files for each engine. you can compare the same model across engines, but each needs a compatible format
llama.cpp, vLLM, SGLang, TensorRT and MLX behind one OpenAI-compatible endpoint is more than I expected from a local server.
