Video wird geladen...
Video konnte nicht geladen werden
State → local decision model → action Check out Liquid AI's d1 decision model playing Lmmy Run, all on its own 🦖👾 Served 100% on-device, making decisions in a fraction of a second
20,131 Aufrufe • vor 15 Tagen •via X (Twitter)
5 Kommentare

Stack used for the demo: Model server - @lmstudio Model - d1 by @liquidai Inference runtime - @llama_cpp Lmmy Run is based on Check out the docs for how to do it yourself!

@liquidai Cool to see d1 on llama.cpp. makes me wanna revise d1 again perhaps on a different framework.

@liquidai On Windows when something is animated, like even the answer is streaming in the chat, the tokens per second drop by substantial amount. Can you fix it? I use Radeon AI PRO R9700 as a second dedicated GPU for inference, but that doesn't matter.

@liquidai running a decision model 100% on-device at sub-second latency — that's the part everyone said was still years out. local models are catching up fast

@liquidai Leave me 🦖 out of your 🦀

