Here is how to deploy and serve any LLM on HF with a single command in less than 3 minutes with llama.cpp $ bash -c "$(curl -s

More info

Would be cool to have some kind of auth system, where you can have it check against a list of auth tokens before serving the request.

thanks for the demo, nice done

Does server binary supports LLaVA/BakLLaVA models ?

@memdotai mem it

@ggerganov Saved! Here's the compiled thread: 🪄 AI-generated summary: "This thread provides instructions on how to deploy and serve any LLM on HF with a single command in less than 3 minutes using llama.cpp. More information can be found at the...

Time to fill up my runpod credits. 😁 It would be lovely if supports llava.

They should have made you CEO of OpenAI 😅😇💪