正在加载视频...

视频加载失败

After a small patch to mlx-lm, I have tool parameter streaming \o/

21,687 次观看 • 6 个月前 •via X (Twitter)

12 条评论

ichim-david 的头像
ichim-david6 个月前

Please share, it drives me insane that with pi I can see the tool call streaming with gguf models through llama.cpp while none of the mlx engines do so and I wait 4 min to get a html page write without getting a feel of tps.

ichim-david 的头像
ichim-david6 个月前

Finally got the time to test it, I ran into an issue with qwen3.6-35b-ud-mlx_3bit where it added an ? at the end of the file name. I had gpt 5-5 check it out and it made these changes Here is also the thread to showcase its findings

ichim-david 的头像
ichim-david6 个月前

I was trying also some json parsing but I don't trust that yet, this is the write tool call failure, the original gist, somehow you cannot delete a new revision of a gist :(

Nadeem Wani 🐻‍❄️ 的头像
Nadeem Wani 🐻‍❄️6 个月前

@badlogicgames Is it local model running on ur mac ? Can you share specs of model and ur machine

Spok 的头像
Spok6 个月前

@awnihannun That’s nice, it’s so annoying to wait without the feedback

Joel Nishanth 的头像
Joel Nishanth6 个月前

@awnihannun We got for local processing chains for MLX Swift. Map-reduce, stuff, and adaptive strategies for local LLM inference on Apple Silicon.

Faouzi Braza 的头像
Faouzi Braza6 个月前

Silly request. What is your terminal font it looks so shar and readable.

Armin Ronacher ⇌ 的头像
Armin Ronacher ⇌6 个月前

@braza_faouzi I think this one is Berkeley Mono.

Faouzi Braza 的头像
Faouzi Braza6 个月前

Merci :-)

Jian Su 的头像
Jian Su6 个月前

Any particular reason why you chose mlx over llama.cpp?

Rahim Nathwani 的头像
Rahim Nathwani6 个月前

How does your Pi invoke mlx-lm? I've been using omlx, which wraps mlx-lm and mlx-vlm. But maybe there's a simpler way?

Max Tesla 的头像
Max Tesla6 个月前

@badlogicgames what are your essential pi plugins?

相关视频