正在加载视频...
视频加载失败
After a small patch to mlx-lm, I have tool parameter streaming \o/
21,687 次观看 • 6 个月前 •via X (Twitter)
12 条评论

Please share, it drives me insane that with pi I can see the tool call streaming with gguf models through llama.cpp while none of the mlx engines do so and I wait 4 min to get a html page write without getting a feel of tps.

Finally got the time to test it, I ran into an issue with qwen3.6-35b-ud-mlx_3bit where it added an ? at the end of the file name. I had gpt 5-5 check it out and it made these changes Here is also the thread to showcase its findings

I was trying also some json parsing but I don't trust that yet, this is the write tool call failure, the original gist, somehow you cannot delete a new revision of a gist :(

@badlogicgames Is it local model running on ur mac ? Can you share specs of model and ur machine

@awnihannun That’s nice, it’s so annoying to wait without the feedback

@awnihannun We got for local processing chains for MLX Swift. Map-reduce, stuff, and adaptive strategies for local LLM inference on Apple Silicon.

Silly request. What is your terminal font it looks so shar and readable.

@braza_faouzi I think this one is Berkeley Mono.

Merci :-)

Any particular reason why you chose mlx over llama.cpp?

How does your Pi invoke mlx-lm? I've been using omlx, which wraps mlx-lm and mlx-vlm. But maybe there's a simpler way?

@badlogicgames what are your essential pi plugins?
