DeepSeek V4.1 Flash 1017 on TensorFold ⚡️ - 16 concurrent session - Each with full native 1M context - Non-greedy, thinking on. OFFICIAL WEIGHTS. Recipe dropping very soon!

@volatilemarkts 4x DGX Sparks SUPPORTS SWITCHLESS

@volatilemarkts Out now

@ashxhart @volatilemarkts Can I change the setting to TP3 and use it on 3 Spark?

@ashxhart @volatilemarkts Initial release supports only 4

@ashxhart @volatilemarkts Okay. I'll wait. Thanks as always for sharing your awesome work!

@ashxhart @volatilemarkts running your SGLang recipe on 4 Sparks in a switchless ring. stock DeepSeek V4.1 Flash here: 70.0 tok/s on code single stream, 72.9 total at 4 streams (26.2 each). 16 sessions at full 1M is a different league. will rerun my prompts on TensorFold when the recipe drops

@ashxhart @volatilemarkts Oh man, this is most definitely worth trying out. Can't wait for the recipe!

@ashxhart @volatilemarkts Nice achievement 👍

@ashxhart @volatilemarkts Did you get the 4 spark @MiaAI_lab

@ashxhart @volatilemarkts Yes a while ago :)

@ashxhart @volatilemarkts nice Im happy, Hope it made a difference to your workflow

@ashxhart @volatilemarkts how about single stream peak?

@ashxhart @volatilemarkts Publish sustained throughput latency and memory under full load

@ashxhart @volatilemarkts Ablit recipe too please 🙏

@ashxhart @volatilemarkts Nice, now I want two extra Sparks. :(

@ashxhart @volatilemarkts running 1M context natively on a 16gb mac mini via ollama sounds wild, how does tensorfold handle KV cache paging for those concurrent sessions?

@ashxhart @volatilemarkts 16 sessions each at full 1m context on one box is wild

@ashxhart @volatilemarkts Sayy whaa 2x DGX?

@ashxhart @volatilemarkts Nah... this is 4

@ashxhart @volatilemarkts Dang, what's the best quant on TP=3? 4bit?

@ashxhart @volatilemarkts OFFICIAL WEIGHTS !!!!

@ashxhart @volatilemarkts Dang, looking forward to it! 🤞🏼🙌🏼

@ashxhart @volatilemarkts have you tried minimax yet?

@ashxhart @volatilemarkts Yes. I skipped it.

@ashxhart @volatilemarkts

@ashxhart @volatilemarkts ok where is the kv cache even living

@ashxhart @volatilemarkts Offical weights and 16 concurrent sessions is crazy 🔥

@ashxhart @volatilemarkts The only way to do it is crazy.

@ashxhart @volatilemarkts If you have time, can you please update the 2x DGX Spark version too please ♥️

@ashxhart @volatilemarkts running 16 concurrent sessions at 1m context on flash weights sounds wild for local infra

@ashxhart @volatilemarkts I don’t see a graphical front end for TensorFold on Spark. Only Mac

@ashxhart @volatilemarkts I assume you need 4×GB10 for 16 concurrent?

@ashxhart @volatilemarkts With official weights, yes, but it will support 3 sparks later on too

@ashxhart @volatilemarkts 16 sessions with full 1M context running simultaneously? That memory bandwidth must be absolutely insane.

@ashxhart @volatilemarkts 16 concurrent sessions, each with full 1M context — that's the kind of spec that makes 'just run it locally' sound almost reasonable.

@ashxhart @volatilemarkts the number I’m watching for in the recipe: tokens/sec per session with all 16 contexts filled. A 1M window tells us capacity; sustained decoding tells us how responsive those sessions are

@ashxhart @volatilemarkts Should be interesting to see how this compares single-stream compared to my ground-up Rust stack.

@ashxhart @volatilemarkts 1M上下文+16并发+非贪心，这配方一出直接封神。

@ashxhart @volatilemarkts prefil? sglang can do 5k+/s on TP4

@ashxhart @volatilemarkts I can only test switchless for now, don't have a switch. Switchless gets 4k prefill on 256k

@ashxhart @volatilemarkts I'm on TF 1.0.12 with DS 4 spark and I have images. But sad face TF does not cache images so reads are expeeeeensive.

@ashxhart @volatilemarkts But is it better than glm 5.3 flash? @MiaAI_lab

@ashxhart @volatilemarkts ❤️❤️❤️ will this be significantly better than the exl3 on 2 sparks ?!