🚀 Qwen 3.8 27b NVFP4 inside TensorFold native CLI on Windows 11 cooking on a NVIDIA RTX 5090. 🤯 Single-stream decode was ~220 tps for prose. Prefill was ~12,200 tps C=4 was over 500 tps decode. Will be in the next engine release 👌🏼

@Windows @nvidia Brother got 5090 and now it’s over 😭

@Windows @nvidia Your 5090 is already useful ⚡️

@Windows @nvidia does the 220 hold up on code or only on prose

@Windows @nvidia That 220 figure is prose. Code needs its own measurement.

@MiaAI_lab @Windows @nvidia Booooo! I need you working on MLX not NVFP4 😂

@MiaAI_lab @Windows @nvidia Just having a play with my 5090 for local development and trying to speed up TensorFold's development.

@MiaAI_lab @Windows @nvidia Yeah, I know. I don't blame you. If I had a 5090, I'd be playing with it, too. I don't think a 5090 will even fit in my PC.

@Windows @nvidia Dude, use tern for omp and thank me later

@Windows @nvidia Interesting I’ve not used this before

@Windows @nvidia Just try it out, game changer, I'm blow away and @stencil_labs are sitting on a beautiful piece of software, once this is released publicly it will defo jump to #1 harness in most lists (in mine it's already there for quite a while)

@Windows @nvidia @stencil_labs I’ll try it out later today thanks dude

@Windows @nvidia do you have rtx pro 6000 yet or is that on the to do list ?

@Windows @nvidia I'd love one dude, just can't afford one at the moment.

@Windows @nvidia 🤩😍😍 thanks for the 5090 effort

@Windows @nvidia V100, linux, go!

@Windows @nvidia I don't have one of them to try.

@Windows @nvidia dmed you

@Windows @nvidia Will you fine tune an open model around omarchy to make it extra awesome and useful out of the box?

I’d be interested in that dude, nothing decided yet though.

@Windows @nvidia I’ve been hoping for this since you mentioned x64, I’m running a small Gemma 4 on a 5060 Ti and the 12B ran at about half the speed when I tried it, so if NVFP4 works on the smaller cards I’d happily test it, since I’m testing the 12B this weekend anyway.

@Windows @nvidia i am doing 400 now'

@Windows @nvidia Stock Ninfer already gives me 250 tps sustained decode at a 256k context window with an NVFP4 quant on a 5090 tho

@Windows @nvidia Nice to see a native Windows path for TensorFold.

@Windows @nvidia What's max tps with 8 or 16 concurrency? Managing 1k tps c8 with VLLM currently

@Windows @nvidia This is the best speed/quality inference engine for Qwen 3.9 27B on a single 5090 BTW:

@Windows @nvidia I need that 😍

@Windows @nvidia prefill speed, how measured?

@Windows @nvidia By dumping 8000 tokens in to see how quickly it can compute them

@Windows @nvidia Holy cow!!

@Windows @nvidia 👀

@Windows @nvidia Try link it to you MAC for prefil please

@Windows @nvidia why not support two 5090s with tensor parallelism?

@Windows @nvidia How many context tokens?