Jev is now live on the Venice API. Watch it classify 24,000 Hacker News posts into 12 categories in about 2 minutes

That is insane.

saw the venice news earlier so clean

awesome 🙌

Cost for this run?

24000 posts in 2 minutes is about 200 a second. at that rate, relabeling the whole corpus costs about what labeling it once did, so you can throw out the 12 categories and rerun.

You are crushing it as always @sabrinaesaquino . Any suggestions on how to use this model at its best?

Nice you did some performance benchmarking on Venice with specific LLms? Want to understand if there is some latency because of the privacy layer

Not for this exact comparison, but the privacy layer itself adds negligible latency for us. In practice, performance mostly comes down to the model and available capacity

You forgot to highlight how much it costs to do this

Doing this but with code review @blinkdotreview Isn’t it great

24000 posts in 2 minutes is ~200 classifications a second. at that throughput jev stops being a routing call and becomes something you run on every event