正在加载视频...
视频加载失败
Introducing *Switch*: a *harness framework* for *decision models* like Jev I looked for this the other day and was surprised to not find anything like it You obviously cannot use a decision model like an LLM in a coding agent harness like Pi, because Jev cannot generate tool calls... show more
106,096 次观看 • 5 天前 •via X (Twitter)
21 条评论

@pidotdev Lol, and we once again came back to fsm's, eternally

@pidotdev letting the judge page through 200 pages instead of dumping every page image into context is what sold me. evals get expensive fast that way

@pidotdev symbolic ai + continuous learning + ensemble of hundreds of jevs decision coordinated by astra has to be something right?

@pidotdev Thanks for the code & inspiration - I'm working exactly on this one + on pi for refactoring & tcr tasks.

@pidotdev So the harness lists the valid tool calls and Jev just picks one?

@pidotdev Looks cool, I will try to integrate with Gust ( orchestrator for dynamic DAGs

@pidotdev turning open-ended generation into a discrete routing problem makes running small specialized models way more realistic

@pidotdev Yep, needed!

@pidotdev @bot could any of our projects benefit from this? I feel we are increasingly implementing Jev into our apps and tools

@pidotdev Well done, will be trying out. I think this will go places. Would be nice to start a list of use-cases where decision objectively outperforms reasoning model

@pidotdev This is awesome @marco_derossi @zenzhenyu

@pidotdev wow very 🆒

@pidotdev solid take brochacho

@pidotdev Nice idea, picking from a finite set of generated actions seems much easier to validate than free-form tool calls. How do you keep that set small when the state gets big, like a late-game chess position?

@pidotdev A harness for a model that can't write text is a fun constraint. What does the tool loop look like when the output is just a decision?

@pidotdev Generated menus still omit legal moves

@pidotdev Congrats on putting Switch out there! Exploring new ways to make decision models useful is worth pursuing. Keep building! If you'd like more AI builders to hear about it through creators, happy to lend a hand

@pidotdev That matches TypeSafe's own dashboard. Jev landed at 67.8% agreement with the reference against 74.1% for Opus 5 in workflow mode, so I'd treat it as the bootstrap and keep the production judge elsewhere.

@pidotdev nice. the interesting part of a decision-model harness is that the options are code, not prose. do you have a benchmark?

@pidotdev Jev picking from a generated menu instead of inventing tool calls feels like the right inversion. The 200-page judge example sold me: paging a haystack beats dumping it all into context. Where does the menu break down first, a large choice set or a messy state?

@pidotdev for the 200-page judge, what generates the choices at each step: a flat page list, search hits, or a table of contents? and when jev picks wrong, how do you tell a bad pick from the right page never being offered?
