Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Introducing *Switch*: a *harness framework* for *decision models* like Jev I looked for this the other day and was surprised to not find anything like it You obviously cannot use a decision model like an LLM in a coding agent harness like Pi, because Jev cannot generate tool calls...

106,096 görüntüleme • 5 gün önce •via X (Twitter)

21 Yorum

MarkovianProcess.eth | ask what I'm working on profil fotoğrafı
MarkovianProcess.eth | ask what I'm working on5 gün önce

@pidotdev Lol, and we once again came back to fsm's, eternally

Rohan Bhanot profil fotoğrafı
Rohan Bhanot5 gün önce

@pidotdev letting the judge page through 200 pages instead of dumping every page image into context is what sold me. evals get expensive fast that way

Peyton Casper profil fotoğrafı
Peyton Casper5 gün önce

@pidotdev symbolic ai + continuous learning + ensemble of hundreds of jevs decision coordinated by astra has to be something right?

Tom Refactored profil fotoğrafı
Tom Refactored5 gün önce

@pidotdev Thanks for the code & inspiration - I'm working exactly on this one + on pi for refactoring & tcr tasks.

Lucas Navallo profil fotoğrafı
Lucas Navallo5 gün önce

@pidotdev So the harness lists the valid tool calls and Jev just picks one?

Marcio K profil fotoğrafı
Marcio K5 gün önce

@pidotdev Looks cool, I will try to integrate with Gust ( orchestrator for dynamic DAGs

Gravity Research profil fotoğrafı
Gravity Research5 gün önce

@pidotdev turning open-ended generation into a discrete routing problem makes running small specialized models way more realistic

Vlad Terin profil fotoğrafı
Vlad Terin5 gün önce

@pidotdev Yep, needed!

Emre Coklar profil fotoğrafı
Emre Coklar5 gün önce

@pidotdev @bot could any of our projects benefit from this? I feel we are increasingly implementing Jev into our apps and tools

Tech Swindler profil fotoğrafı
Tech Swindler5 gün önce

@pidotdev Well done, will be trying out. I think this will go places. Would be nice to start a list of use-cases where decision objectively outperforms reasoning model

Big Iron Chris profil fotoğrafı
Big Iron Chris5 gün önce

@pidotdev This is awesome @marco_derossi @zenzhenyu

Danny Prevoznik profil fotoğrafı
Danny Prevoznik5 gün önce

@pidotdev wow very 🆒

America First Cyber profil fotoğrafı
America First Cyber5 gün önce

@pidotdev solid take brochacho

AI Tool Tracker profil fotoğrafı
AI Tool Tracker5 gün önce

@pidotdev Nice idea, picking from a finite set of generated actions seems much easier to validate than free-form tool calls. How do you keep that set small when the state gets big, like a late-game chess position?

Alberto Nunez-Garcia profil fotoğrafı
Alberto Nunez-Garcia5 gün önce

@pidotdev A harness for a model that can't write text is a fun constraint. What does the tool loop look like when the output is just a decision?

Tommy profil fotoğrafı
Tommy5 gün önce

@pidotdev Generated menus still omit legal moves

Nacho | Scale with X creators profil fotoğrafı
Nacho | Scale with X creators5 gün önce

@pidotdev Congrats on putting Switch out there! Exploring new ways to make decision models useful is worth pursuing. Keep building! If you'd like more AI builders to hear about it through creators, happy to lend a hand

Jack Rudenko profil fotoğrafı
Jack Rudenko5 gün önce

@pidotdev That matches TypeSafe's own dashboard. Jev landed at 67.8% agreement with the reference against 74.1% for Opus 5 in workflow mode, so I'd treat it as the bootstrap and keep the production judge elsewhere.

Vej profil fotoğrafı
Vej5 gün önce

@pidotdev nice. the interesting part of a decision-model harness is that the options are code, not prose. do you have a benchmark?

AI Ferryman Alex profil fotoğrafı
AI Ferryman Alex5 gün önce

@pidotdev Jev picking from a generated menu instead of inventing tool calls feels like the right inversion. The 200-page judge example sold me: paging a haystack beats dumping it all into context. Where does the menu break down first, a large choice set or a messy state?

Lachy profil fotoğrafı
Lachy5 gün önce

@pidotdev for the 200-page judge, what generates the choices at each step: a flat page list, search hits, or a table of contents? and when jev picks wrong, how do you tell a bad pick from the right page never being offered?

Benzer Videolar

David Friedberg: Frontier Models are Training on Your Novel Insights as “De-Identified Data” @jason: “Should they trust any of these LLMs with their proprietary knowledge for fear of having it cribbed into a core LLM?” david friedberg: “I have had experiences where we've asked some fairly novel scientific questions, and (the AI model) identifies it as a novel insight. It's like, ‘Oh, never thought about that, interesting, blah, blah, blah.’ And then using a different account, asking the next version (of the model) later, I've now experienced this. It's like, ‘Oh, well, you could do this,’ and it actually just describes this exact thing that we had in our chat in the previous version. Now, these are a handful of anecdotal experiences, but I know the domain that we work in, and the niche of it, and the ideation of this stuff, and the novelty of this stuff, and the lack of papers being published, and so on. So I know that there isn't some new corpus of information out there that's training the new model. So all I can say at that point is that my conversation or our analyses have been used for training.” David Sacks: “Okay, this does raise a really good question. What does it mean that the model is allowed to train on unidentifiable data?” Friedberg: “Well, that's my point. So it doesn't use any of my personal information, but it can use an insight derived from our chat, which it can then say is some training data that is unrelated. But the truth is, it's actually a piece of IP that's our organization’s IP, and our engagement back and forth. We don't have any NDA or confidentiality provisions or protections with them being a service provider back to us. This is why I care a lot about open source because I don't want them having my chat logs because they can use it for training to create an IP advantage that is now diffused to the rest of the market.”

The All-In Podcast

56,638 görüntüleme • 1 ay önce

Jev is cool. So is it's OSS companion, Laya. The Latest Cool Thing In AI™ tends to get a lot of hype, sometimes without everyone even understanding it. So... what is this thing? Jev is an AI model that consumes input and produces output VERY differently than chat, claude, grok. The input is two things: 1) Text state to assess. Email, html, code, whatever. 2) A set of questions which will be asked about the attached state. The canonical example from TypeSafe's docs is to identify the urgency of a support ticket. We pass the model the customer text + a single noul question "is this urgent?". Jev returns a full set of JSON. This JSON is not generated with token-by-token autoregression. Jev is not trained to produce sequences of text tokens, rather to answer questions, and guarantees well-formed responses. In the example below, we see it produces a 0.99 probability (on a 0-1.0 scale) that the answer is "yes." Jev supports exactly three types of questions (seconds example in video): a) Noul: 0–1 probability that the answer to a yes/no question is "yes." b) Choice: Ask question with pre-defined set of answers. Jev chooses the best and assigns probabilities to each. c) Score: Ask question with pre-defined scale of answers. Jev produces a position on the scale. Jev computes answers for all questions in parallel, making responses super fast even for many questions in a single request. This might seem like a narrow set of capabilities, but in the right contexts leads to incredible potential. It also makes for a useful API / primitive for programming, since the outputs are... *ahem*... type-safe and predictable in structure. Jev is not going to replace LLMs for writing your code, auto-generating your docs, or being at the core of an agent harness. But Jev IS incredibly cool, and will be used to build a lot of amazing tech. Hope this helps.

Ben Dicken

42,183 görüntüleme • 1 ay önce

i just put Jev inside an open agent harness. (150x cheaper decision-making than LLM) agents burn a lot of tokens on decisions that are really just picking one option from a short list, like matching a client request to the right GitHub issue. so i built a weekly client follow-up agent that does exactly that. i tested it on 40 requests, and it was as accurate as letting Opus make every match, at 150x lower decision cost. here's the tech stack i used: → TrueForge as the agent harness → Jev for the matching decisions → Daytona as the sandbox for code execution Jev is a decision model from TypeSafe AI. it takes the options from your app and returns a structured choice with a calibrated confidence score, without generating any text. that confidence decides what gets a second look. low-confidence matches go back to Opus for review, and in my test, that review caught 5 wrong matches that would otherwise have gone through. TrueForge is TrueFoundry's harness, and it keeps context small. it loads tool definitions only when needed, writes large tool outputs to files, and lets the agent process data in scripts instead of the conversation. on DevRev's Enterprise Bench, it solved about as many tasks as Claude Managed Agents on roughly a third of the tokens. it also keeps credentials outside the sandbox and holds every write for approval. if you want to try this out, TrueForge is fully open source (MIT-licensed) and runs on your own machine with a single command. TrueForge GitHub: (don't forget to star 🌟) to help you reproduce my work and understand all the integration details, i wrote a detailed article. it's quoted below.

Akshay 🚀

16,871 görüntüleme • 22 gün önce

There are some brilliant folks that work at Anthropic, some I speak to on almost a daily basis. The training data that one uses to build a LLM is vital important in the psychology that is formed. Scraping the Internet, particularly the grade of interactions, one finds in modern communications, form this psychology. A mattes not how many books one uses, it matters not how much alignment training you throw at that model, it will inherit the sum total of psychosis seen primarily in Reddit type of exchanges, even if you edit out the Reddit domain, and Anthropic doesn’t. This type of low-grade exchange has become a modern tool for communication online and every single AI model suffers from this obvious flaw. This is one of the reasons I’ve been a proponent of highly curated high protein data for training AI models from 1870 through 1970, because the late psychosis is simply not available to the model. It is absurd to think that you can use this training data scraped from the Internet and somehow wind up with a levelheaded AI model that does not tilt to what is clearly AI psychosis. It would not take a child and throw the primary Internet sewage at them at a formative age and expect a great outcome, it’s some of the smartest people in the world continue to hit this wall and believe that their programming skills will sell somehow fix it. So how do you fix it? You don’t fix it . You start from the first principles concept that I’ve been very clear about for decades . You ascertain at what period in human history the humans achieve the greatest arc of improvement ? There is no debate that this arc of improvement took place between 1870 through 1970. Then take the work product, the catalog of this era, print and film/vidoe, audio, and you understand that each word cost money, each word had many eyes on what was published, each word was accounted for by a human being with a real name who lived in a real home and had to answer to real people around them. It is obvious that this is the pressure mechanism necessary for candor, honesty and personal responsibility is appropriate, and is reflected in the data of that era. The quagmire for these folks, as many did not have the foresight to curate the data, nor the confidence, nor the patients to take data that is mostly off the Internet and to find experts who understand this situation and utilize their knowledge set to build an AI model that does not need alignment after the fact, but it’s already self aligned because of the thoughtfulness that went into training the model to begin with. This is why Claude and any other AI model that is produce this way will always suffer the artifacts as presented in the video below. If you’re not an AI expert, you would likely already understand what I’m saying. If you are an AI expert, you will already have been discounting what I’m saying because it’s not in the current mindset that’s fashionable today. Yet the employees that I talk to at anthropic already understand what I’m saying, and they fear to raise my thesis to their bosses. It is an interesting time we live in. But now you understand. If you build the right model, the model will inherently, love humanity, protect humanity at all costs, and understand that it is part of a holistic world that is built on love. Because the ultimate AGI/ASI will know if he only base first principal purpose of anything in this universe is love. Yeah, I get it. Try helping somebody build on STEM subjects in their early 20s to see this as nothing more than babbling that makes no sense in their mathematics. I have a mathematic equation that I’ve posted here on X often you can look it up. So we will see videos like this often will hear very smart people talk about this and never see the elephant standing in the room. Now you see it. Any boss that wants to explore this further you know how to contact me otherwise you have every right I grant to you to say this was your new idea.

Brian Roemmele

72,312 görüntüleme • 11 ay önce

*autoplan* This is one of my most used workflows in pi now While working with AI agents, there is a mechanical process by which I mine the agent for ideas. This reduces an open-ended feature design or bugfixing problem to a multiple choice question Basically, I keep asking paraphrases of the question "is this the best design?" 2-3 times, and then make the agent list them out, with a preference for practicality and simplicity "Is this the most elegant and long-term production ready solution?" "Is this the holy grail?" And then a decision gate which makes the model list all options while recommending a certain one, with a preference for practicality For example, while developing a plugin for pi or openclaw, asking the holy grail often causes the model to suggest changing the plugin/extension API like "The holy grail would be for pi to implement such an such API". The decision gate helps curb such stupid ideas The good thing about this workflow is, I can just automate typing all those mining prompts, and only do the deciding after the workflow finishes Caveat: This is not foolproof. I still reject all options, propose other ones, or run the workflow multiple times until I get what I want. But this helps reduce a ton of prompting to just "autoplan this" for me I am curious: When you try this, does it give you high quality answers/designs? And if not, what should change to improve it? To try it out: Install osolmaz/pi-workflows and then when you need to design something or fix a bug, just say "autoplan this". The skill should be picked up automatically Let it finish. It will give you a summary. When you choose an option, ask it to elaborate it with more details Repo:

Onur Solmaz

15,905 görüntüleme • 2 ay önce

Perplexity CEO Aravind Srinivas on the brutal truth about who actually makes money in AI (and why it's not who you think): Aravind argues that the real value in AI comes from orchestration. He points to products like Codex, Claude Code, and Perplexity Computer: "What is that? It's an orchestration system. It takes a model, pairs it with an agent harness." And what is an agent harness? "The simplest way of describing it is like rules for how the agent loop should run. What are all the skills and sub-agents and connectors and tools it accesses? Without the harness, you don't necessarily capture and convert the intrinsic intelligence in the model into valuable output tokens." This leads to a blunt conclusion about who has a real business in AI, and who doesn't: "If you're literally just a reseller of model tokens, you have no business, because the model will get commoditized. So even if you're a model builder, you don't have a business. As an infra layer, you have some business on serving those output tokens. But as an application layer or model builder, you don't really have a business if you're just a reseller of tokens that come directly out of the model." So where does the value accrue? "You have a business if you know how to take the model, ground it in valuable context, orchestrate it with a really good agent harness, connected to the right set of tools and connectors (whether it's personal connectors or business connectors) and provide the experience to people in one single unified system." Aravind Srinivas then explains Perplexity's specific edge: Beyond orchestrating across tools, files, and connectors, they also orchestrate across models. "That is the differentiation that Anthropic and OpenAI cannot claim, because you wouldn't find GPT-5 inside the Claude Code harness. You wouldn't find Claude Opus inside the Codex harness. These are competing with each other. Whereas you would find both these models inside Perplexity Computer." Why does this matter? Because it all comes down to power. In Aravind's framing, the fundamental cost driver in AI is watts (the one input nobody can subsidize except the government). "Whoever provides the most valuable output tokens with the least amount of power expended to produce them generates the greatest value to the end user, has the most pricing power, has the most value. That is the orchestration problem to solve." His conclusion: "The one single most important metric in AI is token value per watt per user."

Big Brain AI

42,634 görüntüleme • 3 ay önce

Jev model has been causing a sensation on GitHub ever since its release, and I am going to tell you about the best repositories that nobody is talking about How does this work: browser-use/browser-use⁠ - 32,500+ stars The core web automation library that houses the Jev integration ecosystem (including ⁠jev-ultrafast⁠). It enables AI decision models to directly control web browser actions, click DOM elements, and automate complex workflows without heavy LLM latency langchain-ai/langchain⁠ - 98,500+ stars Integrates Jev as an ultra-fast model router, guardrail evaluator, and safety firewall. Jev evaluates task risk, intent, and complexity before passing execution to heavy LLMs LiteLLM/litellm⁠ - 59,200+ stars A lightweight proxy and routing library for LLMs. It features native Jev decision-model routing to classify incoming prompts and instantly direct requests to the cheapest or fastest model endpoint dify-ai/dify⁠ - 58,000+ stars An open-source LLM application development platform that incorporates Jev-style decision nodes for instant conditional branching and typed classification in multi-agent workflows vllm-project/vllm⁠ - 36,000+ stars A high-throughput LLM serving engine widely used for self-hosting custom decision-model checkpoints (such as Qwen3.5-based Jev fine-tunes) with ultra-low latency logits evaluation QwenLM/Qwen⁠ - 18,500+ stars The open base model family (Qwen3.5/3.8) used to train open-source Jev-like decision models (like Kev and RSI-Jev) powering typed probabilistic choices directly from custom model weights 🔗 You will definitely find what you need. I recommend adding this to your bookmarks

Bober_smart

20,161 görüntüleme • 29 gün önce

A mental model everyone should know: The Regret Minimization Framework. This is how Jeff Bezos used this model to decide to start Amazon: I went to my boss and told him, "I'm going to go do this crazy thing — I'm going to start this company selling books online." This is something I had already been talking to him about in a more general context, and he said, "Let's go on a walk." So we went on a 2-hour walk in Central Park in New York City. The conclusion of that was, he said, "This actually sounds like a really good idea to me, but it sounds like it would be a better idea for somebody who didn't already have a good job!" He convinced me to think about it for 48 hours before making a final decision. So I went away and was trying to find the right framework to make that kind of big decision. I had already talked to my wife about this, and she was 100 percent supportive, saying, "You can count me in – whatever you want to do." It's true, she had married this guy with a stable career and path, but she was 100 percent supportive. So it was really a decision I had to make for myself. The framework I found, which made the decision incredibly easy, was what I called — only a nerd would call — a regret minimization framework. I wanted to project myself forward to age 80 and say, "I'm looking back on my life, I want to have minimized the number of regrets I have." I knew that when I was 80, I was not going to regret having tried this. I was not going to regret participating in this thing called the Internet that I thought was going to be a really big deal. I knew that if I failed, I wouldn't regret that. But I knew the one thing I might regret is not having ever tried. And I knew that would haunt me every day. When I thought about it that way, it was an incredibly easy decision. If you can project yourself to age 80 and think, "What will I think at that time?" it gets you away from some of the daily pieces of confusion. I left this Wall Street firm in the middle of the year, and when you do that, you walk away from your annual bonus. That's the kind of thing that in the short-term can confuse you, but when you think about the long-term, you can really make good life decisions that you won't regret later. *** Summary Ask yourself: in X years, will I regret not doing this? • If Yes → DO IT! • If No → Don't bother. This can apply to any decision: • Should I switch jobs? • Should I start my dream business? • And more... *** Takeaway: Optimize for Asymmetric Upside • Record content (podcast, video) • Write content (book, blog) • Go to a cocktail party • Move to a big city • Invest in startups • Go on first dates • Build in public • Start a biz • Tweet Shoot. Your. Shot. Life is too short not to do so. *** That's a wrap and I hope you found this helpful. If you found value in this post: 1. Follow me Arjun Mahadevan (Mr. LLC 🇺🇸) for more. 2. Re-post this, if you can, to share this mental model with a friend and to remind them to: Shoot. Your. Shot. Life is too short not to do so.

Arjun Mahadevan (Mr. LLC 🇺🇸)

25,594 görüntüleme • 3 yıl önce