Onur Solmaz
@onusoz • 10,474 subscribers
Cooking something new. Ex @huggingface @openclaw
Shorts
Videos
Introducing *Switch*: a *harness framework* for *decision models* like Jev I looked for this the other day and was surprised to not find anything like it You obviously cannot use a decision model like an LLM in a coding agent harness like Pi, because Jev cannot generate tool calls But what if there was a framework that "generated the tool calls" for it programmatically, and Jev only had to pick out of that finite set of choices That is what switch does. You pick a "domain" to implement the decision model in, and it provides a core API to define your state, and the functions that will derive decision model questions out of that state. Chess is a domain, as well as being a rubric judge for evals It basically standardizes what I imagine a lot of people are already vibe-coding, to an abstract interface: calling a decision-model API through multiple steps, i.e. an agent loop over finite choices Because of the finiteness of the output of a decision model, you cannot have a general pure decision model harness. Instead, you need to implement specific ones that generate the choices programmatically (or have an LLM generate the choices, but I find the former more interesting) A primary use case I have for this is use as a judge in an eval. For example, At the end of a task run that involves a bunch of documents, most people currently follow this approach: extract text out of PDF/docx/md files, and dump them right into the context But what if the visual formatting of the output is a judging criterion? Then you have to dump an image of every page in the input state. But what if you have 200 pages, and your question does not need looking at all of them? What if Jev could navigate through the haystack and find the info it needs? I have noticed that Jev is not so good at chess nor being a Decision-Model-as-a-Judge. It sucks at chess, and it does not perform well enough to replace GPT 6.1 Sol as a judge in the domain I am working for But switch is not only for Jev! It is for any decision model including the ones supported by the pi-ai package So once you can generate enough data with a better model, you should be able to pick an open source one and fine-tune it on your use case to perform better! And the model should be trained in a harness like switch's to be especially performant and cost-efficient I wanted to get this out because I find the idea very exciting, at least for benchmarking. Jev is currently not there, but decision models obviously will be there at some point, and we can take them there! There are also a lot of potential gains from picking a specific domain, and creating a looping agent that does a job for 100x cheaper! Let me know if you have any ideas, I am quite keen to try this on new domains! Repo:
Onur Solmaz106,096 views • 5 days ago
I have performed a distillation attack on my PERSONAL TRAINER Since more than 2.5 years, I have recorded over 300 of our sessions, each up to 90 minutes long Why would I do that? Because he is a kinesthetic genius from the Caucasus who developed his own training doctrine He loves helping people become fitter and freer in their bodies, counting every rep and cheering sincerely when they unlock a new skill And now both he and I would like to share it with the rest of the world To begin with, I used Whisper+pyannotate to automatically transcribe all our sessions and used LLMs to extract all the knowledge He taught me over 100 movements and spoke about over 100 topics over the course of 2.5 years You can browse and read all of them here, for free: But again, why all this? 1.5 years ago, I wrote an article called "Our muscles will atrophy as we climb the Kardashev Scale" which went to the Hacker News front page It was sort of a meme, but it reflected a reality that I felt very deeply: Having a sedentary job which only requires me to use my brain is shortening my life and reducing my life quality by a lot Not only mine, but of hundreds of millions, soon billions of people who will spend hours just talking to AIs whole day for their careers It is also a fact that personal physical education and training is close to nonexistent for the average person There are billions of people who could live much healthier lives, just by exercising in the room they already have. But they just don't know how. And there is no system to teach them how, effectively at scale 1 on 1 personal training should be universal and free, with AI We must have reps and sets too cheap to meter An expert AI PT that sees and corrects your every move, that counts for you. Created by people that care, like us. Unlike a franchise gym that just wants to get your subscription and sell you protein powder Once everybody can taste good AI personal training for free, the demand for human personal trainers will explode and whole fitness industry will see unprecedented growth That is one of my life's goals, and I will make that happen one day. Today is the first step My trainer is Orkhan Mirza, a.k.a. Jashin Jashua here and on Instagram, and I am nerdonbars on Instagram The wiki and more are available here: Let me know what you think!
Onur Solmaz69,020 views • 2 months ago
acpx v0.4 ships Agentic Workflows, or as I like to call them "Agentic Graphs" It let's you create node-based workflows on top of ACP (Agent Client Protocol), to drive any coding agent (Codex, Claude Code, pi) through deterministic steps This let's you automate routine, mechanical legwork like triaging incoming PRs, bugs in error reporting, and so on... For example, OpenClaw receives 300~500 new PRs per day. A lot of them are low quality, but they still relate to real issues, so you have to address them somehow You need to: - extract the intent - cluster them based on intent - figure out if the proposed changes are legit, or whether they are slop local solutions, like trying to catch flies instead of drying out the swamp - if the PR is too low quality or the intent is not clear, close them - run AI review on them them and address any issues that come up - refactor them if the changes are half-baked - resolve conflicts - and so on... So that when the PR is presented to the attention of the maintainer, all the routine legwork is done and the only remaining thing is the decision to (a) merge, (b) give feedback to the PR author, or (c) take over the PR work yourself I wanted to build this feature since a couple months now, since Codex got so good. OpenAI models are now good at judging implementation quality, so I found myself repeating the same steps I wrote above over and over I also tried putting all this in a single prompt. But I believe there are workflows that should not be a single prompt, but a sequence of prompts in the same session That is because like humans, LLMs are prone to PRIMING. I claim that putting all steps in the same prompt at the beginning of the context will generally give suboptimal results, compared to revealing the intention to the model step by step Creating such a workflow also gives more OBSERVABILITY into the each step that an agent is supposed to take. Agent generates JSON at the end of each step, and that structured data can be used to monitor thousands of agents running at the same time in an easier way, on a dashboard Similar features have been introduced in e.g. n8n, langflow. But AFAIK they are not integrating ACP like the way I do I wanted to have a fresh approach, and to build an API that I can develop freely the way I want, so I created a new workflow API inside acpx The video is from the workflow run viewer, but that is not where you build the workflow. You build it by using the acpx flow typescript API. See examples/pr-triage in acpx repo Before building that, I started from a Markdown file with a Mermaid chart of the flow I had in mind. The Markdown file acts as a spec for the flow, and I have built the workflow through trial and error. I call this process "workflow tuning" I started working on acpx repo PRs one by one, tuning the flow, slowly scaling to more PRs. Finally, when I felt confident, I ran it in parallel over all external open PRs in the acpx repo. I believe it already saved me hours this week My next goal, if well received, is to set this up on a cloud agent so that it can process the 300~500 PRs the OpenClaw repo receives every day, in real time, as they come in I believe this will save all open source maintainers around the world countless hours and make it much easier to herd and absorb external contributions from everyone!
Onur Solmaz149,693 views • 7 months ago
To me, microduck looks like a droid. So I made it sound like one Microduck is open source and hackable. That means you can program it to do whatever you want! I trained a 1.3m parameter LLM to output droid like responses. It is running in the browser The model has learned to respond with an emotion, alongside English like Happy, Excited, Sad Then the emotion is fed into a Rust based synthesizer to generate R2D2 like sounds that are aligned with the emotion --- completely procedurally, no wav files, no copyright violation You can chat with it. You can also customize it, give it a name and set your own colors! All set in a spaceship hangar backdrop with other friendly ducks---simulated with realistic physics and microduck's actual walking and skating policies! All in the browser, with WASM!!! I can put literally the SAME code in the browser, and then compile it into an ARM binary and put it in the actual robot in real life 🤯 If you like this and want to see more of it, please like and share so that my boss puts me in the priority list for the first microduck batch 😎 All open source of course! Microquack! 🦆 Hugging Face Space:
Onur Solmaz34,473 views • 2 months ago
Struggling to create the level of quality of 3D modeling some of the OAI employees and early accessers are posting with GPT 6 Astra It is OK, but one-shot performance is not mind-blowing like some people were claiming Do the better runs use Blender MCP? Or sth else? I prompted it to put it on a website the first time. On the second time, I prompted it to use purely blender, in a new session. It still ended up putting it on a website, with similar performance Space: === prompt: use blender to create a realistic exposition of the internation space station, orbiting earth, in an as realistic way as possible should give a tour of the space station, show insides, cross sections, different modules, what everything does, including humans that work on stuff, maybe docking shuttles and stuff, its phases through time, in a nice and cinematic way, that also feels like sims or some game should eventually be put on a static hugging face space, a website in typescript inspiration should be one of those books from star wars the phantom menace where you could see to the inside of the spaceship and such should be interactive and playable and fun not sure whether blender is needed, as long as you can put it on a website, anything works
Onur Solmaz13,736 views • 1 month ago
*autoplan* This is one of my most used workflows in pi now While working with AI agents, there is a mechanical process by which I mine the agent for ideas. This reduces an open-ended feature design or bugfixing problem to a multiple choice question Basically, I keep asking paraphrases of the question "is this the best design?" 2-3 times, and then make the agent list them out, with a preference for practicality and simplicity "Is this the most elegant and long-term production ready solution?" "Is this the holy grail?" And then a decision gate which makes the model list all options while recommending a certain one, with a preference for practicality For example, while developing a plugin for pi or openclaw, asking the holy grail often causes the model to suggest changing the plugin/extension API like "The holy grail would be for pi to implement such an such API". The decision gate helps curb such stupid ideas The good thing about this workflow is, I can just automate typing all those mining prompts, and only do the deciding after the workflow finishes Caveat: This is not foolproof. I still reject all options, propose other ones, or run the workflow multiple times until I get what I want. But this helps reduce a ton of prompting to just "autoplan this" for me I am curious: When you try this, does it give you high quality answers/designs? And if not, what should change to improve it? To try it out: Install osolmaz/pi-workflows and then when you need to design something or fix a bug, just say "autoplan this". The skill should be picked up automatically Let it finish. It will give you a summary. When you choose an option, ask it to elaborate it with more details Repo:
Onur Solmaz15,905 views • 2 months ago
Speaking of graphs... here is something I wanted to build since 3 months, and finally had the chance to, thanks to Pi I often have these sequence of prompts that emerge while I work. Not just sequences but conditionals that necessitate control flow For example, one workflow that resembles socratic questioning: 1. Discuss some problem 2. "What is the most elegant and long-term production ready solution for this?" -> Agent replies 3. "Is that the holy grail?" 4. Agent can reply "yes it is, basically" or "no, it is not, it is instead ..." 5. If yes, continue to "autoimplement". If no, think about it and decide what to do And "autoimplement" is a single prompt of 6-7 sequential steps, which I've been meaning to make more deterministic as well But I wasn't sure how to build it I had previously built acpx workflows to be a swiss army knife, "something like n8n, but can drive codex through deterministic steps, nodes in a graph. or claude code. or pi. it uses acp..." But it had one problem. It was run from outside the harness, like a CI orchestrator I wanted to integrate acpx into pi. Because pi was the only CLI that could enable building of such a thing. But I wasn't sure how to reconcile a general ACP-based tool into a single coding agent I was being too accommodative of all the other harnesses, claude code, codex. I was trying to be too general I have changed my mind since then ACP is great and lets you integrate a harness into other software in cool ways But maybe, if a harness is proprietary, does not accept outside contributions, or does not even *support ACP*, maybe, it does not deserve cool features 😤 (they know who they are) So I ripped out ACP, and built it natively, only for pi No need for a web viewer... Just view it in a native widget, right inside pi! I cannot put into words how awesome it is to be able to do this! I am still tinkering, discovering. It is at osolmaz/pi-workflows if you want to take a look
Onur Solmaz17,138 views • 3 months ago
No more content to load