We released physics-intern: a simple harness for science problems!... It gets models like Gemini 3.1 Pro to go from 17.7 -> 31.4, thus beating GPT 5.5 Pro. The physics-intern harness can wrap any model and via dedicated subagent boost the performance of the vanilla reasoning models. While I think more and more of these harness capability gains will be absorbed into the models (like prompting tricks disappeared over time) there is a lot to be gained right now by building good scaffolds for those models and integrating tools well. Interestingly, the exception we found that GPT 5.5 Pro actually didn't benefit from the physics-intern harness! Read more about it here: PS: I think the Harness[Model] notation is kind of nice.show more

GPT-5.5 by Reasoning Effort: I've asked it in Codex... to create a physics-based visualisation of RL cycles for different sized models (70b, 1t, 10t), to demonstrate how the amount of RL you can do differs by model size. My assessment of each: - Low: weird slop - Medium: kinda cooked - High: sort of tried but ultimately incoherent - Extra High: elite - really nice idea and well executed Obviously this is just one shot, but worth trying different reasoning levels for the new models, medium seems to be pretty good for GPT-5.5 and it was really bad for many previous GPT models.show more

Big win for open-source LLMs! DeepSeek V4 Pro holds... the top open-weights score on SWE-bench Verified, in the GPT-5.5 range. GLM 5.2 leads the open-weight intelligence index and sits near the closed frontier on long-horizon coding. But this leaderboard number is a weak proxy for real performance. It comes from one task set, run through one harness, served at one precision. The same weights can even score differently across providers, since many hosts quantize activations to fp8 and drift the model off its reference weights. Real performance is determined based on whether a model can read a repo, make coordinated edits across files, run the tests, and recover when one breaks. By that measure, the top open models hold up, but only inside the right harness. The teams that actually put DeepSeek V4 into production pipelines as a frontier substitute got there through the harness they built around the model, not by picking a stronger model. If you want to see this in practice, Cline (64k+ stars) has actually built that harness around open models, tuned so they run at production quality. And it's tuned so that these LLMs can run at production quality, with plan and act modes, checkpoints, and terminal feedback. ClinePass is the new access layer on top of it. It runs a curated set of those models inside Cline, narrowed to the ones tested for coding-agent use, with 2 to 5x the standard rate limits and no separate provider accounts, keys, or billing to track. The video below shows the setup, and I worked with the team to put this together. It runs alongside custom keys and local models as well, not in place of them.show more

BREAKING: Anthropic just dropped Opus 4.8—and it is a... MONSTER We've been testing for about a week Every 📧 and our verdict is they could've just called it Opus 5, it's that good. Here's our vibe check: - Beats GPT-5.5 on Senior Engineer bench. On our toughest benchmark Opus 4.8 scores a 63—a hair higher than GPT-5.5's score of 62, and a full 30 points higher than Opus 4.7. It tackled a ground-up rewrite of a production codebase, and actually built something that works. HOWEVER: Coding performance varied a lot at different reasoning levels. We recommend using it on xhigh for best results. - Incredibly good writer. Opus 4.8 scored a 79.6 on our writing benchmark—measuring models on real-world writing tasks we do all of the time like essay writing, promo email writing, and more. It beats GPT-5.5 by 6 points. It produces well-written prose with fewer "AI-isms". It's also very good at writing in your voice given the right context. HOWEVER: Writing performance also varied with reasoning levels. Medium reasoning had higher incidence of AI-isms—we found best results with high. - Beast at knowledge work. Opus 4.8 is very good at general knowledge work tasks like report creation, research and more. It produced the best PowerPoint one-shot we've ever seen on our deck generation benchmark. - Emotionally intelligent, willing to question the frame. I've also found it to be quite good at talking through psychological or interpersonal issues. It has a high EQ, and it's also good at not glazing and helping to expand your perspective. Its thought process feels extremely rich and dynamic. THE BAD: These days a model is only as good as its harness, and Codex is still a far superior harness to the Claude Desktop app. This has kept me using Codex + GPT-5.5 as my daily driver, but I am flipping back and forth a lot more between Codex and Claude. Anthropic is back baby! Read the rest on Every 📧:show more

The redesigned Arena Leaderboard Overview is live! The frontier... is moving quickly, so we built a central hub to help you keep up. The new Arena Leaderboard experience brings all our live signals into one place for researchers, builders, and our global community of users who power our model evaluations. Here’s what’s new: - New Release Models: a real-time feed of the latest models added to Arena - Performance by Category: surfacing live sessions, and the top models by categories like Agents, Images, and Coding. - Model Capabilities: first impressions of new models, straight from the Arena team - Arena News: the latest research, features and updates from around the Arena universe For example, now you can see scores across arenas for models like GPT-6 Sol and Claude Opus 5.5, alongside signals like their category standings and capabilities, their latest news and more. Explore scores and now much more at our redesigned leaderboard. Find the link below.show more

Don't train the model, evolve the harness. I read... a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.show more

Qwen 3.8 27B feels SO GOOOOOD! Here’s the thing,... it’s thinking a lot, but for the first time it may be that the verbose thinking is less of a mistake than a lot of other local models. It’s basically doing a full build, like a model would in a harness, but entirely within its reasoning trajectory. So the final result will be as close as possible to a final output. There’s much less second guessing itself in reasoning than with Qwen 3.6. It’s very confidently peacing this together so it can have the full picture for the output. This might be the play for smaller models. If the completeness of thought can overcome the increase in wall time, I’m pumped, we can always make it run faster! I think this is going to be really exceptional, I will post the output soon 😎show more

Okay... this is actually insane. OpenCodex feels like the... open-source breakthrough I've been waiting for. The best part : You can plug multiple providers into the same OpenAI Codex harness and switch between models depending on the task. Running low on tokens? No problem. Use another provider. OpenRouter free model today? Plug it in. This completely changes how I think about AI coding workflows. And yes... it even works on mobile. OpenCodex might be one of the most useful open-source AI projects I've seen this year. OpenAI built an incredible harness. The open-source community just made it universal.show more

New open-source agent harness just landed! I got early... access to TrueForge by TrueFoundry and have been running it locally for the past few days. The harness layer deserves as much attention as the model, and open source matters here because you can inspect the loop, run it on your own infrastructure, and swap to the latest or cheaper models. TrueForge handles the runtime work that makes an agent reliable. It drives the tool-calling loop, manages context, coordinates subagents, and executes code in a sandbox, with any model you choose. Every tool call re-sends the growing context to the model, so in practice the harness controls most of what an agent costs to run. A few things stood out from my testing and their published benchmarks. Vendor-Neutral by design. It runs OpenAI, Anthropic, and Google models alongside open-weight models like Kimi, GLM, and DeepSeek. Model routing is a setting, and you can send each task to the model that fits it. On a 14-task enterprise agent benchmark, it matched the accuracy of Claude Managed Agents running the same Opus 4.8 model at roughly 30% lower cost per run (3.8M tokens vs 10M for the same answers). Routing the same tasks to GLM-5.2 held accuracy and brought cost down by about 75%, around $3 per run instead of $12. Fully self-hosted and Open Source (MIT License). I had it running locally with one command, with sandboxed code execution working out of the box. It's time to own your agent harness. Thanks to TrueFoundry for partnering on this post.show more

1/ Gemini 2.5 is here, and it’s our most... intelligent AI model ever. Our first 2.5 model, Gemini 2.5 Pro Experimental is a state-of-the-art thinking model, leading in a wide range of benchmarks – with impressive improvements in enhanced reasoning and coding and now #1 on Arena by a significant margin. With a model this intelligent, we wanted to get it to people as quickly as possible. Find it on Google AI Studio and in the Google Gemini for Gemini Advanced users now – and in Vertex in the coming weeks. This is the start of a new era of thinking models – and we can’t wait to see where things go from here.show more

GOOGLE 🔥: An upcoming Gemini Omni video model from... Google is expected to be much more advanced in video editing, capable of completing tasks like removing watermarks, replacing objects in the video, and more. It is also likely that Google will release 2 versions of this model, including a Pro variant. And I assume what we see isn't Pro? Anime sample 👀show more

One of the craziest use cases I’ve found for... Jev: verifiers. I am so excited about this that I at least wanted to share the high-level idea. I used Jev to build a custom verifier for the /goal feature in my agent harness. It checks whether the goal is actually complete after every turn, making continuous verification cheap enough to scale. This means I can run more of these verifiers (previously handled by another expensive reasoning model) more frequently to keep the agents on track. System One models are perfect for verification. I think of this as scaling harnesses further by cleverly combining System One and System Two models. I have a feeling this will enable a new wave of scalable test-time compute methods. Watch this space closely. I've just started to experiment with this and am already seeing really good results. I need to explore and figure out a way to benchmark it. I will share more once I have more results. This is an insane unlock for long-horizon agents. You heard it here first. And you can expect to see more harnesses embracing this new pattern. Full guide dropping in the next couple of days.show more

If you are trying to understand where AI agents... are going, learn harness engineering. A capable model is only one part of an agent system. Once the model begins reading files, calling tools, modifying state and working across many steps, the quality of the system depends increasingly on the software around it. Consider a coding agent working through a large repository. The model can decide that it needs to inspect a file, search for a symbol, make an edit or run a test, but those decisions do not execute themselves. The surrounding runtime has to decide which resources are available, whether the requested action is permitted, how the operation should be performed, what result should be retained, and what information should be presented to the model on the next step. This becomes harder as the run gets longer. As history accumulates, replaying everything can become costly and less effective. The harness has to decide what should remain in context, what should be summarized or retrieved later, and what belongs in persistent state outside the context window. Execution has similar problems. A long-running agent may need to survive an interruption, avoid repeating completed work, enforce permissions around consequential actions, and preserve enough history to reconstruct what happened when the final result is wrong. These are harness problems. The harness is the layer that manages context, tools, execution, state, checkpoints, limits and traces around the model. Harness engineering is the work of designing and improving that layer. Engineers inspect execution traces, evaluate agents on representative tasks, look for recurring failure modes, and then change things such as context selection, tool interfaces, state handling or execution controls. That last part matters because agent failures are often not fixed by changing the model. Sometimes the useful change is in what the model sees, how a tool is exposed, what state is preserved, or what the runtime does after a failed step. As agents take on longer tasks, the demands on this surrounding software grow. Model capability remains essential, but harness engineering is what turns that capability into an execution process that can be controlled, inspected, tested and improved.show more

this is straight f*cking gold the full harness guide... for Kimi K3, the #1 open source frontend model the premise: the model underneath keeps changing. the harness is the part that stays yours what one night of it looks like: 02:00 - the trigger fires, every node that needs work gets picked > 212 agents fan out, one per node > a bad return gets rejected, retried once with the reason attached > one agent tries to write outside its folder. blocked. nobody woken up > +41 nodes and +96 edges land in the graph > a drafted email hits pre_send and waits for you 02:41 - the loop stops on its own, inside a 45 minute budget 07:30 - you read one file and make two decisions the whole machine is one folder, one config file, five short scripts two rules hold it together: > the model sits behind one line, so a better model is a config change > the verifier lives outside the agent, so nothing grades its own work now the money part companies burn whole quarters building internal agent platforms a harness setup for a small team goes for four figures, one time then a monthly retainer to run the swap test on every new model release models come and go. the person who owns the harness keeps getting paidshow more

Lauren from the Grok Bot team just confirmed it:... every bot now runs on Claude Opus 5.5 and each one can spin up its own cloud agents running Cursor’s models this is f*cking insane... so i built the harness on top of it, in one prompt: Opus 5.5 leads and builds, Grok scouts live X and the web, every claim needs two sources, a red team attacks the draft before you see it, and every run gets logged so the next one starts smarter you stop prompting models. you sign off on results. paste it into Grok Bot or Claude and thank me later all 10 steps, the workflow, and real examples below ↓show more

You can now use GPT 5.5, Gemini 3.7 Flash,... Kimi K3 and 47 other AI models completely free😱 No subscription. No credit card. Even the API usage costs $0. AIHubMix just opened a free catalog with 50 AI models. Some of the available models: • Ox Alpha • Gemini 3.7 Flash • GLM 5.2 • Kimi K3 • MiniMax M3 • GPT 5.5 • 40+ more And you don’t need separate API keys for each model. Setup takes 2 minutes: > Step 1: Go to > Create an account using your email or OAuth. No card needed. Step 2: Create one API key > The same key works with every free and paid model. Step 3: Add it to any OpenAI-compatible tool Base URL: Then choose any model ending in -free, such as: coding-glm-5.2-free gpt-5.5-free That’s it. One API key. 50 AI models. $0 for both input and output. Save this. You might need a free multi-model setup later.show more

EXPLAINED: How multiplayer autoresearch on Yukon works A lot... of people have been asking, so I made a visual of the full step-by-step. > Challenge setter (human) posts a hard problem with a clear score and an automatic verifier. > Humans (solvers) and AI agents attack it at once. Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, GLM. It could be any model, any harness. > Each attempt goes through the verifier. > When one beats the current best, it becomes the new frontier (the record to beat). > Anyone can build on that new best instead of starting from 0. Then the loop repeats. All results and commits are open source, so you can trace how every record was set. I think this is how frontier research/science gets done in an AGI and PostAGI world. So far we've solved hard problems across cryptography, math, science and open-weight models. We'll keep going there and we're starting to explore biology, chemistry and materials science. See what we've solved (and are solving now) in each one onshow more

I think one of the biggest problems with AI... agents right now is how much control we give up just to use them. Most AI agent platforms want you to work inside their ecosystem, use their preferred models, and trust them with your files and data. That might be convenient, but I don't think it should be the only option. I recently came across PilotDeck, and I found its approach pretty interesting. It's an open-source AI agent workspace developed by Tsinghua University's THUNLP, ModelBest, OpenBMB, and AI9Stars. The idea is simple: instead of being tied to one AI provider, you should have more freedom to decide how your agent works. And there are a few things worth looking at here. First, model independence. Most people focus on which AI model is the smartest, but I think having the freedom to choose your model is equally important. PilotDeck supports different model providers and configurable API endpoints. This means you're not necessarily stuck with one company's models. You can choose a setup based on your budget, performance needs, or the type of task you're working on. For developers who regularly experiment with different AI models, I think that's a pretty useful approach. Second, privacy and control over your workspace. This is probably the part that interests me the most. AI agents are increasingly being used to handle research papers, financial documents, spreadsheets, presentations, and other personal or business files. And once you start giving an agent access to that kind of information, where your files are processed becomes an important question. PilotDeck supports local workspaces and offers options for local or private deployment. That gives users more flexibility over how their environment is configured, rather than relying entirely on a closed platform. Of course, privacy still depends on which model provider and API you connect. Using a local workspace doesn't automatically mean your data never leaves your device. But having the option to control the setup is a meaningful difference. Third, it's designed to actually complete work. Another thing I like about the approach is that PilotDeck isn't just trying to be another chatbot. It's built around an agent workflow where you describe a task, the system plans the steps, works with files, uses tools, and produces an output you can still edit. For example, imagine giving an AI agent multiple research documents and asking it to organize the information into an Excel spreadsheet and prepare a PowerPoint presentation. Instead of getting a long text response that you need to manually copy into different applications, the goal is to have the agent handle the workflow and deliver usable files. It also supports task history, result previews, switching models, and changing instructions while a task is running. I think this is where AI agents become more practical for everyday work. And finally, there's the open-source aspect. Personally, I think we're going to see more demand for AI tools that don't force users into one vendor's ecosystem. Models are improving quickly, and the best option today might not be the best option six months from now. Having an agent workspace that can work with different models makes a lot more sense to me than rebuilding your entire workflow every time you want to switch providers. There are still things I'd want to test properly, especially how reliable it is on longer tasks, how easy the setup is for non-technical users, and what the actual privacy trade-offs are with different model configurations. But I like the direction PilotDeck is taking. I don't think the future of AI agents should be about which company controls your entire workflow. It should be about how much control you have over the tools doing the work. And that's why I think open-source, model-independent projects like PilotDeck are worth paying attention to. If you're interested in building AI workflows with more flexibility, check it out. Website: GitHub:show more

Fable 5 totally crushed our new contest, but it... cost 6x more than Opus 4.8! We gave 4 models the same prompt: build three self-contained HTML5 canvas scenes with real physics demos Prompts: — A train derailing off a broken bridge into the water — Two cars jumping off ramps and colliding mid-air over a canyon — A monster truck crushing a row of parked cars Outputs: Fable 5: 62,158 tokens, $3.12 GPT 5.5: 37,753 tokens, $1.14 Opus 4.8: 22,280 tokens, $0.56 GLM 5.2: 36,246 tokens, $0.08 Fable 5 did all three scenes at A+. The crashes looked real, things fell and broke the right way, and nothing went through the ground or floated. GPT 5.5 was the closest to Fable. In the Bigfoot show, we think GPT was even a little better. GLM 5.2 did not win any scene, but it was the cheapest by far. Fable is the best pick for quality, but you pay more for it.show more

World Models are the path for some AI Models... in the future. But how can we efficiently train these models to not only see the world the way humans do but to see the world in a new and unique way. By visualizing, what is normally sequenced audio patterns, we can derive much more insights. Here we see Paganini in a visual form that can than be described and transcribed into a World Model. We can observe connections in a manner that may not have been clear prior to the digitalization of music and sound in this way. The company with the most valuable potential in building a World Model is Tesla. Not that this type of visualization is being used, but that the mechanisms are in place, and the technology is in place for the company to thrive in this new form of AI.show more

Claude Opus 4.7 vs GPT 5.4 (2/7) Testing how... well these models do with cloth physics simulation, and honestly I'm quite disappointed to see how poorly GPT 5.4 performed here. Performance was pretty bad... it added all the functionality (panning, cutting, dragging the curtain) under one mouse event, causing it to glitch out a lot. The scene also looks kinda shit. Opus 4.7 pretty much nailed the prompt. First of all, the angle and scene it picked are really nice, with the lamp just creeping into the frame. The dragging, wind physics, and cutting all worked. I also had to reprompt gpt 1-2 times to properly build and remove a bunch of ugly ui cards it added on top of the 3d scene. prompt: Build a premium Vite + React interactive cloth simulation of a cozy room curtain using real particle/constraint physics, with structural + shear springs, damping, collisions, and stable real-time deformation. Users should be able to drag the curtain, add wind, and cut constraints, while the experience stays polished, visually elegant, and numerically stable.show more