Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

We released physics-intern: a simple harness for science problems! It gets models like Gemini 3.1 Pro to go from 17.7 -> 31.4, thus beating GPT 5.5 Pro. The physics-intern harness can wrap any model and via dedicated subagent boost the performance of the vanilla reasoning models. While I think...

lvwerra's profile picture

Leandro von Werra

12,904 subscribers

97,519 görüntüleme • 5 ay önce •via X (Twitter)

Sağlık & İyilikBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

GPT-5.5 by Reasoning Effort: I've asked it in Codex to create a physics-based visualisation of RL cycles for different sized models (70b, 1t, 10t), to demonstrate how the amount of RL you can do differs by model size. My assessment of each: - Low: weird slop - Medium: kinda cooked - High: sort of tried but ultimately incoherent - Extra High: elite - really nice idea and well executed Obviously this is just one shot, but worth trying different reasoning levels for the new models, medium seems to be pretty good for GPT-5.5 and it was really bad for many previous GPT models.
0:15

GPT-5.5 by Reasoning Effort: I've asked it in Codex to create a physics-based visualisation of RL cycles for different sized models (70b, 1t, 10t), to demonstrate how the amount of RL you can do differs by model size. My assessment of each: - Low: weird slop - Medium: kinda cooked - High: sort of tried but ultimately incoherent - Extra High: elite - really nice idea and well executed Obviously this is just one shot, but worth trying different reasoning levels for the new models, medium seems to be pretty good for GPT-5.5 and it was really bad for many previous GPT models.

Peter Gostev (SF: 22-26 June)

209,258 görüntüleme • 6 ay önce

Big win for open-source LLMs! DeepSeek V4 Pro holds the top open-weights score on SWE-bench Verified, in the GPT-5.5 range. GLM 5.2 leads the open-weight intelligence index and sits near the closed frontier on long-horizon coding. But this leaderboard number is a weak proxy for real performance. It comes from one task set, run through one harness, served at one precision. The same weights can even score differently across providers, since many hosts quantize activations to fp8 and drift the model off its reference weights. Real performance is determined based on whether a model can read a repo, make coordinated edits across files, run the tests, and recover when one breaks. By that measure, the top open models hold up, but only inside the right harness. The teams that actually put DeepSeek V4 into production pipelines as a frontier substitute got there through the harness they built around the model, not by picking a stronger model. If you want to see this in practice, Cline (64k+ stars) has actually built that harness around open models, tuned so they run at production quality. And it's tuned so that these LLMs can run at production quality, with plan and act modes, checkpoints, and terminal feedback. ClinePass is the new access layer on top of it. It runs a curated set of those models inside Cline, narrowed to the ones tested for coding-agent use, with 2 to 5x the standard rate limits and no separate provider accounts, keys, or billing to track. The video below shows the setup, and I worked with the team to put this together. It runs alongside custom keys and local models as well, not in place of them.
0:14

Big win for open-source LLMs! DeepSeek V4 Pro holds the top open-weights score on SWE-bench Verified, in the GPT-5.5 range. GLM 5.2 leads the open-weight intelligence index and sits near the closed frontier on long-horizon coding. But this leaderboard number is a weak proxy for real performance. It comes from one task set, run through one harness, served at one precision. The same weights can even score differently across providers, since many hosts quantize activations to fp8 and drift the model off its reference weights. Real performance is determined based on whether a model can read a repo, make coordinated edits across files, run the tests, and recover when one breaks. By that measure, the top open models hold up, but only inside the right harness. The teams that actually put DeepSeek V4 into production pipelines as a frontier substitute got there through the harness they built around the model, not by picking a stronger model. If you want to see this in practice, Cline (64k+ stars) has actually built that harness around open models, tuned so they run at production quality. And it's tuned so that these LLMs can run at production quality, with plan and act modes, checkpoints, and terminal feedback. ClinePass is the new access layer on top of it. It runs a curated set of those models inside Cline, narrowed to the ones tested for coding-agent use, with 2 to 5x the standard rate limits and no separate provider accounts, keys, or billing to track. The video below shows the setup, and I worked with the team to put this together. It runs alongside custom keys and local models as well, not in place of them.

Avi Chawla

44,124 görüntüleme • 3 ay önce

✨ Watch the full tutorial: The Hip Cuffs Harness is the definition of less is more. It’s simple, comfortable, and minimalistic yet properly restraining. You can explore feelings of helplessness while keeping the body in a positionally comfortable tie, allowing your play to last longer. 💖 If you’d like a full step-by-step guide with images for tying this beginner-friendly harness, comment “HARNESS” and we’ll send you the downloadable Harness Guide PDF.
0:15

✨ Watch the full tutorial: The Hip Cuffs Harness is the definition of less is more. It’s simple, comfortable, and minimalistic yet properly restraining. You can explore feelings of helplessness while keeping the body in a positionally comfortable tie, allowing your play to last longer. 💖 If you’d like a full step-by-step guide with images for tying this beginner-friendly harness, comment “HARNESS” and we’ll send you the downloadable Harness Guide PDF.

Shibari Study

67,802 görüntüleme • 7 ay önce

Harness the power of generative AI without just prompting and hoping for the best. From Unbound Slop to Tripo🔜SIGGRAPH2026 to Unreal Engine, we love this workflow for generating 3D models.
0:17

Harness the power of generative AI without just prompting and hoping for the best. From Unbound Slop to Tripo🔜SIGGRAPH2026 to Unreal Engine, we love this workflow for generating 3D models.

Unbound

26,532 görüntüleme • 6 ay önce

BREAKING: Anthropic just dropped Opus 4.8—and it is a MONSTER We've been testing for about a week Every 📧 and our verdict is they could've just called it Opus 5, it's that good. Here's our vibe check: - Beats GPT-5.5 on Senior Engineer bench. On our toughest benchmark Opus 4.8 scores a 63—a hair higher than GPT-5.5's score of 62, and a full 30 points higher than Opus 4.7. It tackled a ground-up rewrite of a production codebase, and actually built something that works. HOWEVER: Coding performance varied a lot at different reasoning levels. We recommend using it on xhigh for best results. - Incredibly good writer. Opus 4.8 scored a 79.6 on our writing benchmark—measuring models on real-world writing tasks we do all of the time like essay writing, promo email writing, and more. It beats GPT-5.5 by 6 points. It produces well-written prose with fewer "AI-isms". It's also very good at writing in your voice given the right context. HOWEVER: Writing performance also varied with reasoning levels. Medium reasoning had higher incidence of AI-isms—we found best results with high. - Beast at knowledge work. Opus 4.8 is very good at general knowledge work tasks like report creation, research and more. It produced the best PowerPoint one-shot we've ever seen on our deck generation benchmark. - Emotionally intelligent, willing to question the frame. I've also found it to be quite good at talking through psychological or interpersonal issues. It has a high EQ, and it's also good at not glazing and helping to expand your perspective. Its thought process feels extremely rich and dynamic. THE BAD: These days a model is only as good as its harness, and Codex is still a far superior harness to the Claude Desktop app. This has kept me using Codex + GPT-5.5 as my daily driver, but I am flipping back and forth a lot more between Codex and Claude. Anthropic is back baby! Read the rest on Every 📧:
0:11

BREAKING: Anthropic just dropped Opus 4.8—and it is a MONSTER We've been testing for about a week Every 📧 and our verdict is they could've just called it Opus 5, it's that good. Here's our vibe check: - Beats GPT-5.5 on Senior Engineer bench. On our toughest benchmark Opus 4.8 scores a 63—a hair higher than GPT-5.5's score of 62, and a full 30 points higher than Opus 4.7. It tackled a ground-up rewrite of a production codebase, and actually built something that works. HOWEVER: Coding performance varied a lot at different reasoning levels. We recommend using it on xhigh for best results. - Incredibly good writer. Opus 4.8 scored a 79.6 on our writing benchmark—measuring models on real-world writing tasks we do all of the time like essay writing, promo email writing, and more. It beats GPT-5.5 by 6 points. It produces well-written prose with fewer "AI-isms". It's also very good at writing in your voice given the right context. HOWEVER: Writing performance also varied with reasoning levels. Medium reasoning had higher incidence of AI-isms—we found best results with high. - Beast at knowledge work. Opus 4.8 is very good at general knowledge work tasks like report creation, research and more. It produced the best PowerPoint one-shot we've ever seen on our deck generation benchmark. - Emotionally intelligent, willing to question the frame. I've also found it to be quite good at talking through psychological or interpersonal issues. It has a high EQ, and it's also good at not glazing and helping to expand your perspective. Its thought process feels extremely rich and dynamic. THE BAD: These days a model is only as good as its harness, and Codex is still a far superior harness to the Claude Desktop app. This has kept me using Codex + GPT-5.5 as my daily driver, but I am flipping back and forth a lot more between Codex and Claude. Anthropic is back baby! Read the rest on Every 📧:

Dan Shipper

354,962 görüntüleme • 5 ay önce

You can now post-train a model inside your existing production harness with our platform, AC2. A production harness is a whole engineered system around the LLM, with its own context management, tools, sandboxing, and control flows. Porting that into a new training runtime can be expensive and could introduce train-test mismatch, where the policy is optimized against a simulated harness and then struggles in production. All you need to do is swap out the harness’ LLM response endpoint to one provided by AC2, and expose a lightweight protocol for AC2 to initiate and grade rollouts; the trainer handles the rest.
0:13

You can now post-train a model inside your existing production harness with our platform, AC2. A production harness is a whole engineered system around the LLM, with its own context management, tools, sandboxing, and control flows. Porting that into a new training runtime can be expensive and could introduce train-test mismatch, where the policy is optimized against a simulated harness and then struggles in production. All you need to do is swap out the harness’ LLM response endpoint to one provided by AC2, and expose a lightweight protocol for AC2 to initiate and grade rollouts; the trainer handles the rest.

Applied Compute

182,507 görüntüleme • 2 ay önce

The redesigned Arena Leaderboard Overview is live! The frontier is moving quickly, so we built a central hub to help you keep up. The new Arena Leaderboard experience brings all our live signals into one place for researchers, builders, and our global community of users who power our model evaluations. Here’s what’s new: - New Release Models: a real-time feed of the latest models added to Arena - Performance by Category: surfacing live sessions, and the top models by categories like Agents, Images, and Coding. - Model Capabilities: first impressions of new models, straight from the Arena team - Arena News: the latest research, features and updates from around the Arena universe For example, now you can see scores across arenas for models like GPT-6 Sol and Claude Opus 5.5, alongside signals like their category standings and capabilities, their latest news and more. Explore scores and now much more at our redesigned leaderboard. Find the link below.
0:17

The redesigned Arena Leaderboard Overview is live! The frontier is moving quickly, so we built a central hub to help you keep up. The new Arena Leaderboard experience brings all our live signals into one place for researchers, builders, and our global community of users who power our model evaluations. Here’s what’s new: - New Release Models: a real-time feed of the latest models added to Arena - Performance by Category: surfacing live sessions, and the top models by categories like Agents, Images, and Coding. - Model Capabilities: first impressions of new models, straight from the Arena team - Arena News: the latest research, features and updates from around the Arena universe For example, now you can see scores across arenas for models like GPT-6 Sol and Claude Opus 5.5, alongside signals like their category standings and capabilities, their latest news and more. Explore scores and now much more at our redesigned leaderboard. Find the link below.

Arena.ai

24,630 görüntüleme • 1 ay önce

Qwen 3.8 27B feels SO GOOOOOD! Here’s the thing, it’s thinking a lot, but for the first time it may be that the verbose thinking is less of a mistake than a lot of other local models. It’s basically doing a full build, like a model would in a harness, but entirely within its reasoning trajectory. So the final result will be as close as possible to a final output. There’s much less second guessing itself in reasoning than with Qwen 3.6. It’s very confidently peacing this together so it can have the full picture for the output. This might be the play for smaller models. If the completeness of thought can overcome the increase in wall time, I’m pumped, we can always make it run faster! I think this is going to be really exceptional, I will post the output soon 😎
0:18

Qwen 3.8 27B feels SO GOOOOOD! Here’s the thing, it’s thinking a lot, but for the first time it may be that the verbose thinking is less of a mistake than a lot of other local models. It’s basically doing a full build, like a model would in a harness, but entirely within its reasoning trajectory. So the final result will be as close as possible to a final output. There’s much less second guessing itself in reasoning than with Qwen 3.6. It’s very confidently peacing this together so it can have the full picture for the output. This might be the play for smaller models. If the completeness of thought can overcome the increase in wall time, I’m pumped, we can always make it run faster! I think this is going to be really exceptional, I will post the output soon 😎

Kyle Hessling

65,585 görüntüleme • 2 ay önce

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.
0:18

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.

Akshay 🚀

245,663 görüntüleme • 3 ay önce

This blew up more than I expected. To minimize the negative impact of these changes, my orchestrator can now smoothly switch/handoff between any provider/model (e.g., fable 5 -> gpt-5-sol) Just one of the benefits of owning the harness & orchestrator.
0:13

This blew up more than I expected. To minimize the negative impact of these changes, my orchestrator can now smoothly switch/handoff between any provider/model (e.g., fable 5 -> gpt-5-sol) Just one of the benefits of owning the harness & orchestrator.

elvis

62,278 görüntüleme • 3 ay önce

Okay... this is actually insane. OpenCodex feels like the open-source breakthrough I've been waiting for. The best part : You can plug multiple providers into the same OpenAI Codex harness and switch between models depending on the task. Running low on tokens? No problem. Use another provider. OpenRouter free model today? Plug it in. This completely changes how I think about AI coding workflows. And yes... it even works on mobile. OpenCodex might be one of the most useful open-source AI projects I've seen this year. OpenAI built an incredible harness. The open-source community just made it universal.
0:10

Okay... this is actually insane. OpenCodex feels like the open-source breakthrough I've been waiting for. The best part : You can plug multiple providers into the same OpenAI Codex harness and switch between models depending on the task. Running low on tokens? No problem. Use another provider. OpenRouter free model today? Plug it in. This completely changes how I think about AI coding workflows. And yes... it even works on mobile. OpenCodex might be one of the most useful open-source AI projects I've seen this year. OpenAI built an incredible harness. The open-source community just made it universal.

CHOI

42,894 görüntüleme • 3 ay önce

New open-source agent harness just landed! I got early access to TrueForge by TrueFoundry and have been running it locally for the past few days. The harness layer deserves as much attention as the model, and open source matters here because you can inspect the loop, run it on your own infrastructure, and swap to the latest or cheaper models. TrueForge handles the runtime work that makes an agent reliable. It drives the tool-calling loop, manages context, coordinates subagents, and executes code in a sandbox, with any model you choose. Every tool call re-sends the growing context to the model, so in practice the harness controls most of what an agent costs to run. A few things stood out from my testing and their published benchmarks. Vendor-Neutral by design. It runs OpenAI, Anthropic, and Google models alongside open-weight models like Kimi, GLM, and DeepSeek. Model routing is a setting, and you can send each task to the model that fits it. On a 14-task enterprise agent benchmark, it matched the accuracy of Claude Managed Agents running the same Opus 4.8 model at roughly 30% lower cost per run (3.8M tokens vs 10M for the same answers). Routing the same tasks to GLM-5.2 held accuracy and brought cost down by about 75%, around $3 per run instead of $12. Fully self-hosted and Open Source (MIT License). I had it running locally with one command, with sandboxed code execution working out of the box. It's time to own your agent harness. Thanks to TrueFoundry for partnering on this post.
0:16

New open-source agent harness just landed! I got early access to TrueForge by TrueFoundry and have been running it locally for the past few days. The harness layer deserves as much attention as the model, and open source matters here because you can inspect the loop, run it on your own infrastructure, and swap to the latest or cheaper models. TrueForge handles the runtime work that makes an agent reliable. It drives the tool-calling loop, manages context, coordinates subagents, and executes code in a sandbox, with any model you choose. Every tool call re-sends the growing context to the model, so in practice the harness controls most of what an agent costs to run. A few things stood out from my testing and their published benchmarks. Vendor-Neutral by design. It runs OpenAI, Anthropic, and Google models alongside open-weight models like Kimi, GLM, and DeepSeek. Model routing is a setting, and you can send each task to the model that fits it. On a 14-task enterprise agent benchmark, it matched the accuracy of Claude Managed Agents running the same Opus 4.8 model at roughly 30% lower cost per run (3.8M tokens vs 10M for the same answers). Routing the same tasks to GLM-5.2 held accuracy and brought cost down by about 75%, around $3 per run instead of $12. Fully self-hosted and Open Source (MIT License). I had it running locally with one command, with sandboxed code execution working out of the box. It's time to own your agent harness. Thanks to TrueFoundry for partnering on this post.

elvis

11,303 görüntüleme • 2 ay önce

A harness turns a model into an agent. At it’s core it provides 4 things: - a system prompt - tools - an agentic loop - a translation layer across models New blog post from Earendil co-founder colin hanna on what a harness is, and how you can own yours. Full post below
0:18

A harness turns a model into an agent. At it’s core it provides 4 things: - a system prompt - tools - an agentic loop - a translation layer across models New blog post from Earendil co-founder colin hanna on what a harness is, and how you can own yours. Full post below

Pi

426,345 görüntüleme • 2 ay önce

Gemini 3.1 Pro is now available for all paid tiers! The default model router, Auto (Gemini 3) will use Gemini 3.1 Pro as it's pro model for complex prompts. You can also set the new model via /model to try it out. We are excited to see how you put it to use!
0:12

Gemini 3.1 Pro is now available for all paid tiers! The default model router, Auto (Gemini 3) will use Gemini 3.1 Pro as it's pro model for complex prompts. You can also set the new model via /model to try it out. We are excited to see how you put it to use!

Gemini CLI

400,185 görüntüleme • 8 ay önce

Strap in for a ride with the Lovense Harness. It's the perfect match for Lapis and the entire Lovense lineup. Get ready for endless possibilities and unparalleled pleasure. #Lovense #Lovenseway #harness Get Lovense Harness from here👇
0:14

Strap in for a ride with the Lovense Harness. It's the perfect match for Lapis and the entire Lovense lineup. Get ready for endless possibilities and unparalleled pleasure. #Lovense #Lovenseway #harness Get Lovense Harness from here👇

LovenseOfficial

57,671 görüntüleme • 2 yıl önce

Had early access to Gemini with Deep Think. Very good model, big gains over standard Gemini 2.5 Pro for a lot of problems. Here is the first attempt at the starship control panel prompt I try with every model. First time I have seen a model make a 3D interface in response.
0:14

Had early access to Gemini with Deep Think. Very good model, big gains over standard Gemini 2.5 Pro for a lot of problems. Here is the first attempt at the starship control panel prompt I try with every model. First time I have seen a model make a 3D interface in response.

Ethan Mollick

253,001 görüntüleme • 1 yıl önce

1/ Gemini 2.5 is here, and it’s our most intelligent AI model ever. Our first 2.5 model, Gemini 2.5 Pro Experimental is a state-of-the-art thinking model, leading in a wide range of benchmarks – with impressive improvements in enhanced reasoning and coding and now #1 on Arena by a significant margin. With a model this intelligent, we wanted to get it to people as quickly as possible. Find it on Google AI Studio and in the Google Gemini for Gemini Advanced users now – and in Vertex in the coming weeks. This is the start of a new era of thinking models – and we can’t wait to see where things go from here.
0:14

1/ Gemini 2.5 is here, and it’s our most intelligent AI model ever. Our first 2.5 model, Gemini 2.5 Pro Experimental is a state-of-the-art thinking model, leading in a wide range of benchmarks – with impressive improvements in enhanced reasoning and coding and now #1 on Arena by a significant margin. With a model this intelligent, we wanted to get it to people as quickly as possible. Find it on Google AI Studio and in the Google Gemini for Gemini Advanced users now – and in Vertex in the coming weeks. This is the start of a new era of thinking models – and we can’t wait to see where things go from here.

Sundar Pichai

864,989 görüntüleme • 1 yıl önce

GOOGLE 🔥: An upcoming Gemini Omni video model from Google is expected to be much more advanced in video editing, capable of completing tasks like removing watermarks, replacing objects in the video, and more. It is also likely that Google will release 2 versions of this model, including a Pro variant. And I assume what we see isn't Pro? Anime sample 👀
0:10

GOOGLE 🔥: An upcoming Gemini Omni video model from Google is expected to be much more advanced in video editing, capable of completing tasks like removing watermarks, replacing objects in the video, and more. It is also likely that Google will release 2 versions of this model, including a Pro variant. And I assume what we see isn't Pro? Anime sample 👀

🚨 AI News | TestingCatalog

180,557 görüntüleme • 5 ay önce