正在加载视频...

视频加载失败

Finding 1: Harness affects cost more than correctness. Fable 5 can cost twice as much for a 1.1-point gain in success rate. On SWE-bench Lite: - Claude Code: 97.8% accuracy, $1.33/rollout - Pi: 96.7% accuracy, $0.67/rollout While Claude Code reaches the highest success rate on the SWE-bench Lite frontier,...

14,563 次观看 • 6 天前 •via X (Twitter)

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Big win for open-source LLMs! DeepSeek V4 Pro holds the top open-weights score on SWE-bench Verified, in the GPT-5.5 range. GLM 5.2 leads the open-weight intelligence index and sits near the closed frontier on long-horizon coding. But this leaderboard number is a weak proxy for real performance. It comes from one task set, run through one harness, served at one precision. The same weights can even score differently across providers, since many hosts quantize activations to fp8 and drift the model off its reference weights. Real performance is determined based on whether a model can read a repo, make coordinated edits across files, run the tests, and recover when one breaks. By that measure, the top open models hold up, but only inside the right harness. The teams that actually put DeepSeek V4 into production pipelines as a frontier substitute got there through the harness they built around the model, not by picking a stronger model. If you want to see this in practice, Cline (64k+ stars) has actually built that harness around open models, tuned so they run at production quality. And it's tuned so that these LLMs can run at production quality, with plan and act modes, checkpoints, and terminal feedback. ClinePass is the new access layer on top of it. It runs a curated set of those models inside Cline, narrowed to the ones tested for coding-agent use, with 2 to 5x the standard rate limits and no separate provider accounts, keys, or billing to track. The video below shows the setup, and I worked with the team to put this together. It runs alongside custom keys and local models as well, not in place of them.

Avi Chawla

44,124 次观看 • 2 个月前

this is the worst local ai will ever be. it only gets better from here. if you are not expanding your mind with these small models you are missing what's happening right now 99 percent tool call success rate. when steered well with the right skills and a framework like hermes agent the node becomes a cognition layer. not a chatbot. not a toy. an extension of how you think. i was cranking this node at 35 to 50 tok/s all day on personal experiments and now after all the work is done qwen 3.5 9B is iterating on its own code. the game it created. fixing its own bugs autonomously. and the part you should probably not miss is that all of this is happening on a RTX 3060. not an H100. not an A100. the card most of you have sitting in a drawer right now. if you just open that drawer and put that intelligence to work every tensor core on that card should be running for you. your work. your experiments. your thinking. you all have it but because nobody told you what this hardware can actually do in 2026 you never tried. the day it unlocks is the day you test your workload, understand the tradeoffs, debug the loops, and then decide if you need to scale the hardware. there is no point buying 3 mac studios when things done well you can squeeze a similar level of intelligence from 9B compared to 70B. but only when you create the right environment for your model through the right harness. and let me tell you i have tried claude code as a local harness. i have tried opencode. i have tried various others. somehow i landed on hermes agent and never left. there is something magical going on at Nous Research. the tool call parsers, the skills system, the way it handles small models natively. nothing else comes close for local inference. own your cognition. your AI. your agent. your prompts. your experiments. why give them away for free. those are who you are and they don't belong on someone else's servers being monitored. just give it a shot with your existing hardware. you run into a problem the community will help you. and if you are migrating from openclaw to hermes i will personally help you make the switch.

Sudo su

58,717 次观看 • 6 个月前

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.

Akshay 🚀

245,252 次观看 • 2 个月前

A lot of DeFi borrowers these days aren’t really scared of high rates. They’re scared of rates that can change while they’re sleeping. You borrow USDC at 3%, utilization jumps overnight, and suddenly your cost looks nothing like what you expected. This cycle the real damage for a lot of people wasn’t liquidation. It was never knowing what their borrow cost would be next month. That’s why the latest numbers from TermMax | Fixed Rate Borrowing & Lending stood out. They’re showing fixed USDC borrow rates against cbBTC and WBTC at roughly 2.3% through May 31 and 2.5% through June 30, with July already looking cheaper than most big floating pools on Ethereum. And the rate stays locked the whole time. Most people’s first reaction is still “fixed rates are supposed to be more expensive, right?” These ones are competitive while removing the guesswork. The setup is straightforward. One collateral type, fixed term, risk visible before you borrow. You already know what you’re posting, how long you’re borrowing for, and what the cost should be during that window. No waking up to a completely different number. Floating rate markets keep moving. Liquidity changes, demand changes, utilization changes. A position that feels fine today can reprice hard a few days later. That constant uncertainty becomes its own hidden cost when you’re actually trying to manage cash flow. What they keep saying makes sense once you’ve felt it: known rate, known term, known risk. The risk doesn’t vanish, but at least you see it upfront instead of getting surprised later. Of course there are tradeoffs. Lock in now and rates could drop, leaving you paying more than you might have otherwise. Liquidity and flexibility probably won’t match the biggest variable rate pools either. Still, the mindset in DeFi lending feels like it’s shifting. A year or two ago everyone was just chasing the lowest APY. Now more people seem to care whether they can actually understand what they’re stepping into before they commit. With tokenized assets getting real traction and big projections coming out, that kind of predictability might start mattering more than pure yield chasing. You can actually plan around it. Tired of rate surprises wrecking your positions? Fixed terms like this change how you think about borrowing.

Domingo_gou | 火币赚币🐬

11,892 次观看 • 4 个月前

Claude Code + Google Stitch 2.0 is f*cking cracked 🤯 Google just dropped a free AI design agent that solves Claude Code's biggest weakness: frontend design. One screenshot of a high-converting landing page → a production-ready site for your brand in minutes. All inside Google Stitch + Claude Code. Perfect for DTC brands and agencies who are building advertorial pages and product launch pages for Meta but burning days on designer back-and-forth. If you're running Meta ads and need 5-10 different landing pages testing different hooks, angles, and offers — each one targeting a different audience and pain point — you know the bottleneck isn't the ads. It's the pages. Briefing designers, waiting for revisions, paying $2-5K per page. Stitch eliminates the design bottleneck: → Find a high-converting advertorial that's scaling on Meta → Screenshot it and drop it into Stitch (powered by Gemini 3.1) → Stitch redesigns it with your brand's colors, fonts, and imagery using Nano Banana 2 → Edit sections visually — headlines, CTAs, layouts — without touching code → Export the code and paste it into Claude Code → Claude builds the full production site and deploys to Vercel or Netlify in 60 seconds No designer. No $3K per landing page. No Claude Code frontend that looks like a template from 2019. What you get: → Designer-quality landing pages and advertorials built in minutes, not weeks → Visual editing so you actually see the design before you code it → Nano Banana 2 generating on-brand product imagery and hero shots → A repeatable system — new angle, new page, same pipeline Built 100% with Google Stitch 2.0 + Claude Code. I put together a full playbook showing the exact workflow: how to find winning pages, redesign them in Stitch, and deploy with Claude Code. Want it for free? > Like this post > Comment "STITCH" And I'll send it over (must be following so I can DM)

Mike Futia

126,469 次观看 • 6 个月前

Claude Fable 5 + Claude Design is f*cking insane 🤯 Anthropic just dropped its most intelligent model ever, and the first thing I pointed it at was email design. I built a complete email campaign design in Claude Design, and the difference is night and day: tighter layouts, cleaner hierarchy, on-brand from the first generation. All inside Claude Design with Fable 5. Perfect for DTC brands and agencies who are still paying email agencies $3-5K/month for campaign designs that take 2 weeks to ship. If your campaign calendar is packed but every new email means briefing a designer, waiting on mockups, sending notes, and waiting again... This workflow eliminates the entire bottleneck: → Load your brand design system into Claude Design once (colors, fonts, logo, button styling) → Switch the model to Claude Fable 5 — Anthropic's new state-of-the-art model with the best vision of any AI → Prompt the campaign email section by section: header, hero, headline, offer block, CTA → Fable 5 nails layout and brand details that older models fumbled → Iterate inline — swap images, adjust styling, color-pick directly in the canvas → Export the finished email and hand off to your ESP No briefing a designer. No 2-week turnaround on a single campaign. No paying an agency $4K/month for 4 emails. What you get: → Campaign emails designed in minutes, not weeks → A reusable design system every new email pulls from automatically → Noticeably smarter design decisions from Fable 5's upgraded vision → Full inline editing before anything touches your ESP Built 100% with Claude Design + Claude Fable 5. I recorded a full walkthrough showing exactly how this works. Want it for free? > Like this post > Comment "FABLE" And I'll send it over (must be following so I can DM)

Mike Futia

43,201 次观看 • 3 个月前

somebody explain this because i refuse to accept it someone ran 48 scored trials and one agent beat a whole fleet of them on all 6 task families, at 0.93 cents a run against 1.9, while openai's best fleet shape was paying $0.008 for every single point of accuracy it bought i read it expecting a hit piece and found the opposite: the fleets that partitioned the dependency graph properly lifted pass rate 14% and cut wall-clock 2.10x on the same tasks, and one of them beat claude code with agent teams the thing that decides it has a name, Graph Engineering, and it is a property of the diagram rather than the model: - partition on the real dependency graph pulled from static analysis, never by folder or by file, because the gains land hardest on the most dependency-dense projects - isolate the structural hub files first, since those are the nodes every partition would otherwise have to share - measure the critical path and treat it as the floor, because a chain that genuinely feeds itself cannot be replaced by more workers and wrapping it in a scheduler does not shorten it - match the topology to the coupling instead of defaulting to parallel: on coupled work a static parallel shape drops below a single agent, so the mismatch is worse than no orchestration - remember each worker serialises its own subtasks, which adds edges inside every agent that were never in your plan - budget the fan-out before you fire it, because three agents already burn roughly three times the tokens and the multiplier compounds across sessions - check worker count against your rate limit, since fifteen workers at ten requests a second walk straight through a hundred-per-second ceiling and cascade - put a script gate in front of the planner: it costs 0.15 seconds and zero tokens, and it lets the expensive model skip 43 to 63% of the steps for at most 1.4 points of accuracy the catch is the coordination tax, and it scales with how clever the shape looks: 58% extra reasoning turns for independent workers, 263% decentralised, 285% centralised, and 515% for the hybrid setup everyone reaches for first the same paper found that hybrid then collapses hardest on tool-heavy work at a 0.452 success rate, while the plainer decentralised shape beat centralised outright despite carrying more overhead, because parallel efficiency is what survives bookmark this, the whole build sits in the article ↓

Argona

32,932 次观看 • 1 个月前