Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Most discussions around Claude Code focus on model quality or capability. But when you actually use it, the friction shows up earlier, more at the system level than the model itself. Token limits get the blame, but a lot of it comes from inefficient sessions: repeated context, extra tool...

15,079 görüntüleme • 6 ay önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

install these before your next long Claude Code session if your Opus 5.5 bill hurts and you don’t know where to start, copy this: claude-code-router: sits in front of Claude Code and sends each request to the cheapest model that can do it. Haiku does the grep, Opus only gets the hard part. 37k stars rtk: shrinks test, build and git output before the model sees it. one binary, up to 90% less bash output in your context. 83k stars claude-mem: remembers what happened in your past sessions and feeds back only the relevant part, so you stop paying to re-explain your project every morning. 100k stars context-mode: a huge tool output goes into a local database instead of your context window, and Claude pulls out only the lines it needs. 18k stars caveman: makes Claude answer in short telegraphic sentences. code stays identical, and output is the expensive side on Opus 5.5 ($20 per million). 51k stars token-optimizer: a local dashboard that shows where your tokens actually went, and keeps your work alive across compactions. 1.9k stars token-savior: replaces whole-file reads with symbol lookups, so Claude reads the function and not the 2,000 lines around it code-review-graph: maps your repo as a graph and gives Claude only the code connected to the change you’re reviewing jev-ultrafast: a browser agent where Jev picks every click, and a text model wakes up only when something has to be typed. 16,758 stars none of them makes Opus smarter. every one stops it from paying for work it never needed to do the full setup with scripts is in the article. do the measuring step first, so you know what you actually saved

rvaniaaa

24,787 görüntüleme • 25 gün önce

Opus 4.7 - 400k vs 1m context - is there a difference? I've heard Theo - t3.gg talk about the fact that it is unlikely that Anthropic would have offered up a model with 1m context at the same cost, if it wasn't a different (i.e. cheaper to serve) model. I did a test where I toggled the 1m default model on & off in Claude Code (otherwise default settings, xHigh reasoning) and compared the outputs with 3x generations - same prompts etc. My observations: - Models feel DIFFERENT - often when you ask a model for the same generation, you get a somewhat different answer, but it feels & smells the same. Here 400k and 1m are very different every time - 400k model seems better - not that 1m is trash and 400k is amazing, but there are definitely issues with the level of ambition and accuracy that 1m model seems to have Examples of 1m failing: - Voxel Rome: the colosseum is nowhere near as impressive - Golden Gate: cars go sideways, waves not very high, bridge goes into land; though the structure of the bridge is a bit better - Stonehenge: structure is more 'wrong', lighting, shadows & textures are more flat and not as rich This isn't a conclusive evidence of course, but at least to me the two models do not behave the same way. Anecdotally as well when building 1m felt like it was doing more weird validation (e.g. going around in circles) and 400k was more straightforward. These sorts of things are harder to capture in tests, but you'd notice in Claude Code. You can review the hosted generations, see the code & prompts in the links below

Peter Gostev (SF: 22-26 June)

29,203 görüntüleme • 6 ay önce

Sam Altman made the case for open-source harnesses in July. a month later, someone shipped it, and it's more efficient than most managed harnesses. here is the problem it was aimed at: a large share of your agent's token bill is the model rereading things it already read. that isn't the model's doing. the runtime around it decides what goes into every prompt and how often the model gets called. for example, an agent queries a CRM at step four and gets back 400 rows. those rows get piled up in the conversation history. by step nineteen, the model has to read those rows fifteen times unnecessarily, and every token read is billed at input rates. it happened because your harness assembled that prompt on every turn and kept the rows in it. that gives you two levers: how much context the harness carries forward, and how often it calls the model. there are four practical ways to keep the prompt from growing unnecessarily: → load tool schemas on demand. a server with 100 tools doesn't need to put all 100 into every prompt when the agent only calls two. → offload large results to disk. turn a large response into a short preview and a file path instead of replaying the entire result on every turn. → delegate to subagents. let a subagent spend thirty tool calls in its own context and return one summary to the root agent. → run toolchains in code. one script calls three tools, joins the results, and returns a table instead of three turns each dragging a full response. but reducing context is only half the job. you also need to control how often the model gets called. a good harness should avoid unnecessary planning, verification, and reflection when the work can be completed in fewer steps. TrueFoundry's open-source agent harness, TrueForge, is built around both of those controls. it sits between the model and the tools, deciding what goes into every prompt and when another model call is actually needed. it also breaks token usage down across the harness, skills, instructions, tools, and messages. DevRev's Enterprise-Bench is where this gets tested, on multi-step tasks of the kind where an agent pulls records from one system and reconciles them against another. TrueFoundry ran TrueForge there against Claude Managed Agents, both on the same model, and both finished the same number of tasks. the tie is the part that matters, because it means the gap underneath is not a quality tradeoff. TrueForge reached that score on close to a third of the tokens, with roughly 40% fewer trips back to the model. for the same result, that comes out around 2.7x cheaper than Claude Managed Agents. swapping in an open model made it sharper still. TrueForge with GLM-5.2 scored a little higher than either setup above, and the entire benchmark run cost about $3 at list prices. being open source matters beyond the license here. the model underneath can be swapped without rewriting the agent, and the whole thing can run inside your own environment when the data cannot leave it. all of this comes down to the runtime around the model, the context it carries, the tools it exposes, and how many times it goes back to the model. that is what a production harness actually owns. the full task list, the per-run numbers, and the MIT-licensed code are on GitHub: (don't forget to star 🌟) you can read more about the same in the article quoted below. thanks to the TrueForge team for working with me on this one.

Akshay 🚀

76,998 görüntüleme • 2 ay önce

Qwen3.8-Flash-Next is still going strong at 364.7K tokens of context on an M5 Max. And this isn’t just a static long-context test. The model was reasoning about how to speed up its own workflow while using tools, and the tool calls kept working without misses. Setup: • Qwen3.8-Flash-Next • M5 Max • 128GB unified memory • MLX-Serve PR #363 • OpenCode 2 • 364.7K context The interesting part isn’t simply getting hundreds of thousands of tokens into memory. It’s what happens once the context gets this large. Long-context inference usually comes with a painful tradeoff. As the KV cache grows, memory pressure increases and generation can slow down. But this setup is still pushing through 364K tokens while maintaining a usable agent workflow. The model can reason, call tools, inspect results, continue working, and keep the session moving. And the tool calls reportedly haven’t missed so far. That’s important for agentic coding. A huge context window is only useful if the model can actually operate reliably inside it. A 400K-token context that constantly breaks tool calls isn’t very useful. A 364K session that can keep reasoning and executing tools is a different story. And the test isn’t finished yet. The current run is approaching 400K tokens, with the expectation that it can keep going. This is also another interesting example of why Apple Silicon keeps showing up in local LLM experiments. The M5 Max’s unified memory gives a large model and its growing KV cache access to one shared memory pool. With MLX-Serve continuing to improve, these machines are becoming surprisingly capable long-context inference boxes. The bigger takeaway: Context length is becoming a workload, not just a model specification. Running a model at 256K is one thing. Keeping an agent alive at 300K+ while it reasons and uses tools is much more interesting. And Qwen3.8-Flash-Next is showing that this can be pushed surprisingly far on a single 128GB Mac. 364.7K and counting. Next stop: 400K.

FHILY👑

39,982 görüntüleme • 2 ay önce

your agent reviewing its own work is not a check. it is a second opinion from the same source. this is the most common gap in agent systems and it hides in plain sight, because the step exists. there is a review. it just cannot do the thing you think it does. here is the mechanism. the model produced an output from a context. you then ask the same model, holding the same context, whether that output is correct. it answers fluently, because that is what it does. and the answer is drawn from the same distribution that produced the thing being judged. same weights, same window, same blind spots. if the reason the output is wrong is something the model does not know, the review does not know it either. if the reason is something the context does not contain, the review has the same context. the failure mode and the detector share a cause. > why it feels like it works because most of the time the output is fine, and the review says fine. agreement is not evidence of detection. a reviewer that says pass on everything agrees with reality most of the time too. what you actually want to measure is what happens on the cases that are wrong. that is the only place a check earns its name, and it is exactly the place where a self-review is weakest. there is research on this. Huang and colleagues at DeepMind showed at ICLR 2024 that intrinsic self-correction, revising without external grounding, does not reliably help and often makes things worse. > what to actually do move the check outside the model. a test that runs, a schema that validates, a file that exists or does not, an exit code from something you did not write. these are not smarter than the model. they are just not correlated with it, and that is the entire value. when the judgement genuinely needs a model, at minimum use a different family. same family means shared blind spots, and frontier judges measurably inflate scores for outputs that look like their own. and split the work by kind. anything objectively checkable goes to code. only the genuinely semantic calls go to a judge, and those get a rubric written as one line. a review inside the loop tells you the model is confident. a check outside it tells you whether the work is done. save this - then read the eval setup below

Hanako

14,325 görüntüleme • 3 ay önce

i been running Qwen3.5-35B-A3B UD-Q4_K_XL through Claude Code since llama.cpp merged the Anthropic endpoint. configured it in minutes. everything was great. projects grew from single scripts to multifile systems with 8 modules and 3,000+ lines. then the chains started breaking. 3 to 5 minutes of pure autonomy and suddenly it stops. tool call fails. reprompt. it recovers. 2 minutes later it stops again. the model is fine. the harness is the bottleneck. saw a comment suggesting OpenCode. installed it. pointed it at the same localhost endpoint running the same model on the same GPU. the game is different. instead of stopping on a bad tool call it just keeps going. on wrong read it adjusts. if file not found it retries. the flow is unbroken. i watched it plan a refactor across 8 files, read every module, and start building without a single pause. in Claude Code that same task would have stopped 4 times. the tradeoff is sometimes it loops. same tool call repeated because the model loses track of what it already read. but here is the thing. i choose loops over pauses. a loop you can interrupt and redirect. a broken chain stops the flow and you have to reprompt to get it moving again. someone is solving this at the core level and i have a feeling it is the open source community. the fact that i can run this level of autonomous coding intelligence on a single consumer GPU with 24gb VRAM at 112 tokens per second. respect to the chinese labs. respect to the open source builders making this possible.

Sudo su

67,104 görüntüleme • 8 ay önce