Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Opus 5, Max, Fast: Prompts: 8 Input (cache write): 1,181,215 Input (no cache): 227,275 Cache read: 70,291,242 Output: 302,449 Total tokens: 72,002,181 Cost (USD): $117.7400 Most of the money was spent on trying to get the body and the swing not to look insane. I had to link like... show more

ThePrimeagen

384,676 subscribers

154,705 Aufrufe • vor 2 Monaten •via X (Twitter)

Kunst Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

35 Kommentare

Profilbild von ThePrimeagen

ThePrimeagenvor 2 Monaten

also i love that the ball gets relatively bigger to my body the further i hit the ball. Somehow that just makes me so happy

Profilbild von Michael🧙

Michael🧙vor 2 Monaten

Woh, thats crazy. you know back in 2002 this game would have made $5,000,000,000,000 on steam and now you can make it for $117

Profilbild von Zachary Meyer

Zachary Meyervor 2 Monaten

wow dude 5 yrs ago this would have been worth $15m total at least and you did this for only $117???

Profilbild von Barutoi

Barutoivor 2 Monaten

Is this supposed to be good or bad? Looks like wasted time and money to me.

Profilbild von Rovshan

Rovshanvor 2 Monaten

Just don't tell your wife on what you spent this $117

Profilbild von Jason

Jasonvor 2 Monaten

I just one shotted that shit with Gemini 3 flash preview for $0.02. Output Tokens: 5,729 Total Tokens: 19,589 Total Cost (USD): $0.021740

Profilbild von waterfowl

waterfowlvor 2 Monaten

there's something sad about the fact people are spending 72 million tokens on what is essentially a crappy flash game from 2001

Profilbild von Levi

Levivor 2 Monaten

“Vibe coding” prompts? Or were you in the loop?

Profilbild von ThePrimeagen

ThePrimeagenvor 2 Monaten

/loop

Profilbild von Nate Wilson

Nate Wilsonvor 2 Monaten

The future is here! We can finally make 2005 flash games again. Thank you AI overlords!

Profilbild von montano

montanovor 2 Monaten

it’s so over

Profilbild von Steve Brownlie

Steve Brownlievor 2 Monaten

and yet this is what golf games looked like on donkey hardware 36 years before ai...

Profilbild von ink

inkvor 2 Monaten

i can't overstate how bad this looks

Profilbild von Ahmad Awais

Ahmad Awaisvor 2 Monaten

@naymur_dev time to do this with open models using @CommandCodeAI :P

Profilbild von keccak

keccakvor 2 Monaten

You know back in the day you could have made it for free over a period of months and no one would have cared. Now you spent 117 $ over a few hours and no one cares.

Profilbild von Michael Lulev

Michael Lulevvor 2 Monaten

Try creating the same thing using Grok 4.5 high in Grok Build. Compare the time it takes and the cost.

Profilbild von Brandon Falk

Brandon Falkvor 2 Monaten

Please god tell me you're making a golf game but with qwop controls for the golfers arms/legs/swing.

Profilbild von Aman Shaikh 🇮🇳

Aman Shaikh 🇮🇳vor 2 Monaten

Wasted time and money I would prefer Opus 5 and Fable 5 for prototypes not for the final product Also geni 3 is good too

Profilbild von David

Davidvor 2 Monaten

Its only me or is 117$ a lot of money?

Profilbild von Michael Swengel

Michael Swengelvor 2 Monaten

And this seemed like a good use of money?

Profilbild von Chandraprakash Darji

Chandraprakash Darjivor 2 Monaten

$117, is more than my infra bill. See this kind of ui/frontend/design with AI, I feel like AI is very far behind at building frontend.

Profilbild von troneras

tronerasvor 2 Monaten

Can I add your game as an Easter egg in my app website? It’s about golf. I would keep this like an art piece generated by @ThePrimeagen Now that anyone can build, like painting, we may get more people that admires good craftmanship…

Profilbild von Nat Miletic

Nat Mileticvor 2 Monaten

It's these dumb things usually that eat up all of the tokens I swear LOL

Profilbild von Nicholas Gilmet

Nicholas Gilmetvor 2 Monaten

Love all your stuff, Prime. No hate from me here; I default to agreeing with your takes on most things. Just wanna ask: is this you saying “look how good this model is!” Or “look how bad this model did here!” ? I can’t tell. I could just be dumb tho

Profilbild von Prasenjit Sarkar

Prasenjit Sarkarvor 2 Monaten

Campfire ( ) generates a cost card per session: backend, model, token breakdown, USD total. Shareable with the team. Cost-cascade mode runs the same task across multiple backends in isolated worktrees, so you can see what each one actually costs before committing.

Profilbild von Mefa

Mefavor 2 Monaten

Lmfao what is this slop 😭

Profilbild von Mikhail Rogov

Mikhail Rogovvor 2 Monaten

72M tokens for $118. Cache reads completely change what token count means.

Profilbild von Carlos

Carlosvor 2 Monaten

this is where I am puzzled; I see this slop and I am like well those are similar results to what I get; but then I see in X some people claiming they one shot a Call of Duty with 3d?! someone must be lying 😛

Profilbild von relentless.build

relentless.buildvor 2 Monaten

People: "Look at this absurd golf thing I made for $117!" Also people: "$90 for GTA 6 is too expensive!"

Profilbild von intern

internvor 2 Monaten

$117? That’s wild

Profilbild von Tor Tor

Tor Torvor 2 Monaten

I have a 20 subscription Used cliproxyapi To serve my own agent Did a similar test run yesterday Total token cap set to 10m It works fine, not even trigger the 5h limit. API price is not for individuals...

Profilbild von Jorge

Jorgevor 2 Monaten

I think at this point the best AI workflow pipeline is scaffold stuff → tweak a few things → ce-simplify-code → ask the AI to explain to you the codebase and get back to tweak a few things manually The smallest details you want changed live in a few easily identifiable files, it makes no sense to get an AI to take like 3 minutes for a 20 second surgical change you can Ctrl-F / rg

Profilbild von Tasa - 🇪🇸 🏴‍☠️

Tasa - 🇪🇸 🏴‍☠️vor 2 Monaten

Yeah you'd need some skills you know.

Profilbild von Alexandru Bogdan

Alexandru Bogdanvor 2 Monaten

Are you content with using this AI model?

Profilbild von Amadej Jerlah

Amadej Jerlahvor 2 Monaten

hope there's room for 1 million ads

Ähnliche Videos

How to cut your AI bill by 60% switching to Opus 5.5: Most people will swap the model name, save 24%, and stop there. The other 37% is sitting in your settings. Here's the real before and after on an example agent workload. One month: 100M input tokens (80M cached reads, 5M cache writes, 15M uncached) 10M output tokens BEFORE: Opus 5 Cache reads: 80M × $0.50 = $40 Cache writes: 5M × $6.25 = $31.25 Uncached input: 15M × $5 = $75 Output: 10M × $25 = $250 Total: $396.25 STEP 1. Just switch the model. Same tokens, new prices. Cache reads $0.20. Writes $5. Input $4. Output $20. $16 + $25 + $60 + $200 = $301 24% cheaper. That's what everyone's screenshotting. STEP 2. Let it write less. Box measured Opus 5.5 at 40% less verbose with no drop in accuracy. 10M output tokens becomes 6M. Output drops from $200 to $120. Total: $221. Now you're at 44% off. STEP 3. Stop running every call at max effort. Thinking can't be switched off on 5.5 anymore, so effort is your lever. Classifying, routing, formatting, summarizing? Drop the effort. Save the high settings for the calls that actually reason. Say that trims output another 25%, 6M down to 4.5M. Output: $90. Total: $191. 52% off. STEP 4. Cache the stuff you keep resending. This is the one nobody does. Cache reads went from $0.50 to $0.20. That's 60% off the cheapest line on your bill. Move your system prompt, tool definitions and repo context into the cache. Uncached input drops from 15M to 5M, cache reads go up to 90M. $18 + $25 + $20 + $90 = $153 AFTER: $153 From $396.25. 61% cheaper. Same work. The model gave you 24%. You gave yourself the other 37%. Anthropic's own number is 40% cheaper than Opus 5 on typical workloads. Your number depends on how much of your bill is output and how much of your prompt you're resending uncached every call. So check those two first. Bookmark this for when you migrate. follow CyrilXBT

How to cut your AI bill by 60% switching to Opus 5.5: Most people will swap the model name, save 24%, and stop there. The other 37% is sitting in your settings. Here's the real before and after on an example agent workload. One month: 100M input tokens (80M cached reads, 5M cache writes, 15M uncached) 10M output tokens BEFORE: Opus 5 Cache reads: 80M × $0.50 = $40 Cache writes: 5M × $6.25 = $31.25 Uncached input: 15M × $5 = $75 Output: 10M × $25 = $250 Total: $396.25 STEP 1. Just switch the model. Same tokens, new prices. Cache reads $0.20. Writes $5. Input $4. Output $20. $16 + $25 + $60 + $200 = $301 24% cheaper. That's what everyone's screenshotting. STEP 2. Let it write less. Box measured Opus 5.5 at 40% less verbose with no drop in accuracy. 10M output tokens becomes 6M. Output drops from $200 to $120. Total: $221. Now you're at 44% off. STEP 3. Stop running every call at max effort. Thinking can't be switched off on 5.5 anymore, so effort is your lever. Classifying, routing, formatting, summarizing? Drop the effort. Save the high settings for the calls that actually reason. Say that trims output another 25%, 6M down to 4.5M. Output: $90. Total: $191. 52% off. STEP 4. Cache the stuff you keep resending. This is the one nobody does. Cache reads went from $0.50 to $0.20. That's 60% off the cheapest line on your bill. Move your system prompt, tool definitions and repo context into the cache. Uncached input drops from 15M to 5M, cache reads go up to 90M. $18 + $25 + $20 + $90 = $153 AFTER: $153 From $396.25. 61% cheaper. Same work. The model gave you 24%. You gave yourself the other 37%. Anthropic's own number is 40% cheaper than Opus 5 on typical workloads. Your number depends on how much of your bill is output and how much of your prompt you're resending uncached every call. So check those two first. Bookmark this for when you migrate. follow CyrilXBT

CyrilXBT

15,803 Aufrufe • vor 25 Tagen

Kimi K3: free to use with zero cost, or $5 in for $10 to spend if you want the API. Here's what's actually on the account page: • Balance on signup - $0.00, no free trial credit • First recharge bonus - 100% match, one time only. Put in $5, get $10 to spend • Pricing once you're spending - $3/1M input, $15/1M output, $0.30/1M on cache hits • Free alternative - no card, no recharge needed No credit card required to test the chat. That 10x gap between cache hit ($0.30) and cache miss ($3.00) input pricing adds up fast on long-context agents. Structure prompts to reuse cached context before you scale usage.

Kimi K3: free to use with zero cost, or $5 in for $10 to spend if you want the API. Here's what's actually on the account page: • Balance on signup - $0.00, no free trial credit • First recharge bonus - 100% match, one time only. Put in $5, get $10 to spend • Pricing once you're spending - $3/1M input, $15/1M output, $0.30/1M on cache hits • Free alternative - no card, no recharge needed No credit card required to test the chat. That 10x gap between cache hit ($0.30) and cache miss ($3.00) input pricing adds up fast on long-context agents. Structure prompts to reuse cached context before you scale usage.

kozh ./

14,927 Aufrufe • vor 2 Monaten

I asked Opus 5.5 to show me what's inside the buildings it runs in it came up with this after 1 hour 53 minutes in one shot, $38.99 API cost it's fully interactive 3D: overload a rack to watch its GPUs throttle, then follow the heat out the roof -- The API cost breakdown is the interesting part: 99% of the input was cache hits Every step resends the whole session. That's 202 calls and 119M input tokens, almost all read from cache at 1/20th of the price. Without caching the same run would have cost about $486 It also read my earlier projects on disk to match the style we'd built in other sessions. So it really is one prompt from idea to finished site, with no follow-ups until I asked to deploy /usage breakdown below

I asked Opus 5.5 to show me what's inside the buildings it runs in it came up with this after 1 hour 53 minutes in one shot, $38.99 API cost it's fully interactive 3D: overload a rack to watch its GPUs throttle, then follow the heat out the roof -- The API cost breakdown is the interesting part: 99% of the input was cache hits Every step resends the whole session. That's 202 calls and 119M input tokens, almost all read from cache at 1/20th of the price. Without caching the same run would have cost about $486 It also read my earlier projects on disk to match the style we'd built in other sessions. So it really is one prompt from idea to finished site, with no follow-ups until I asked to deploy /usage breakdown below

Ryan Sael

78,696 Aufrufe • vor 25 Tagen

I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x. All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework. On a 4-token prompt with 252 generated tokens: - Original: 0.76 tok/s - KV cache fp32: 27.21 tok/s - KV cache int8 (quantized): 27.29 tok/s Try it out yourself here: In practice: - KV caching gave us about a 35x end-to-end speedup - INT8 KV cache kept roughly the same speed as fp32 but cut KV cache memory by 3.78x FP32 cache used 4.5 MB in this run while the INT8 cache used only 1.19 MB This simple change to inference created a huge impact on performance. To learn more about the KV cache and other optimizations like this, check out the blog at

I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x. All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework. On a 4-token prompt with 252 generated tokens: - Original: 0.76 tok/s - KV cache fp32: 27.21 tok/s - KV cache int8 (quantized): 27.29 tok/s Try it out yourself here: In practice: - KV caching gave us about a 35x end-to-end speedup - INT8 KV cache kept roughly the same speed as fp32 but cut KV cache memory by 3.78x FP32 cache used 4.5 MB in this run while the INT8 cache used only 1.19 MB This simple change to inference created a huge impact on performance. To learn more about the KV cache and other optimizations like this, check out the blog at

Reese Chong

53,026 Aufrufe • vor 6 Monaten

Jev + Opus 5.5: Anthropic's new model beats GPT-6 Astra for 1/5 the cost, and 4 API changes will 400 your agent before it writes a single line I pulled these 10 steps from the migration docs so you don't learn them in production step 1 → $4 / $20 per 1M. Opus 5 was $5 / $25. cache reads dropped from $0.50 to $0.20 step 2 → 66.4% on Terminal-Bench 4.0 vs GPT-6 Astra 57.9% and Opus 5 52.3%. +14.1 points in one release, and on FrontierCode it beats Astra at default effort for 1/5 the cost step 3 → thinking can't be turned off anymore. send thinking: disabled and you get a 400. drop the field, set effort step 4 → tool_choice any and tool are gone. 400. switch to auto + strict step 5 → edit anything above a thinking block and the request dies. append only, or opt into drop_block step 6 → computer_20251124 is dead on the API. 400. move to computer_toolset_20260801 step 7 → the quiet one: default effort fell from high to medium. your agent thinks less than you set it up to and nothing tells you step 8 → hop Opus 5.5 → Sonnet 5 → Opus 5.5 and you pay 4.36 instead of 3.32. +31%, the cache dies and Sonnet can't read Opus's reasoning step 9 → change effort at the top of the request and the cache is gone. Jev sets it per message and the cache stays step 10 → switch fast - standard mid-session and it's a full cache miss. Jev picks speed once, on turn one one model, three knobs, zero 400s. that is Jev + Opus 5.5 send this to your Claude Code before you touch the model ID, then read my full Jev deep dive in the article below ↓

Jev + Opus 5.5: Anthropic's new model beats GPT-6 Astra for 1/5 the cost, and 4 API changes will 400 your agent before it writes a single line I pulled these 10 steps from the migration docs so you don't learn them in production step 1 → $4 / $20 per 1M. Opus 5 was $5 / $25. cache reads dropped from $0.50 to $0.20 step 2 → 66.4% on Terminal-Bench 4.0 vs GPT-6 Astra 57.9% and Opus 5 52.3%. +14.1 points in one release, and on FrontierCode it beats Astra at default effort for 1/5 the cost step 3 → thinking can't be turned off anymore. send thinking: disabled and you get a 400. drop the field, set effort step 4 → tool_choice any and tool are gone. 400. switch to auto + strict step 5 → edit anything above a thinking block and the request dies. append only, or opt into drop_block step 6 → computer_20251124 is dead on the API. 400. move to computer_toolset_20260801 step 7 → the quiet one: default effort fell from high to medium. your agent thinks less than you set it up to and nothing tells you step 8 → hop Opus 5.5 → Sonnet 5 → Opus 5.5 and you pay 4.36 instead of 3.32. +31%, the cache dies and Sonnet can't read Opus's reasoning step 9 → change effort at the top of the request and the cache is gone. Jev sets it per message and the cache stays step 10 → switch fast - standard mid-session and it's a full cache miss. Jev picks speed once, on turn one one model, three knobs, zero 400s. that is Jev + Opus 5.5 send this to your Claude Code before you touch the model ID, then read my full Jev deep dive in the article below ↓

Carnage

16,674 Aufrufe • vor 26 Tagen

Redis built a cache that cuts LLM costs by 90%! Production LLM apps do not receive completely new questions every time. A customer-support assistant might receive all three of these: - "Can I get a refund after buying the monthly plan?" - "Is the monthly subscription refundable?" - "Can I cancel the plan and get my money back?" The wording is different, but the underlying question and its answer remain the same. Yet LLM apps process every version as a new request. They assemble the prompt, send it to the model, and generate an answer that may have already been generated. Prefix caching reduces part of these repeated calls. When requests begin with the same system prompt or context, the model can reuse the KV states already computed for that shared prefix. But the request still hits the LLM. The new tokens must be processed, and the complete answer must still be decoded. So even with a prefix-cache hit, there's another generation call involved. To solve this, instead of only caching computation inside the model, the application can cache the generated response outside it. When another question arrives, the system embeds it and compares it with previously answered questions. If it finds a sufficiently close match, it returns the stored response without invoking the LLM again. A cache hit removes the input tokens, output tokens, and decoding time associated with another LLM call. In practice, it is important to decide which questions can safely share an answer since a production setup needs well-tuned similarity thresholds, expiration policies, data isolation, and monitoring for incorrect matches. If you want to use this in practice, Redis already implements it as a managed service called Redis LangCache. Under the hood, it generates embeddings, searches previous responses, and returns a matching answer before another model call occurs. Redis also handles access scopes, custom filtering, TTL and eviction controls, and cache monitoring through Redis Cloud. I built an interface to compare it against direct LLM inference. The video below shows this in action, and I worked with Redis on this post to put this together. For the paraphrased question in my run, direct inference took 2.232 seconds and consumed 514 input tokens plus 250 output tokens. Redis returned the earlier response in 0.37 seconds with zero LLM input or output tokens. That was roughly 6x faster in this run. Redis reports API cost savings of up to 90% and cache-hit responses up to 15x faster. The actual result depends on how much safe repetition exists in the workload. You can try Redis LangCache here: If you want to dive deeper, I have already written a detailed breakdown of KV, prefix, prompt, and semantic caching in the article quoted below. This demo builds on the final technique and shows it running in practice. Read it below.

Redis built a cache that cuts LLM costs by 90%! Production LLM apps do not receive completely new questions every time. A customer-support assistant might receive all three of these: - "Can I get a refund after buying the monthly plan?" - "Is the monthly subscription refundable?" - "Can I cancel the plan and get my money back?" The wording is different, but the underlying question and its answer remain the same. Yet LLM apps process every version as a new request. They assemble the prompt, send it to the model, and generate an answer that may have already been generated. Prefix caching reduces part of these repeated calls. When requests begin with the same system prompt or context, the model can reuse the KV states already computed for that shared prefix. But the request still hits the LLM. The new tokens must be processed, and the complete answer must still be decoded. So even with a prefix-cache hit, there's another generation call involved. To solve this, instead of only caching computation inside the model, the application can cache the generated response outside it. When another question arrives, the system embeds it and compares it with previously answered questions. If it finds a sufficiently close match, it returns the stored response without invoking the LLM again. A cache hit removes the input tokens, output tokens, and decoding time associated with another LLM call. In practice, it is important to decide which questions can safely share an answer since a production setup needs well-tuned similarity thresholds, expiration policies, data isolation, and monitoring for incorrect matches. If you want to use this in practice, Redis already implements it as a managed service called Redis LangCache. Under the hood, it generates embeddings, searches previous responses, and returns a matching answer before another model call occurs. Redis also handles access scopes, custom filtering, TTL and eviction controls, and cache monitoring through Redis Cloud. I built an interface to compare it against direct LLM inference. The video below shows this in action, and I worked with Redis on this post to put this together. For the paraphrased question in my run, direct inference took 2.232 seconds and consumed 514 input tokens plus 250 output tokens. Redis returned the earlier response in 0.37 seconds with zero LLM input or output tokens. That was roughly 6x faster in this run. Redis reports API cost savings of up to 90% and cache-hit responses up to 15x faster. The actual result depends on how much safe repetition exists in the workload. You can try Redis LangCache here: If you want to dive deeper, I have already written a detailed breakdown of KV, prefix, prompt, and semantic caching in the article quoted below. This demo builds on the final technique and shows it running in practice. Read it below.

Avi Chawla

166,230 Aufrufe • vor 1 Monat

Qwen3.8 Flash Next built a whole shooter from one prompt. I hit Enter and left. omp with nothing but that model wired in. no follow ups, no me watching it. Hours later I opened the logs. It was taking screenshots of its own game, looking at them, and going back to fix what looked off. I never asked for that. The lighting is the part I did not expect. This is just an incredibly beautiful and voluminous light. What one prompt cost: 5h 22m wall time 70M tokens read from cache 2.9M fresh input 429K output, 301K of it reasoning Same prompt goes to glm-5.3 flash next. Then Ill know if that was the model or luck. Link in reply

Qwen3.8 Flash Next built a whole shooter from one prompt. I hit Enter and left. omp with nothing but that model wired in. no follow ups, no me watching it. Hours later I opened the logs. It was taking screenshots of its own game, looking at them, and going back to fix what looked off. I never asked for that. The lighting is the part I did not expect. This is just an incredibly beautiful and voluminous light. What one prompt cost: 5h 22m wall time 70M tokens read from cache 2.9M fresh input 429K output, 301K of it reasoning Same prompt goes to glm-5.3 flash next. Then Ill know if that was the model or luck. Link in reply

Alexey Fateev

22,032 Aufrufe • vor 1 Monat

Opus 5.5 being “2x more expensive” is one of those facts that dies the second you run a long agent loop [the loop changes the math] 1. at 20K context, Opus is still 1.88x Sonnet per turn 2. at 150K, that falls to 1.49x. by 400K, it is only 1.26x 3. the reason is cache. both models pay the same $0.20 / 1M for cache reads, so old context stops widening the bill 4. the ugly part is switching late. drag 300K of failed context into Opus and you spend $1.50 just rebuilding cache 5. catch the failure around turn 6 and hand off 20K instead. that costs about $0.10 at 150K, 30 Sonnet turns cost $1.76 20 Opus turns cost $1.75 the real optimization is not “use the cheaper model” it is knowing when to stop paying for the wrong one ↓

Opus 5.5 being “2x more expensive” is one of those facts that dies the second you run a long agent loop [the loop changes the math] 1. at 20K context, Opus is still 1.88x Sonnet per turn 2. at 150K, that falls to 1.49x. by 400K, it is only 1.26x 3. the reason is cache. both models pay the same $0.20 / 1M for cache reads, so old context stops widening the bill 4. the ugly part is switching late. drag 300K of failed context into Opus and you spend $1.50 just rebuilding cache 5. catch the failure around turn 6 and hand off 20K instead. that costs about $0.10 at 150K, 30 Sonnet turns cost $1.76 20 Opus turns cost $1.75 the real optimization is not “use the cheaper model” it is knowing when to stop paying for the wrong one ↓

Gipp 🦅

11,770 Aufrufe • vor 10 Tagen

someone just rebuilt claude code in pure python and made it 230x cheaper to run it's called clawcodex. a full from-scratch port of the claude code agent loop. 230k+ lines of python, MIT licensed it keeps your request prefix byte-stable, so DeepSeek's prompt cache covers your entire system + tools + history on every turn cache-hit input then bills at ~$0.0435 per 1M tokens the same 1M tokens on Claude Fable 5 runs you $10 so the longer you code, the more the cache pays off what you actually get: → 25 LLM providers (Claude, GPT, DeepSeek, GLM, local ollama) → 1M token context window → real agent loop: tools, skills, REPL, session history → one-line install, mac/linux/wsl and it's sitting at 860 stars. basically nobody is looking yet bookmark this before it gone

someone just rebuilt claude code in pure python and made it 230x cheaper to run it's called clawcodex. a full from-scratch port of the claude code agent loop. 230k+ lines of python, MIT licensed it keeps your request prefix byte-stable, so DeepSeek's prompt cache covers your entire system + tools + history on every turn cache-hit input then bills at ~$0.0435 per 1M tokens the same 1M tokens on Claude Fable 5 runs you $10 so the longer you code, the more the cache pays off what you actually get: → 25 LLM providers (Claude, GPT, DeepSeek, GLM, local ollama) → 1M token context window → real agent loop: tools, skills, REPL, session history → one-line install, mac/linux/wsl and it's sitting at 860 stars. basically nobody is looking yet bookmark this before it gone

alex

14,212 Aufrufe • vor 1 Monat

someone just rebuilt claude code in pure python and made it 230x cheaper to run it's called clawcodex. a full from-scratch port of the claude code agent loop. 230k+ lines of python, MIT licensed it keeps your request prefix byte-stable, so DeepSeek's prompt cache covers your entire system + tools + history on every turn cache-hit input then bills at ~$0.0435 per 1M tokens the same 1M tokens on Claude Fable 5 runs you $10 so the longer you code, the more the cache pays off what you actually get: → 25 LLM providers (Claude, GPT, DeepSeek, GLM, local ollama) → 1M token context window → real agent loop: tools, skills, REPL, session history → one-line install, mac/linux/wsl and it's sitting at 677 stars. basically nobody is looking yet bookmark this before it gone

someone just rebuilt claude code in pure python and made it 230x cheaper to run it's called clawcodex. a full from-scratch port of the claude code agent loop. 230k+ lines of python, MIT licensed it keeps your request prefix byte-stable, so DeepSeek's prompt cache covers your entire system + tools + history on every turn cache-hit input then bills at ~$0.0435 per 1M tokens the same 1M tokens on Claude Fable 5 runs you $10 so the longer you code, the more the cache pays off what you actually get: → 25 LLM providers (Claude, GPT, DeepSeek, GLM, local ollama) → 1M token context window → real agent loop: tools, skills, REPL, session history → one-line install, mac/linux/wsl and it's sitting at 677 stars. basically nobody is looking yet bookmark this before it gone

alex

158,896 Aufrufe • vor 2 Monaten

During an agentic session, a quick “hi” after a coffee break can cost a full dollar. But it’s not because of the “hi.” It’s the million tokens of context you’re repaying for the moment your cache goes cold. Here's the caching math from Evan: 0:00 Why you're billed for last turn's context, not just this one 0:17 Cache hits: ~10% of full price 0:41 Agentic AI = way more back-and-forth than chat 1:16 Paying full price for context even on a 1-token tool call 1:40 Cache hit vs. cache miss 2:13 The $1 "hi": walk away for an hour, come back to a cold cache 2:59 Cost grows by the square, even with caching 3:28 How context compaction helps 4:03 Why Claude Code/Codex may compact at ~200-300K, not the full 1M window

During an agentic session, a quick “hi” after a coffee break can cost a full dollar. But it’s not because of the “hi.” It’s the million tokens of context you’re repaying for the moment your cache goes cold. Here's the caching math from Evan: 0:00 Why you're billed for last turn's context, not just this one 0:17 Cache hits: ~10% of full price 0:41 Agentic AI = way more back-and-forth than chat 1:16 Paying full price for context even on a 1-token tool call 1:40 Cache hit vs. cache miss 2:13 The $1 "hi": walk away for an hour, come back to a cold cache 2:59 Cost grows by the square, even with caching 3:28 How context compaction helps 4:03 Why Claude Code/Codex may compact at ~200-300K, not the full 1M window

Arena.ai

53,273 Aufrufe • vor 2 Monaten

someone just rebuilt claude code in pure python and made it ~230x cheaper to run it's called clawcodex. a full from-scratch port of the claude code agent loop. 230k+ lines of python, MIT licensed it keeps your request prefix byte-stable, so DeepSeek's prompt cache covers your entire system + tools + history on every turn cache-hit input then bills at ~$0.0435 per 1M tokens the same 1M tokens on Claude Fable 5 runs you $10 so the longer you code, the more the cache pays off what you actually get: → 25 LLM providers (Claude, GPT, DeepSeek, GLM, local ollama) → 1M token context window → real agent loop: tools, skills, REPL, session history → one-line install, mac/linux/wsl and it's sitting at 677 stars. basically nobody is looking yet bookmark this before it gone

someone just rebuilt claude code in pure python and made it ~230x cheaper to run it's called clawcodex. a full from-scratch port of the claude code agent loop. 230k+ lines of python, MIT licensed it keeps your request prefix byte-stable, so DeepSeek's prompt cache covers your entire system + tools + history on every turn cache-hit input then bills at ~$0.0435 per 1M tokens the same 1M tokens on Claude Fable 5 runs you $10 so the longer you code, the more the cache pays off what you actually get: → 25 LLM providers (Claude, GPT, DeepSeek, GLM, local ollama) → 1M token context window → real agent loop: tools, skills, REPL, session history → one-line install, mac/linux/wsl and it's sitting at 677 stars. basically nobody is looking yet bookmark this before it gone

Argona

51,369 Aufrufe • vor 3 Monaten

The input code for LAN Mode in Mario Kart World is the same as 8 Deluxe. On the main menu, hold L+R (Not ZL/ZR) & Click in the left stick! Connect the consoles to same access point like a router (wired/wireless) to get a full 24 player LAN Party, unlike wireless play's max of 8.

The input code for LAN Mode in Mario Kart World is the same as 8 Deluxe. On the main menu, hold L+R (Not ZL/ZR) & Click in the left stick! Connect the consoles to same access point like a router (wired/wireless) to get a full 24 player LAN Party, unlike wireless play's max of 8.

Mankalor

552,260 Aufrufe • vor 1 Jahr

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

Avi Chawla

273,779 Aufrufe • vor 3 Monaten

What does it cost to sort 1,700 emails with Jev? I gave JEV each full email object: - Subject - body - sender No special changes. Jev sent back 4 values for each email: - Category: work, shopping, finance, security - Priority: low to urgent - Spam score: a percentage - Reply score: how much the email needs a reply from me One user wrote that their account had a violation. JEV gave it a 90% reply score. The run used 4.2 million input tokens and 500,000 output tokens. The final cost was 18 cents. The total time was 12 seconds.

What does it cost to sort 1,700 emails with Jev? I gave JEV each full email object: - Subject - body - sender No special changes. Jev sent back 4 values for each email: - Category: work, shopping, finance, security - Priority: low to urgent - Spam score: a percentage - Reply score: how much the email needs a reply from me One user wrote that their account had a violation. JEV gave it a 90% reply score. The run used 4.2 million input tokens and 500,000 output tokens. The final cost was 18 cents. The total time was 12 seconds.

The Startup Ideas Podcast (SIP) 🧃

11,588 Aufrufe • vor 1 Monat

Interesting approach to long context/continual learning here from Baseten at Cursor Compact long trajectories by compressing a prefix of the KV cache using an MLP/autoencoder. You can train this "compactor" MLP by learning to reconstruct activations that the original KV cache would produce on subsequent tokens. This maximally reconstructs information from long context that's useful for subsequent outputs. If you run an agent over extremely long context and run this compaction recursively, the activations of this compressed KV cache become like trained weights. Similar to task-specific LoRA or "cartridges". Would not be surprised if OpenAI is running a similar algorithm for their blackbox compaction. Clear benefits here if you can avoid busting the cache, as e.g. compacting via writing to a text file would. Seeing many emerging approaches to fixed size latent states for LLMs that seem promising. If building a task-specific KV cache compression ends up being more sample efficient than running backprop, getting this to "work" feels like one of the 2-3 remaining breakthroughs on the path to true AGI

Interesting approach to long context/continual learning here from Baseten at Cursor Compact long trajectories by compressing a prefix of the KV cache using an MLP/autoencoder. You can train this "compactor" MLP by learning to reconstruct activations that the original KV cache would produce on subsequent tokens. This maximally reconstructs information from long context that's useful for subsequent outputs. If you run an agent over extremely long context and run this compaction recursively, the activations of this compressed KV cache become like trained weights. Similar to task-specific LoRA or "cartridges". Would not be surprised if OpenAI is running a similar algorithm for their blackbox compaction. Clear benefits here if you can avoid busting the cache, as e.g. compacting via writing to a text file would. Seeing many emerging approaches to fixed size latent states for LLMs that seem promising. If building a task-specific KV cache compression ends up being more sample efficient than running backprop, getting this to "work" feels like one of the 2-3 remaining breakthroughs on the path to true AGI

Jay Hack

30,031 Aufrufe • vor 3 Monaten

How is it possible that this swing can shoot around even par on a golf course I've never played or even seen before? Here's just a few god awful looking swings of mine due to the fact my back was in pain every shot... (lower back seized up) You know what I learned from playing golf crippled... I learned that you don't actually need to do much to play good golf & hit the golf ball where you want I was swinging the club no faster than 95mph (If I was lucky), I did however hit it pretty damn straight most of the time, got up & down a lot, & holed a few putts I wasn't trying to stop early extension. I wasn't trying to "drop it in the slot". I wasn't trying to "get open at impact (I couldn't). All I tried to do was swing my arms & control where the face was at impact I played better golf than 99% of golfers on the planet & I had no speed + a crippled lower back I think we need to ask ourselves what is truely required to play good golf. It actually doesn't take that much...

How is it possible that this swing can shoot around even par on a golf course I've never played or even seen before? Here's just a few god awful looking swings of mine due to the fact my back was in pain every shot... (lower back seized up) You know what I learned from playing golf crippled... I learned that you don't actually need to do much to play good golf & hit the golf ball where you want I was swinging the club no faster than 95mph (If I was lucky), I did however hit it pretty damn straight most of the time, got up & down a lot, & holed a few putts I wasn't trying to stop early extension. I wasn't trying to "drop it in the slot". I wasn't trying to "get open at impact (I couldn't). All I tried to do was swing my arms & control where the face was at impact I played better golf than 99% of golfers on the planet & I had no speed + a crippled lower back I think we need to ask ourselves what is truely required to play good golf. It actually doesn't take that much...

Ryan Mouque

173,483 Aufrufe • vor 4 Monaten

i spent 3 hours finding the sweet spot for hermes 4.3 36B on a single RTX 3090. saving you the trouble, anon. the model is 21.8GB at Q4_K_M. that leaves 2.2GB free on 24GB VRAM. not much room for KV cache. here's what actually happened: 4K: 35.3 tok/s 8K: 35.2 tok/s 16K: 34.8 tok/s 32K: 34.6 tok/s 64K: 6.4 tok/s 128K: 1.9 tok/s flat from 4K to 32K. then it falls off a cliff at 64K. the trick is quantized KV cache. without it you OOM at 16K. with quantized KV cache you get 32K at full speed. all 64 layers on GPU. at 64K something weird happens. ngl 99 (all layers on GPU) = 3.96 tok/s. the KV cache silently spills to CPU. drop to ngl 55 and speed jumps to 6.37. drop to ngl 48 and it gets worse again (3.46). there's an offload sweet spot where you free just enough VRAM for the cache without losing too much compute to PCIe transfers. 128K works at ngl 32 but you're at 1.95 tok/s. half the model on CPU. usable for batch work, not for interactive. the sweet spot command: llama-server -m hermes-4.3-36b-Q4_K_M.gguf -ngl 99 -c 32768 --cache-type-k q4_0 --cache-type-v q4_0 32K context. 34.6 tok/s. all on GPU. this is where dense 36B lives on 24GB. for comparison, qwen 3.5 (35B MoE, 3B active) holds 112 tok/s from 4K all the way to 262K on the same GPU. no speed drop. same total params, completely different architecture. hybrid linear attention means flat context scaling. dense pays for every token in the KV cache. code and quality comparison coming next. fast vs slow generation side by side in the videos below.

i spent 3 hours finding the sweet spot for hermes 4.3 36B on a single RTX 3090. saving you the trouble, anon. the model is 21.8GB at Q4_K_M. that leaves 2.2GB free on 24GB VRAM. not much room for KV cache. here's what actually happened: 4K: 35.3 tok/s 8K: 35.2 tok/s 16K: 34.8 tok/s 32K: 34.6 tok/s 64K: 6.4 tok/s 128K: 1.9 tok/s flat from 4K to 32K. then it falls off a cliff at 64K. the trick is quantized KV cache. without it you OOM at 16K. with quantized KV cache you get 32K at full speed. all 64 layers on GPU. at 64K something weird happens. ngl 99 (all layers on GPU) = 3.96 tok/s. the KV cache silently spills to CPU. drop to ngl 55 and speed jumps to 6.37. drop to ngl 48 and it gets worse again (3.46). there's an offload sweet spot where you free just enough VRAM for the cache without losing too much compute to PCIe transfers. 128K works at ngl 32 but you're at 1.95 tok/s. half the model on CPU. usable for batch work, not for interactive. the sweet spot command: llama-server -m hermes-4.3-36b-Q4_K_M.gguf -ngl 99 -c 32768 --cache-type-k q4_0 --cache-type-v q4_0 32K context. 34.6 tok/s. all on GPU. this is where dense 36B lives on 24GB. for comparison, qwen 3.5 (35B MoE, 3B active) holds 112 tok/s from 4K all the way to 262K on the same GPU. no speed drop. same total params, completely different architecture. hybrid linear attention means flat context scaling. dense pays for every token in the KV cache. code and quality comparison coming next. fast vs slow generation side by side in the videos below.

Sudo su

52,268 Aufrufe • vor 7 Monaten

> 20 free daily tokens covers a full working session on GPT 5.5 or GLM > Opus 4.7 handles 2-3 serious architecture tasks per day at zero cost > one GitHub account can link to multiple Tembo accounts, the math on that is obvious > 5 minutes to sign up. 10 minutes to your first merged PR > no clarifying questions, prompts have to be precise, the agent just executes > the $100/month tools and the $0 tool are now producing the same output on most tasks and most developers haven't noticed yet this is not a discount → this is the same models → the same → capability → the same output → for nothing!

> 20 free daily tokens covers a full working session on GPT 5.5 or GLM > Opus 4.7 handles 2-3 serious architecture tasks per day at zero cost > one GitHub account can link to multiple Tembo accounts, the math on that is obvious > 5 minutes to sign up. 10 minutes to your first merged PR > no clarifying questions, prompts have to be precise, the agent just executes > the $100/month tools and the $0 tool are now producing the same output on most tasks and most developers haven't noticed yet this is not a discount → this is the same models → the same → capability → the same output → for nothing!

Shadow Nick

26,539 Aufrufe • vor 4 Monaten

I spent 2.5 billion tokens, every limit I had and five days to make this. A GTA-style city, built with GPT-6 using only prompts. No code or 3D by hand. The whole point was to push it as far as it goes and wire in every API I had. The setup: Codex driving Unreal Engine 5, with Higgsfield, Tripo, Meshy, Seedance, Seed Audio and fal handling the buildings, characters, animations, materials and sound. 8 buildings taken from concept to game, 247 generated voice lines for residents and police, 10 radio tracks across 5 stations. Here's the honest part. Most of those tokens didn't go into building. They went into fixing. The model writes a mechanic, runs it, watches it break, and tries again for hours. Getting a character to open a car door and actually sit in the seat took an absurd number of passes. A carjacking needed two separate bodies to stay locked to the same grab. Half the work was animation that looked fine on its own and fell apart the moment it ran in the game. And almost all of that 2.5B is the same project history being re-read from cache on every request, not unique code. Procedural generation from prompts is genuinely fun to watch and wildly inefficient at the same time. The city started as glowing blocks with working driving, money and jobs underneath, and ended up somewhere I'd actually screenshot. Full breakdown:

I spent 2.5 billion tokens, every limit I had and five days to make this. A GTA-style city, built with GPT-6 using only prompts. No code or 3D by hand. The whole point was to push it as far as it goes and wire in every API I had. The setup: Codex driving Unreal Engine 5, with Higgsfield, Tripo, Meshy, Seedance, Seed Audio and fal handling the buildings, characters, animations, materials and sound. 8 buildings taken from concept to game, 247 generated voice lines for residents and police, 10 radio tracks across 5 stations. Here's the honest part. Most of those tokens didn't go into building. They went into fixing. The model writes a mechanic, runs it, watches it break, and tries again for hours. Getting a character to open a car door and actually sit in the seat took an absurd number of passes. A carjacking needed two separate bodies to stay locked to the same grab. Half the work was animation that looked fine on its own and fell apart the moment it ran in the game. And almost all of that 2.5B is the same project history being re-read from cache on every request, not unique code. Procedural generation from prompts is genuinely fun to watch and wildly inefficient at the same time. The city started as glowing blocks with working driving, money and jobs underneath, and ended up somewhere I'd actually screenshot. Full breakdown:

Stefan 3D AI

129,492 Aufrufe • vor 1 Monat