Switching the model or reasoning mid-session can be expensive

Especially if you already have a lot of context in the session - switching the model/thinking level results in the KV cache miss, making the model recompute all previous tokens On the API it costs more money, on a sub it drains your usage limits faster

dang i figured on model switching but just reasoning switching too 😭

Yep, agent harnesses should do a much better job of exposing this info. Pi is very good at this.

No need for model switch, going for a walk and sending next message after a while is also expensive.

Wait what? Isn't reasoning effort basically a limit on reasoning token's budget? Why should it impact the cache? From model's pov, the tokens are already cached.

Uninstalled auto-thinking extension after verifying with pi's cache miss notices. Thought it was helping with cost management. Thanks!

I didn’t know reasoning switching caused a cache miss. Why? Any articles explaining this? Thanks for sharing.

didn't know about the reasoning part!

I know this was the case with model change, but reasoning, really?

Whaaaaa! I do this regularly.

are you 1000% sure switching reasoning also causes cache miss?

good point, thanks

Isn't that only for model switches with Sol? (at least that's what ChatGPT app itself implies)

I thought this only happened during model switches. Learn something new every day.

@grok 深度调研确认下，这个结论是不是真的

That's true. That's why choosing the right model for the right task is important. Miss caches and had to read the earlier conversation to continue which also open rooms for hallucination

Why on earth does switching reasoning invalidate cache though

Reminds me of a great, funky tune 🎶 from The Cure - "A Forest (Acoustic Version)" 👂 "... again and again..." 🕺

The cost surprise is real. I see this in the cost dashboard we track across Campfire ( sessions. The insight that helped: do not switch mid-session. Instead, decide model and reasoning level before you start. For tasks where you are uncertain, run an agent race: the same task across multiple backends in isolated worktrees, each starting fresh with its own context and cache. You compare outputs after, not mid-run. No KV cache misses, no token recomputation.

@pidotdev i knew this about the models, but not reasoning. i assume the same with effort on claude. in the web it has a quick answer button at least sometimes.

What's the argument for model routers if this is the case?