Загрузка видео...
Не удалось загрузить видео
run agent harnesses 100% private & offline. (no token costs, no API keys, 100% open-source) your agent runs locally. the model doesn't. every prompt, every file, and every secret still leaves your machine before the agent does anything with it. Magnitude fixes that. it's an open source inference server... show more
56,499 просмотров • 2 месяцев назад •via X (Twitter)
Комментарии: 16

@akshay_pachaar local processing is a game-changer. tackled similar issues building my own thing and it was a real headache. good move!

Nice - the part everyone skips is that bandwidth sets your token rate, not your GPU name. Most people buy silicon and then wonder why the tokens still crawl.

local model, remote prompt. thats the whole privacy pitch falling apart right there

yeah and with tool-use loops it compounds. each tool call result gets appended and re-sent, so by step 5 you've pushed the same files to the API 3-4 times over without realizing it.

The memory bandwidth part is the bit most people miss. Everyone shops by VRAM then wonders why tokens crawl. Handing back a real config instead of a model list is the useful move here.

local model solves the data-leaving-the-machine problem. it doesn't touch the other one: the agent still has your shell and your files. a private model with unbounded write access is still an agent with unbounded write access.

Naming a config as model + compression + context size + expected speed range is the right unit — a bare model list says nothing about whether a long agent loop will finish. Local models still lose on big multi-file rewrites, so harness choice matters more than model name.

Fully offline is a big deal

half the pain with local models is getting the config right for the hardware

离线是舒服了,显存压力也得自己扛了。

the secrets half is the one people underrate. even with a cloud model, my agents never see .env: a script hands out one key at a time, touch id for the important ones, every access logged. local inference fixes the model half. the tooling half needs the same discipline.

The interesting part is making “offline” operationally verifiable, not just local inference. A useful next step would be a built-in network-egress audit that proves the harness stays air-gapped in practice.

The model has to run locally too, not just the harness, or your prompts and files never really stay private.

Real privacy starts when you prompt your own model

Running an agent harness fully local keeps every prompt, file, and secret off the wire before a model ever sees it - most 'private AI' setups still leak that boundary. #AgenticAI #AITools @AnthropicAI

Amazing



