Загрузка видео...

Не удалось загрузить видео

На главную

Pi Agent vs OpenCode token usage A lot of people recommended Pi Agent so I decided to check Pi Agent took 1.1k tokens in first turn OpenCode took 11.5k Setup: 1) Trimmed OpenCode (from usual 30k first turn to 11.5k) - 0 MCPs - 2 lightweight plugins (opencode-env-protect and...

pseudokid's profile picture

raymel 👋

3,620 subscribers

201,405 просмотров • 2 месяцев назад •via X (Twitter)

ОбразованиеНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

FREE DeepSeek V4 Pro and V4 Flash API. Runtime by Bad Theory Labs just dropped it. Same platform, new free model, same 2-minute setup. > Runtime already gave you free Claude Opus 4.8 and GPT 5.5 access. Now they added DeepSeek V4 Pro and V4 Flash to the free tier. Absolutely FREE, YES. Same exact flow as before: → Go to runtime(.)badtheorylabs(.)com → Sign up with Google → Fill in your details during onboarding → Free balance lands automatically(But you don't need it to use DeepSeek - it's completely free right now) → Grab your API key from the dashboard → Plug into OpenCode, OpenClaw, or Hermes - done DeepSeek V4 Pro is one of the strongest coding and reasoning models available right now. Free tier also includes btl-2 - auto-routes between GLM, OpenAI, Anthropic, and Gemma. 10 million tokens per month at zero cost. 344 models total. One OpenAI-compatible endpoint. No code rebuild, no card, no limits hunting. >> Just start using the API, and when choosing a provider, select DeepSeek I tested it myself - fully working. Bookmark this before the free tier fills up.
0:23

FREE DeepSeek V4 Pro and V4 Flash API. Runtime by Bad Theory Labs just dropped it. Same platform, new free model, same 2-minute setup. > Runtime already gave you free Claude Opus 4.8 and GPT 5.5 access. Now they added DeepSeek V4 Pro and V4 Flash to the free tier. Absolutely FREE, YES. Same exact flow as before: → Go to runtime(.)badtheorylabs(.)com → Sign up with Google → Fill in your details during onboarding → Free balance lands automatically(But you don't need it to use DeepSeek - it's completely free right now) → Grab your API key from the dashboard → Plug into OpenCode, OpenClaw, or Hermes - done DeepSeek V4 Pro is one of the strongest coding and reasoning models available right now. Free tier also includes btl-2 - auto-routes between GLM, OpenAI, Anthropic, and Gemma. 10 million tokens per month at zero cost. 344 models total. One OpenAI-compatible endpoint. No code rebuild, no card, no limits hunting. >> Just start using the API, and when choosing a provider, select DeepSeek I tested it myself - fully working. Bookmark this before the free tier fills up.

Atenov int.

26,920 просмотров • 1 месяц назад

I respectfully disagree for several reasons. Calling a customer, whether free or paying, an idiot is simply wrong. OpenCode, like any other coding agent, clearly tries to preserve the prompt cache as much as possible. Otherwise, it would be painfully slow. The “Stop Using OpenCode” article, which I believe Dax is referring to, focused on specific cases that can invalidate the cached prefix. These issues become much more visible and painful when using local models. Modifying AGENTS.md is one example, as shown in this video. During this coding session, cache efficiency is extremely high. But the moment I modify AGENTS.md, boom: 62K tokens need to be processed again as new input. The date change is another valid point raised in the article. I understand that both issues may sound trivial, and personally I've never been impacted by them, but their impact on the OpenCode end-user experience can be significant when using local models. I agree that both the feedback and the article were too direct and were probably written by the author out of frustration. However, they also contained constructive points that could help improve the product. I use OpenCode alongside several other tools, and I like it overall. But I really dislike this kind of reaction to user feedback, regardless of how that feedback is expressed. I would have preferred a response focused more on listening, understanding the problem, and improving the product.
0:28

I respectfully disagree for several reasons. Calling a customer, whether free or paying, an idiot is simply wrong. OpenCode, like any other coding agent, clearly tries to preserve the prompt cache as much as possible. Otherwise, it would be painfully slow. The “Stop Using OpenCode” article, which I believe Dax is referring to, focused on specific cases that can invalidate the cached prefix. These issues become much more visible and painful when using local models. Modifying AGENTS.md is one example, as shown in this video. During this coding session, cache efficiency is extremely high. But the moment I modify AGENTS.md, boom: 62K tokens need to be processed again as new input. The date change is another valid point raised in the article. I understand that both issues may sound trivial, and personally I've never been impacted by them, but their impact on the OpenCode end-user experience can be significant when using local models. I agree that both the feedback and the article were too direct and were probably written by the author out of frustration. However, they also contained constructive points that could help improve the product. I use OpenCode alongside several other tools, and I like it overall. But I really dislike this kind of reaction to user feedback, regardless of how that feedback is expressed. I would have preferred a response focused more on listening, understanding the problem, and improving the product.

Ivan Fioravanti ᯅ

36,983 просмотров • 8 дней назад

NVIDIA is giving away free access to 130+ AI models for a full year > most people building AI agents are paying $50-200/month for API access NVIDIA just made that argument irrelevant models you get: MiniMax M2.7, GLM 5.1, Kimi 2.5, DeepSeek-v4-flash, GPT-OSS-120B and 110+ more setup: > step 1 - get your free key > go to > register -> bind phone -> copy API key > step 2 - add to Hermes agent > open Settings -> Model Provider -> Custom base_url = " api_key = "nvapi-xxxxxxxxxxxxxxxxxxxx" > step 3 - pick a model model = "minimaxai/minimax-m2.7" model = "zhipuai/glm-5.1" model = "moonshot-ai/kimi-2.5" model = "deepseek-ai/deepseek-v4-flash" model = "nvidia/nemotron-3-ultra-550b-a55b" > Hermes already has NVIDIA set as default base_url > paste the key and you're running instantly > works the same in Cursor and OpenCode > cost: $0 > limit: 40 req/min > expires: 1 year while everyone is paying for API access, this is sitting there for free
0:38

NVIDIA is giving away free access to 130+ AI models for a full year > most people building AI agents are paying $50-200/month for API access NVIDIA just made that argument irrelevant models you get: MiniMax M2.7, GLM 5.1, Kimi 2.5, DeepSeek-v4-flash, GPT-OSS-120B and 110+ more setup: > step 1 - get your free key > go to > register -> bind phone -> copy API key > step 2 - add to Hermes agent > open Settings -> Model Provider -> Custom base_url = " api_key = "nvapi-xxxxxxxxxxxxxxxxxxxx" > step 3 - pick a model model = "minimaxai/minimax-m2.7" model = "zhipuai/glm-5.1" model = "moonshot-ai/kimi-2.5" model = "deepseek-ai/deepseek-v4-flash" model = "nvidia/nemotron-3-ultra-550b-a55b" > Hermes already has NVIDIA set as default base_url > paste the key and you're running instantly > works the same in Cursor and OpenCode > cost: $0 > limit: 40 req/min > expires: 1 year while everyone is paying for API access, this is sitting there for free

Mr. Buzzoni

192,952 просмотров • 1 месяц назад

HERMES AGENT WITHOUT THESE 3 FILES IS A CHATBOT. WITH THEM IT KNOWS WHO IT IS, WHO YOU ARE, AND WHAT IT LEARNED. SOUL.md — who the agent is. first thing in the system prompt. defines personality, voice, values, how it operates, what it can and can't do. structure yours like this: → identity (name, role, relationship to you) → values (what matters, what principles guide decisions) → voice (how it communicates, tone, style) → operations (autonomy level, ground rules) → restrictions (what it never does) → failure protocol (how to operate when things break) lives in ~/.hermes/SOUL.md. auto-seeded on first install. edit anytime. scanned for prompt injection on every load. keep it concise. SOUL.md injects into every turn. a 200-line soul burns tokens on every message. aim for 50-80 lines. one paragraph per section. MEMORY.md — what the agent remembers. persistent facts, insights, preferences. survives across sessions and restarts. capped at ~800 tokens by default: memory: memory_char_limit: 2200 the agent writes to this automatically as it learns about your work. USER.md — who you are. your profile, preferences, context. capped at ~500 tokens by default: memory: user_char_limit: 1375 injected every turn so the agent always knows who it's working for. bonus: AGENTS.md — project-specific instructions. drop one in any project folder. subdirectory AGENTS.md files load lazily during tool calls, not at startup. keeps your system prompt light. prompt stack order on every turn: SOUL.md → tool guidance → MEMORY.md + USER.md → skills index → AGENTS.md → platform hints skills come preloaded. 60+ built-in tools. the agent creates more skills from completed work. you focus on these 3 files. each profile gets its own copy: ~/.hermes/SOUL.md (default profile) ~/.hermes/profiles/researcher/SOUL.md ~/.hermes/profiles/ops/SOUL.md different agent, different soul, different memory, same machine. full breakdown of Hermes as a Personal AI OS in the article 👇
0:54

HERMES AGENT WITHOUT THESE 3 FILES IS A CHATBOT. WITH THEM IT KNOWS WHO IT IS, WHO YOU ARE, AND WHAT IT LEARNED. SOUL.md — who the agent is. first thing in the system prompt. defines personality, voice, values, how it operates, what it can and can't do. structure yours like this: → identity (name, role, relationship to you) → values (what matters, what principles guide decisions) → voice (how it communicates, tone, style) → operations (autonomy level, ground rules) → restrictions (what it never does) → failure protocol (how to operate when things break) lives in ~/.hermes/SOUL.md. auto-seeded on first install. edit anytime. scanned for prompt injection on every load. keep it concise. SOUL.md injects into every turn. a 200-line soul burns tokens on every message. aim for 50-80 lines. one paragraph per section. MEMORY.md — what the agent remembers. persistent facts, insights, preferences. survives across sessions and restarts. capped at ~800 tokens by default: memory: memory_char_limit: 2200 the agent writes to this automatically as it learns about your work. USER.md — who you are. your profile, preferences, context. capped at ~500 tokens by default: memory: user_char_limit: 1375 injected every turn so the agent always knows who it's working for. bonus: AGENTS.md — project-specific instructions. drop one in any project folder. subdirectory AGENTS.md files load lazily during tool calls, not at startup. keeps your system prompt light. prompt stack order on every turn: SOUL.md → tool guidance → MEMORY.md + USER.md → skills index → AGENTS.md → platform hints skills come preloaded. 60+ built-in tools. the agent creates more skills from completed work. you focus on these 3 files. each profile gets its own copy: ~/.hermes/SOUL.md (default profile) ~/.hermes/profiles/researcher/SOUL.md ~/.hermes/profiles/ops/SOUL.md different agent, different soul, different memory, same machine. full breakdown of Hermes as a Personal AI OS in the article 👇

YanXbt

28,924 просмотров • 1 месяц назад

I cant believe this guy just made a permanent solution to context bloat and open sourced it all! when we tested this tool (Context+) for solving an issue on the OpenCode repository, the agent using this tool used ~6.5k fewer tokens, found the code and fixed it in half the time! the results were surprising: 6 to 10k tokens saved per prompt, completed task in ~2 minutes while the agent running without the tool took ~4 mins for the same and got stuck in loops bro built an entire beast by using all the modern tools that we could think of: undo trees, semantic search by meaning (by haskellforall), advanced refactoring, blast radius, advanced file context trees, restore points... i can keep going on semantic code search and context trees are the future of agentic coding and this tool proves it the feature i loved the most is semantic search and how it gets things done 2x faster with least possible tokens it makes an agent that actually knows what it’s doing and not just guessing, it makes meaning from your code similar to RAG. if you aren't optimizing your context, you are just burning money the developer says this tool is still under development, it can have unexpected behavior and the docs need updates but the video shows the reality of how fast it can be github: get here:
0:46

I cant believe this guy just made a permanent solution to context bloat and open sourced it all! when we tested this tool (Context+) for solving an issue on the OpenCode repository, the agent using this tool used ~6.5k fewer tokens, found the code and fixed it in half the time! the results were surprising: 6 to 10k tokens saved per prompt, completed task in ~2 minutes while the agent running without the tool took ~4 mins for the same and got stuck in loops bro built an entire beast by using all the modern tools that we could think of: undo trees, semantic search by meaning (by haskellforall), advanced refactoring, blast radius, advanced file context trees, restore points... i can keep going on semantic code search and context trees are the future of agentic coding and this tool proves it the feature i loved the most is semantic search and how it gets things done 2x faster with least possible tokens it makes an agent that actually knows what it’s doing and not just guessing, it makes meaning from your code similar to RAG. if you aren't optimizing your context, you are just burning money the developer says this tool is still under development, it can have unexpected behavior and the docs need updates but the video shows the reality of how fast it can be github: get here:

forloop

226,054 просмотров • 5 месяцев назад

hy3 vs mimo-v2.5 vs deepseek v4 flash vs minimax m3 the four models on top of the openrouter leaderboard by tokens this week: #1 hy3 (Tencent Hy) – 7.5t #2 mimo-v2.5 (Xiaomi MiMo) – 6.56t #3 deepseek v4 flash (DeepSeek) – 5.24t #4 minimax m3 (MiniMax (official)) – 4.21t so we tested them. 3 prompts, single-file html, Three.js from a cdn, fully procedural, no external assets. all run via AI/ML API each prompt is a transparent cutaway machine that has to be mechanically correct, not decorative: • 4-stroke engine with full oil circulation – slider-crank kinematics, cam at 2:1, valve lift driven by lobes, oil loop from sump to gallery to big-end • watt walking-beam steam engine – four-bar vector-loop closure, eccentric-driven slide valve, steam events synced to real port position • francis reaction water turbine – 20 guide vanes on a regulating ring, 17 lofted runner blades, gpu particle advection, precessing vortex rope at part load the takeaway up front: none of the four cleared all three scenes on the first attempt. but the price spread between them is roughly 70x – hy3 fixed included costs less than two cents overall results (summed across all 3 scenes): cost #1 hy3 – $0.016 #2 deepseek v4 flash – $0.025 #3 mimo-v2.5 – $0.97 #4 minimax m3 – $1.17 tokens #1 hy3 – 19,326 #2 deepseek v4 flash – 63,126 #3 mimo-v2.5 – 322,523 #4 minimax m3 – 702,900 lines of code #1 hy3 – 1,047 #2 mimo-v2.5 – 2,759 #3 deepseek v4 flash – 3,273 #4 minimax m3 – 3,354 scenes needing a second attempt #1 hy3 – 1 (engine) #1 mimo-v2.5 – 1 (turbine) #1 minimax m3 – 1 (turbine) #4 deepseek v4 flash – 2 (steam engine, turbine) observations: 1. the token spread is the real story – minimax burns 36x hy3's tokens and lands in the same place, one retry, ~3.3k lines 2. hy3 is the outlier on density: 1,047 lines total, fewest tokens, cheapest run, and only one scene needed a second pass. deepseek is the opposite trade – near-hy3 pricing but the most retries 3. mimo and minimax seem to overthink instead of writing the code. minimax spent 359.1k tokens on the steam engine and produced 1,346 lines – the tokens are going somewhere other than the file 4. the francis turbine broke three of the four. the spec that separates them is the one with 20 linked guide vanes and gpu particle advection, not the one with the most parts overall impression: none of these models excelled at any of the tasks we gave them. but they were close, and they were extremely cheap. the gap that matters isn't quality anymore – it's that hy3 ran all three scenes for less than two cents while the frontier labs charge dollars for the same work right now you pick these because they're good for the zero price you pay. soon that's something openai and anthropic will have to think about follow thehype. for 24/7 ai news, analysis and breakdowns
0:55

hy3 vs mimo-v2.5 vs deepseek v4 flash vs minimax m3 the four models on top of the openrouter leaderboard by tokens this week: #1 hy3 (Tencent Hy) – 7.5t #2 mimo-v2.5 (Xiaomi MiMo) – 6.56t #3 deepseek v4 flash (DeepSeek) – 5.24t #4 minimax m3 (MiniMax (official)) – 4.21t so we tested them. 3 prompts, single-file html, Three.js from a cdn, fully procedural, no external assets. all run via AI/ML API each prompt is a transparent cutaway machine that has to be mechanically correct, not decorative: • 4-stroke engine with full oil circulation – slider-crank kinematics, cam at 2:1, valve lift driven by lobes, oil loop from sump to gallery to big-end • watt walking-beam steam engine – four-bar vector-loop closure, eccentric-driven slide valve, steam events synced to real port position • francis reaction water turbine – 20 guide vanes on a regulating ring, 17 lofted runner blades, gpu particle advection, precessing vortex rope at part load the takeaway up front: none of the four cleared all three scenes on the first attempt. but the price spread between them is roughly 70x – hy3 fixed included costs less than two cents overall results (summed across all 3 scenes): cost #1 hy3 – $0.016 #2 deepseek v4 flash – $0.025 #3 mimo-v2.5 – $0.97 #4 minimax m3 – $1.17 tokens #1 hy3 – 19,326 #2 deepseek v4 flash – 63,126 #3 mimo-v2.5 – 322,523 #4 minimax m3 – 702,900 lines of code #1 hy3 – 1,047 #2 mimo-v2.5 – 2,759 #3 deepseek v4 flash – 3,273 #4 minimax m3 – 3,354 scenes needing a second attempt #1 hy3 – 1 (engine) #1 mimo-v2.5 – 1 (turbine) #1 minimax m3 – 1 (turbine) #4 deepseek v4 flash – 2 (steam engine, turbine) observations: 1. the token spread is the real story – minimax burns 36x hy3's tokens and lands in the same place, one retry, ~3.3k lines 2. hy3 is the outlier on density: 1,047 lines total, fewest tokens, cheapest run, and only one scene needed a second pass. deepseek is the opposite trade – near-hy3 pricing but the most retries 3. mimo and minimax seem to overthink instead of writing the code. minimax spent 359.1k tokens on the steam engine and produced 1,346 lines – the tokens are going somewhere other than the file 4. the francis turbine broke three of the four. the spec that separates them is the one with 20 linked guide vanes and gpu particle advection, not the one with the most parts overall impression: none of these models excelled at any of the tasks we gave them. but they were close, and they were extremely cheap. the gap that matters isn't quality anymore – it's that hy3 ran all three scenes for less than two cents while the frontier labs charge dollars for the same work right now you pick these because they're good for the zero price you pay. soon that's something openai and anthropic will have to think about follow thehype. for 24/7 ai news, analysis and breakdowns

thehype.

17,145 просмотров • 16 дней назад

I have been testing DeepSeek-V4-Pro with the Pi coding agent. I am mindblown by how well it works out of the box. A few notes: I spent a few hours building an LLM wiki with an agent powered entirely by DeepSeek-V4-Pro on Fireworks AI inference. This is the first time I feel like there is an open-weight model that can reason at the level of Claude and Codex. And it does this in a cost-effective way with support for 1M context length. To be clear, I am using DeepSeek-V4-Pro inside of Pi without any special configuration. It works out of the box. It's exciting that there is a model that can just be plugged into a basic harness like Pi, and it just works. I've never seen that before. Most models require lots of configuration and setup. DeepSeek's DeepSeek-V4-Pro is clearly good at agentic coding (probably the best from the open-weight models), but the model is also great on knowledge-intensive tasks where reasoning matters. The agent pulled agentic engineering best practices from different company docs (Anthropic, OpenAI, Google, Stripe, Meta, Modal, DeepSeek, Mistral, Cohere), searched and digested Reddit and HN threads, summarized arxiv papers, and surfaced trending GitHub repos. Then it distilled everything into actionable tips across categories. I love the Wiki it built. The quality is really good. Here is a snapshot of what the wiki looks like: DeepSeek-V4-Pro handled the task without breaking stride. Multi-step research queries, code generation for scaffolding, context-heavy reasoning across disparate sources. For coding specifically, this is the first open-weight model that genuinely feels like a Codex or Claude Code experience. It compares in capability and actual multi-turn agentic work. What made the loop feel so responsive was Fireworks' inference speed (the fastest in the market) and the fact that they actually validate models at the systems level before shipping. No corrupted reasoning traces. Just fast, reliable iteration. The hybrid CSA and HCA attention design cuts KV cache to just 10% and inference FLOPs by nearly 4x at 1M-token context. This is what makes the agent loop actually fast and cheap enough to run in practice. For devs who've been watching open-weight models close the gap but haven't found one that actually delivers in practice, this is the closest I've seen. Try it here:
0:31

I have been testing DeepSeek-V4-Pro with the Pi coding agent. I am mindblown by how well it works out of the box. A few notes: I spent a few hours building an LLM wiki with an agent powered entirely by DeepSeek-V4-Pro on Fireworks AI inference. This is the first time I feel like there is an open-weight model that can reason at the level of Claude and Codex. And it does this in a cost-effective way with support for 1M context length. To be clear, I am using DeepSeek-V4-Pro inside of Pi without any special configuration. It works out of the box. It's exciting that there is a model that can just be plugged into a basic harness like Pi, and it just works. I've never seen that before. Most models require lots of configuration and setup. DeepSeek's DeepSeek-V4-Pro is clearly good at agentic coding (probably the best from the open-weight models), but the model is also great on knowledge-intensive tasks where reasoning matters. The agent pulled agentic engineering best practices from different company docs (Anthropic, OpenAI, Google, Stripe, Meta, Modal, DeepSeek, Mistral, Cohere), searched and digested Reddit and HN threads, summarized arxiv papers, and surfaced trending GitHub repos. Then it distilled everything into actionable tips across categories. I love the Wiki it built. The quality is really good. Here is a snapshot of what the wiki looks like: DeepSeek-V4-Pro handled the task without breaking stride. Multi-step research queries, code generation for scaffolding, context-heavy reasoning across disparate sources. For coding specifically, this is the first open-weight model that genuinely feels like a Codex or Claude Code experience. It compares in capability and actual multi-turn agentic work. What made the loop feel so responsive was Fireworks' inference speed (the fastest in the market) and the fact that they actually validate models at the systems level before shipping. No corrupted reasoning traces. Just fast, reliable iteration. The hybrid CSA and HCA attention design cuts KV cache to just 10% and inference FLOPs by nearly 4x at 1M-token context. This is what makes the agent loop actually fast and cheap enough to run in practice. For devs who've been watching open-weight models close the gap but haven't found one that actually delivers in practice, this is the closest I've seen. Try it here:

elvis

59,803 просмотров • 3 месяцев назад

HERMES AGENT NOW RUNS A FULL BUSINESS FOR $8/MONTH. CONTENT. CODE. INBOX. ALL AGENTS. EVERYTHING WHILE YOU SLEEP. one user built this in one session. zero terminal commands. browser only. the team: → content agent: finds video topics, writes first drafts, helps with animations → code builder: writes website and SaaS code → ops agent: organizes and answers email total setup cost: $1. monthly runtime: $8 VPS + cheap model API. the trick is matching model to task. frontier models (Opus 4.8, GPT-5.5) for complex /goals only. $5/million tokens. mid-tier (Claude Sonnet, Haiku, Kimiko 2.6) for daily content and research work. budget (DeepSeek V4 Flash, MiniMax 2.7, Gemma 4, Qwen 3) for cron jobs and routine. run the initial setup on Opus. switch to DeepSeek for everything after. your bill drops 90% with zero quality loss on routine work. the agent team picks up tasks from kanban. each completed task creates a skill. week 3 the same workflows run faster than week 1. full breakdown on Hermes Os in the article 👇
14:53

HERMES AGENT NOW RUNS A FULL BUSINESS FOR $8/MONTH. CONTENT. CODE. INBOX. ALL AGENTS. EVERYTHING WHILE YOU SLEEP. one user built this in one session. zero terminal commands. browser only. the team: → content agent: finds video topics, writes first drafts, helps with animations → code builder: writes website and SaaS code → ops agent: organizes and answers email total setup cost: $1. monthly runtime: $8 VPS + cheap model API. the trick is matching model to task. frontier models (Opus 4.8, GPT-5.5) for complex /goals only. $5/million tokens. mid-tier (Claude Sonnet, Haiku, Kimiko 2.6) for daily content and research work. budget (DeepSeek V4 Flash, MiniMax 2.7, Gemma 4, Qwen 3) for cron jobs and routine. run the initial setup on Opus. switch to DeepSeek for everything after. your bill drops 90% with zero quality loss on routine work. the agent team picks up tasks from kanban. each completed task creates a skill. week 3 the same workflows run faster than week 1. full breakdown on Hermes Os in the article 👇

YanXbt

30,910 просмотров • 1 месяц назад

I cut Fable 5 token usage 2.5x with just one change! - Before: 5.5 M tokens · 7 errors · $8.94 - After: 2.3 M tokens · 0 errors · $4.17 The final build was the same for both, but the path the agent took wildly differed. In both runs, the agent started with the same thing, i.e., it understood the backend before building anything, like: - Permission policies - Available storage buckets - Auth providers configured - How edge functions are deployed The first run used Firebase, which was built for a human dev using a dashboard. While the dev can read the above state by clicking through tabs, an agent has no dashboard. So it gathered the same info through API calls. And there's no single Firebase call that returned this info. The agent required to query multiple times, and each query over-returned. For instance, when the agent asked how sign-in is configured, Firebase also returned the entire auth surface and every method it supported. This was far more context than what it needed. And it repeated across every part of the backend it inspected. Some states (like which auth providers are active) weren't queryable at all. I provided it myself. Otherwise, the agent would have guessed. Errors further compounded the token usage. When a dev sees "permission denied," they can look at the console and figure out whether it's a rule, a path, or an unauthenticated request. Firebase returned the same string to the agent as well, and it had none of that surrounding context to debug. So it guessed again, picked the most likely cause, and rewrote code, utilizing more tokens. This Firebase setup cost me 5.5M tokens and 7 manual interventions during errors on a full-stack RAG app. But I brought that down to 2.3M tokens and 0 manual interventions by using InsForge as the backend context engineering layer (open-source and self-hostable via Docker). It provides the same primitives as Supabase/Firebase, but structures the entire information layer for agents, instead of dashboards. In one CLI call that consumed ~500 tokens, the agent saw the full backend topology before writing a single line of code. This included auth, database, storage, edge functions, model gateway, micro VMs, and deployment. Also, instead of loading the entire product surface into context on every task, four narrowly scoped skills activated only when relevant to keep cognitive load minimal. And to ensure efficient retries if needed, every CLI operation returned structured JSON with meaningful exit codes, so the agent never guessed what to do next. Here's the InsForge GitHub Repo: (don't forget to star it ⭐) The video below depicts the final build, comparing Firebase and InsForge. To dive deeper, I recently published a full walkthrough building the same RAG app on both backends and inspected them end-to-end. Read it below.
0:29

I cut Fable 5 token usage 2.5x with just one change! - Before: 5.5 M tokens · 7 errors · $8.94 - After: 2.3 M tokens · 0 errors · $4.17 The final build was the same for both, but the path the agent took wildly differed. In both runs, the agent started with the same thing, i.e., it understood the backend before building anything, like: - Permission policies - Available storage buckets - Auth providers configured - How edge functions are deployed The first run used Firebase, which was built for a human dev using a dashboard. While the dev can read the above state by clicking through tabs, an agent has no dashboard. So it gathered the same info through API calls. And there's no single Firebase call that returned this info. The agent required to query multiple times, and each query over-returned. For instance, when the agent asked how sign-in is configured, Firebase also returned the entire auth surface and every method it supported. This was far more context than what it needed. And it repeated across every part of the backend it inspected. Some states (like which auth providers are active) weren't queryable at all. I provided it myself. Otherwise, the agent would have guessed. Errors further compounded the token usage. When a dev sees "permission denied," they can look at the console and figure out whether it's a rule, a path, or an unauthenticated request. Firebase returned the same string to the agent as well, and it had none of that surrounding context to debug. So it guessed again, picked the most likely cause, and rewrote code, utilizing more tokens. This Firebase setup cost me 5.5M tokens and 7 manual interventions during errors on a full-stack RAG app. But I brought that down to 2.3M tokens and 0 manual interventions by using InsForge as the backend context engineering layer (open-source and self-hostable via Docker). It provides the same primitives as Supabase/Firebase, but structures the entire information layer for agents, instead of dashboards. In one CLI call that consumed ~500 tokens, the agent saw the full backend topology before writing a single line of code. This included auth, database, storage, edge functions, model gateway, micro VMs, and deployment. Also, instead of loading the entire product surface into context on every task, four narrowly scoped skills activated only when relevant to keep cognitive load minimal. And to ensure efficient retries if needed, every CLI operation returned structured JSON with meaningful exit codes, so the agent never guessed what to do next. Here's the InsForge GitHub Repo: (don't forget to star it ⭐) The video below depicts the final build, comparing Firebase and InsForge. To dive deeper, I recently published a full walkthrough building the same RAG app on both backends and inspected them end-to-end. Read it below.

Avi Chawla

112,879 просмотров • 1 месяц назад

Cerebras inference is very fast. So fast that it changes how we think about configuring our LLMs for voice agent use cases. Kimi K2.6 is a 1T parameter reasoning model that Cerebras serves at 650 - 1,000 tokens per second (end-to-end throughput), with time to first token metrics as low as 150ms (latency). These numbers are two to three times faster than other similarly capable models. The biggest lever we get from this kind of speed is that we can use the model in reasoning mode, and still have excellent "time to first non-thinking token." This solves a big pain point we have in 2026 for voice agent use cases. Almost all recent innovation in post-training has focused on making models good at reasoning ("test time compute"). This is great, but it makes the user-facing model latency much, much slower. Which is a problem for conversational voice agents. We can run Kimi K2.6 with reasoning turned on, and get responses faster than other models produce with reasoning disabled. On my 30-turn voice agent benchmark, Kimi K2.6 with reasoning enabled ties GPT 5.1 and Haiku 4.5 with reasoning disabled, and is still about 200ms seconds faster! On my primary task agent benchmark, Kimi K2.6 is now the #2 model. It ranks just behind Gemini 3.5 Flash in "high" reasoning mode, and tied with GLM 5, Sonnet 4.6, and GPT 5.4 with reasoning set to "low." But Kimi K2.6 completes each turn in the agent loop in under 500ms. The other four models are all at least 3x slower. (Models only qualify for this benchmark if they can complete task turns at a P50 <4s.) A couple of other things that this speed buys us, for production voice agents: - Tool calls happen fast enough that we don't have to work around tool call latency in our pipeline design. - We can prompt the model to output structured data at the beginning of a response, followed by plain text for voice generation. This opens up possibilities like asking the model to do complex classification/generation tasks that influence the rest of the pipeline. For example, the model could create a detailed style prompt for a steerable TTS model, for each individual conversation turn. And, of course, you can use Kimi K2.6 with reasoning turned off. Cerebras calls this "instant" mode. Here's a video of a Cerebras Kimi K2.6 voice agent with voice-to-voice response time, measured at the client, under 500ms. This is the true response latency as perceived by the user, including all network and audio codec overhead, transcription and turn detection, Kimi K2.6 token generation, and voice generation. 500ms is, effectively, instant. So the Cerebras naming for this mode is a propos. :-)
0:40

Cerebras inference is very fast. So fast that it changes how we think about configuring our LLMs for voice agent use cases. Kimi K2.6 is a 1T parameter reasoning model that Cerebras serves at 650 - 1,000 tokens per second (end-to-end throughput), with time to first token metrics as low as 150ms (latency). These numbers are two to three times faster than other similarly capable models. The biggest lever we get from this kind of speed is that we can use the model in reasoning mode, and still have excellent "time to first non-thinking token." This solves a big pain point we have in 2026 for voice agent use cases. Almost all recent innovation in post-training has focused on making models good at reasoning ("test time compute"). This is great, but it makes the user-facing model latency much, much slower. Which is a problem for conversational voice agents. We can run Kimi K2.6 with reasoning turned on, and get responses faster than other models produce with reasoning disabled. On my 30-turn voice agent benchmark, Kimi K2.6 with reasoning enabled ties GPT 5.1 and Haiku 4.5 with reasoning disabled, and is still about 200ms seconds faster! On my primary task agent benchmark, Kimi K2.6 is now the #2 model. It ranks just behind Gemini 3.5 Flash in "high" reasoning mode, and tied with GLM 5, Sonnet 4.6, and GPT 5.4 with reasoning set to "low." But Kimi K2.6 completes each turn in the agent loop in under 500ms. The other four models are all at least 3x slower. (Models only qualify for this benchmark if they can complete task turns at a P50 <4s.) A couple of other things that this speed buys us, for production voice agents: - Tool calls happen fast enough that we don't have to work around tool call latency in our pipeline design. - We can prompt the model to output structured data at the beginning of a response, followed by plain text for voice generation. This opens up possibilities like asking the model to do complex classification/generation tasks that influence the rest of the pipeline. For example, the model could create a detailed style prompt for a steerable TTS model, for each individual conversation turn. And, of course, you can use Kimi K2.6 with reasoning turned off. Cerebras calls this "instant" mode. Here's a video of a Cerebras Kimi K2.6 voice agent with voice-to-voice response time, measured at the client, under 500ms. This is the true response latency as perceived by the user, including all network and audio codec overhead, transcription and turn detection, Kimi K2.6 token generation, and voice generation. 500ms is, effectively, instant. So the Cerebras naming for this mode is a propos. :-)

kwindla

40,319 просмотров • 2 месяцев назад

I designed a new test specifically for multimodal models: fill out a paper form. And it's much harder than it sounds. This isn't typing into an electronic field that captures your text. The form is just an image. The model has to place each form element: text, checkmarks — at the correct pixel position on the canvas itself. Results: 🟢 Kimi K2.6 → done in 3:45, 16.7k output tokens 🟡 Step 3.7 Flash → half the fields, 57k output tokens 🔴 Gemini 3.5 Flash → 489k output tokens, never finished. I had to kill it. Gemini burned ~29x more output tokens than Kimi on the exact same task, and Kimi's was the only form that actually looked filled out. The test, a mocked application form, contains some challenging parts, such as one-character-per-box fields. I provided every model the same set of tools: > get canvas size > drop probe markers to find coordinates > add text > add checkmarks > move elements > take a screenshot anytime to check their own work > ... etc So it's vision + spatial reasoning + tool use + long context, all at once. Small models (Qwen, Gemma) can't really complete this test, so I skipped them. What happened: > Kimi nailed name, DOB, ID, gender, marital status, nationality, email, phone, address, postal code — placement slightly loose, but content correct. 15 turns. Clean. > Step got maybe half right — fields dropped, "United States" landed in the email line, data floating outside boxes. Burned 1.24M input tokens doing it (81 turns of re-reading the canvas). > Gemini almost got there visually... then spiraled. By turn 40 it was issuing a delete_elements call wiping element IDs 365–425, basically erasing its own work. 31 minutes, 489k output tokens, still streaming. Terminated. The takeaway isn't "Gemini bad." This test is indeed difficult. But token efficiency is capability now. A model that needs 30x the tokens and still can't converge is going to be 30x the cost in production. Kimi K2.6 just quietly did the thing.
0:38

I designed a new test specifically for multimodal models: fill out a paper form. And it's much harder than it sounds. This isn't typing into an electronic field that captures your text. The form is just an image. The model has to place each form element: text, checkmarks — at the correct pixel position on the canvas itself. Results: 🟢 Kimi K2.6 → done in 3:45, 16.7k output tokens 🟡 Step 3.7 Flash → half the fields, 57k output tokens 🔴 Gemini 3.5 Flash → 489k output tokens, never finished. I had to kill it. Gemini burned ~29x more output tokens than Kimi on the exact same task, and Kimi's was the only form that actually looked filled out. The test, a mocked application form, contains some challenging parts, such as one-character-per-box fields. I provided every model the same set of tools: > get canvas size > drop probe markers to find coordinates > add text > add checkmarks > move elements > take a screenshot anytime to check their own work > ... etc So it's vision + spatial reasoning + tool use + long context, all at once. Small models (Qwen, Gemma) can't really complete this test, so I skipped them. What happened: > Kimi nailed name, DOB, ID, gender, marital status, nationality, email, phone, address, postal code — placement slightly loose, but content correct. 15 turns. Clean. > Step got maybe half right — fields dropped, "United States" landed in the email line, data floating outside boxes. Burned 1.24M input tokens doing it (81 turns of re-reading the canvas). > Gemini almost got there visually... then spiraled. By turn 40 it was issuing a delete_elements call wiping element IDs 365–425, basically erasing its own work. 31 minutes, 489k output tokens, still streaming. Terminated. The takeaway isn't "Gemini bad." This test is indeed difficult. But token efficiency is capability now. A model that needs 30x the tokens and still can't converge is going to be 30x the cost in production. Kimi K2.6 just quietly did the thing.

stevibe

25,304 просмотров • 2 месяцев назад

Dr. Fan video translation: First of all, I would like to congratulate more than 16 million Pioneers around the world for transitioning to the open network. This is the result of our joint efforts over the past six years. We should celebrate this historic moment in the development of the network. Looking back at the development of Pi now, it has always been a unique project. Some people will feel that it is different, and yes, it is. Pi is a non-conformist. Many innovations come from non-conformists who challenge conventions, conventions, and established practices. In the case of Pi, in the early days of cryptocurrency development, many projects raised funds through initial coin offerings (ICOs), but Pi never did so. Pi never sold tokens through ICOs and ensured that everyone could get them for free. 80% of Pi tokens belong to the public and the community. When a cryptocurrency project can go online with just a white paper, a smart contract, or even just an emoji today, the Pi community spent six years building its infrastructure, ecosystem, and ensuring that it has usable practical functions. Before opening, most of its addresses were unverified for most blockchains. Pi chooses to verify the identities of millions of users through identity verification (KYC) and business identity verification (KYB) to ensure that the identities of individuals and businesses on the mainnet are authentic and reliable. Because Pi firmly believes that true decentralization is not inconsistent with authenticity and legitimacy. Some cryptocurrency critics worry that Pi's users are too mainstream. Is this a problem? We think not. This is precisely the biggest advantage that the Pi network has been working hard to build and build over the past six years. Ordinary people like Pioneers are the driving force behind Pi's development because Pi aims to solve the problems of large-scale applications and real-world practicality. Pi welcomes both cryptocurrency enthusiasts and mainstream people. History has shown that only by meeting the real needs of the mainstream population can technology develop for a long time. With this development trend, the cryptocurrency industry as a whole will also benefit from the participation of a large number of mainstream audiences to jointly create the real utility of blockchain technology. What does it mean to open the network today? This means that the firewall of the Pi blockchain has been removed and connected to the outside world. This makes it easier for merchants to sell goods and services in the local market; developers can further develop applications and improve business model logic; creators can gain greater influence; and ordinary Pioneers can also connect and trade with each other. For new developers, welcome to this widely distributed crypto network. If you already have a business model, come to the Pi network to test and sell your products. For developers who don't have a business model but have an excellent application experience, the Pi application network has prepared a business model for you. The platform will bring you traffic and make you profitable. At the same time, it creates the use value of Pi for all pioneers, which is very beneficial to the Pi network as a whole. So, Pioneers, in the end, don't let external noise distract you. Focus on key matters, focus on things that can have an impact in the crypto field and even the world. Keep working hard, keep creating, and everything else will fall into place. MY MESSAGE Please listen carefully to this video so that you understand, the value is determined by the existing ecosystem, the current ecosystem is GCV $314,159 What small traders do under the auspices of the ambassadors 🥰 Eagle woman 🦅 Nonny Padja NTT 🇮🇩 Believe it or not, it's up to you, the point is we've won with a GCV barter value of $314,159 If you want to succeed Barter with Pioneer and small merchants with GCV value of $314,159❤️
4:38

Dr. Fan video translation: First of all, I would like to congratulate more than 16 million Pioneers around the world for transitioning to the open network. This is the result of our joint efforts over the past six years. We should celebrate this historic moment in the development of the network. Looking back at the development of Pi now, it has always been a unique project. Some people will feel that it is different, and yes, it is. Pi is a non-conformist. Many innovations come from non-conformists who challenge conventions, conventions, and established practices. In the case of Pi, in the early days of cryptocurrency development, many projects raised funds through initial coin offerings (ICOs), but Pi never did so. Pi never sold tokens through ICOs and ensured that everyone could get them for free. 80% of Pi tokens belong to the public and the community. When a cryptocurrency project can go online with just a white paper, a smart contract, or even just an emoji today, the Pi community spent six years building its infrastructure, ecosystem, and ensuring that it has usable practical functions. Before opening, most of its addresses were unverified for most blockchains. Pi chooses to verify the identities of millions of users through identity verification (KYC) and business identity verification (KYB) to ensure that the identities of individuals and businesses on the mainnet are authentic and reliable. Because Pi firmly believes that true decentralization is not inconsistent with authenticity and legitimacy. Some cryptocurrency critics worry that Pi's users are too mainstream. Is this a problem? We think not. This is precisely the biggest advantage that the Pi network has been working hard to build and build over the past six years. Ordinary people like Pioneers are the driving force behind Pi's development because Pi aims to solve the problems of large-scale applications and real-world practicality. Pi welcomes both cryptocurrency enthusiasts and mainstream people. History has shown that only by meeting the real needs of the mainstream population can technology develop for a long time. With this development trend, the cryptocurrency industry as a whole will also benefit from the participation of a large number of mainstream audiences to jointly create the real utility of blockchain technology. What does it mean to open the network today? This means that the firewall of the Pi blockchain has been removed and connected to the outside world. This makes it easier for merchants to sell goods and services in the local market; developers can further develop applications and improve business model logic; creators can gain greater influence; and ordinary Pioneers can also connect and trade with each other. For new developers, welcome to this widely distributed crypto network. If you already have a business model, come to the Pi network to test and sell your products. For developers who don't have a business model but have an excellent application experience, the Pi application network has prepared a business model for you. The platform will bring you traffic and make you profitable. At the same time, it creates the use value of Pi for all pioneers, which is very beneficial to the Pi network as a whole. So, Pioneers, in the end, don't let external noise distract you. Focus on key matters, focus on things that can have an impact in the crypto field and even the world. Keep working hard, keep creating, and everything else will fall into place. MY MESSAGE Please listen carefully to this video so that you understand, the value is determined by the existing ecosystem, the current ecosystem is GCV $314,159 What small traders do under the auspices of the ambassadors 🥰 Eagle woman 🦅 Nonny Padja NTT 🇮🇩 Believe it or not, it's up to you, the point is we've won with a GCV barter value of $314,159 If you want to succeed Barter with Pioneer and small merchants with GCV value of $314,159❤️

NONNY PADJA NTT ❤ Eagle woman 🦅

45,669 просмотров • 1 год назад

Announcing the 2025 Hackathon & GCV Update Happy Saturday, Global Pioneers and Global GCV Ambassadors! This is Doris Yin from Toronto, Canada. How are you doing? I hope everything is going well. Today, I want to share the Core Team announcement about the 2025 Hackathon. The prizes are: First prize: 75,000 Pi Second prize: 45,000 Pi Third prize: 15,000 Pi I know some of you may feel discouraged about GCV because CT uses exchange price for prize. , but there’s no need to worry. Let me explain why. This does not mean GCV has failed, nor does it mean the Core Team supports a low Pi value. As I mentioned before, we have two methods, two paths to strengthen GCV. The first path is offline GCV. This is very important. We still need to implement partial Pi payments at GCV offline, and more merchants are joining in. They appreciate it because it helps local businesses grow. This is already happening in countries like China, Vietnam, the Philippines, Indonesia, Malaysia, Thailand, and India. We can also observe more GCV data being recorded on the blockchain, which is essential to OM fix Pi value. Instead of the Pi value coming from the exchange market, it comes from the community-driven process. You can see this reflected in the blockchain records. Our social media, groups, communities, and education efforts help pioneers recognize and acknowledge the Pi value and GCV. However, we also need online utilities and DApps. This will increase actual usage of Pi after the Open Mainnet. So we have two parts: 1. Value confirmation: We must continue building and creating more GCV data. 2. DApps and utilities usage: We need higher-quality DApps widely usage to guarantee and protect GCV. Currently, DApps are limited and mostly from pioneers, not professionals. We need good-quality DApps. If the Core Team were to simply give a prize of one Pi, it wouldn’t motivate developers—especially top developers inside and outside the community. To truly motivate them, we must use the exchange market price to attract the highest-level developers. For example, the first prize of 75,000 Pi is roughly 20,000–30,000 USD. That’s a meaningful incentive now, and in the future, it could be even higher. Developers can also see the future value of Pi, not just the current 30 cents, which encourages high-quality development. As we develop more, it supports the full realization of GCV. Offline partial Pi payments, partial fiat payments, and industrial alliances in different countries attract more participants. Online, using exchange market price to increase demand and reduce Pi selling pressure, helping to keep the price steadily increasing. When pioneers spend Pi on items, it reduces the circulating supply, creating scarcity, which is positive for the Pi Network and GCV. Some anti-GCV voices laugh and say “GCV has failed,” but they laughed too early because the CT strategy is helping GCV. Even these early critics help GCV by increasing awareness and adoption. I will also write articles to explain this further for everyone. Regarding the Core Team: it wouldn’t make sense for them to target only 30 cents. Their goal is GCV because any project owner wants their product or service to be meaningful and valuable. A higher Pi value is good for the ecosystem. GCV can support the Pi Network long-term, possibly 100 years or even 1,000 years. We don’t have hundreds of billions of Pi in circulation—currently, only about 2 billion exist. When fully released, it will take time, and the value will continue to grow. I hope everyone understands. Happy Saturday! Enjoy your day, keep working, and help the community. I also hope our GCV Ambassadors actively participate in the 2025 Hackathon. Thank you, and goodbye for now! Doris Yin 🪷 🪷🪷 Founder, Global GCV Movement August 16th, 2025
6:41

Announcing the 2025 Hackathon & GCV Update Happy Saturday, Global Pioneers and Global GCV Ambassadors! This is Doris Yin from Toronto, Canada. How are you doing? I hope everything is going well. Today, I want to share the Core Team announcement about the 2025 Hackathon. The prizes are: First prize: 75,000 Pi Second prize: 45,000 Pi Third prize: 15,000 Pi I know some of you may feel discouraged about GCV because CT uses exchange price for prize. , but there’s no need to worry. Let me explain why. This does not mean GCV has failed, nor does it mean the Core Team supports a low Pi value. As I mentioned before, we have two methods, two paths to strengthen GCV. The first path is offline GCV. This is very important. We still need to implement partial Pi payments at GCV offline, and more merchants are joining in. They appreciate it because it helps local businesses grow. This is already happening in countries like China, Vietnam, the Philippines, Indonesia, Malaysia, Thailand, and India. We can also observe more GCV data being recorded on the blockchain, which is essential to OM fix Pi value. Instead of the Pi value coming from the exchange market, it comes from the community-driven process. You can see this reflected in the blockchain records. Our social media, groups, communities, and education efforts help pioneers recognize and acknowledge the Pi value and GCV. However, we also need online utilities and DApps. This will increase actual usage of Pi after the Open Mainnet. So we have two parts: 1. Value confirmation: We must continue building and creating more GCV data. 2. DApps and utilities usage: We need higher-quality DApps widely usage to guarantee and protect GCV. Currently, DApps are limited and mostly from pioneers, not professionals. We need good-quality DApps. If the Core Team were to simply give a prize of one Pi, it wouldn’t motivate developers—especially top developers inside and outside the community. To truly motivate them, we must use the exchange market price to attract the highest-level developers. For example, the first prize of 75,000 Pi is roughly 20,000–30,000 USD. That’s a meaningful incentive now, and in the future, it could be even higher. Developers can also see the future value of Pi, not just the current 30 cents, which encourages high-quality development. As we develop more, it supports the full realization of GCV. Offline partial Pi payments, partial fiat payments, and industrial alliances in different countries attract more participants. Online, using exchange market price to increase demand and reduce Pi selling pressure, helping to keep the price steadily increasing. When pioneers spend Pi on items, it reduces the circulating supply, creating scarcity, which is positive for the Pi Network and GCV. Some anti-GCV voices laugh and say “GCV has failed,” but they laughed too early because the CT strategy is helping GCV. Even these early critics help GCV by increasing awareness and adoption. I will also write articles to explain this further for everyone. Regarding the Core Team: it wouldn’t make sense for them to target only 30 cents. Their goal is GCV because any project owner wants their product or service to be meaningful and valuable. A higher Pi value is good for the ecosystem. GCV can support the Pi Network long-term, possibly 100 years or even 1,000 years. We don’t have hundreds of billions of Pi in circulation—currently, only about 2 billion exist. When fully released, it will take time, and the value will continue to grow. I hope everyone understands. Happy Saturday! Enjoy your day, keep working, and help the community. I also hope our GCV Ambassadors actively participate in the 2025 Hackathon. Thank you, and goodbye for now! Doris Yin 🪷 🪷🪷 Founder, Global GCV Movement August 16th, 2025

Doris Yin 东方紫莲🪷

28,870 просмотров • 11 месяцев назад

I tested Claude Code on a fresh account - 1,500 lines of HTML cost me 50% of my window. Full video and summary is here.. I just ran a recorded test on Claude Code with a fresh account (Pro, not Max - my main account was 20x Max) , and the result is honestly insane. The task was trivial: create 3 simple demo HTML pages, around 500 lines each. Roughly 1,500 lines of code total. Nothing massive. Nothing enterprise-grade. Nothing that should meaningfully stress a premium coding product. And yet Claude Code burned through 40% of my 5-hour window almost immediately. I ran the exact same test with Codex, and it consumed only 2%. Then it got even worse: after the session ended, I did absolutely nothing for 15 minutes, and Claude still ate another 10%. Total: 50% of the 5-hour window gone for a tiny HTML demo. My weekly usage had already started at 2% before I even really used it, and after this tiny test it jumped to 8%. Now let us be generous and assume this entire run used around 30k tokens total. If 30k tokens represents 10% of weekly usage, that implies around 300k tokens per week. That is roughly 1.2M-1.3M tokens per month, and even if you round up aggressively, you are still in the 1.5M token range. Using the Sonnet 4.6 pricing you list: $3 per 1M input tokens $15 per 1M output tokens How exactly is this supposed to make sense for a paid coding product? Because from the user side, this no longer looks like "premium usage protection." It looks like a quota system that is either wildly inefficient, badly broken, or being accounted in a way users are not being told about. And that is before I even get to my main account: my $200 Max plan now dies in a single day. Just a few months ago, similar or heavier usage would last me about a week. So no, I do not buy the "maybe you just used it more" excuse anymore. Something is clearly broken in Claude Code. Either token accounting is broken, context handling is broken, background consumption is broken, or all three. Alex Albert is this really the experience you want users to pay for? Just watch the video. I tried to be very transparent and clear for your team! I was fan of Claude but just disappointed! And if you want, send me the detailed token accounting for this session and let us inspect it together publicly. Because from where I am standing, this is no longer a small pricing annoyance. It looks like something seriously wrong is happening, and users deserve a real explanation.
14:01

I tested Claude Code on a fresh account - 1,500 lines of HTML cost me 50% of my window. Full video and summary is here.. I just ran a recorded test on Claude Code with a fresh account (Pro, not Max - my main account was 20x Max) , and the result is honestly insane. The task was trivial: create 3 simple demo HTML pages, around 500 lines each. Roughly 1,500 lines of code total. Nothing massive. Nothing enterprise-grade. Nothing that should meaningfully stress a premium coding product. And yet Claude Code burned through 40% of my 5-hour window almost immediately. I ran the exact same test with Codex, and it consumed only 2%. Then it got even worse: after the session ended, I did absolutely nothing for 15 minutes, and Claude still ate another 10%. Total: 50% of the 5-hour window gone for a tiny HTML demo. My weekly usage had already started at 2% before I even really used it, and after this tiny test it jumped to 8%. Now let us be generous and assume this entire run used around 30k tokens total. If 30k tokens represents 10% of weekly usage, that implies around 300k tokens per week. That is roughly 1.2M-1.3M tokens per month, and even if you round up aggressively, you are still in the 1.5M token range. Using the Sonnet 4.6 pricing you list: $3 per 1M input tokens $15 per 1M output tokens How exactly is this supposed to make sense for a paid coding product? Because from the user side, this no longer looks like "premium usage protection." It looks like a quota system that is either wildly inefficient, badly broken, or being accounted in a way users are not being told about. And that is before I even get to my main account: my $200 Max plan now dies in a single day. Just a few months ago, similar or heavier usage would last me about a week. So no, I do not buy the "maybe you just used it more" excuse anymore. Something is clearly broken in Claude Code. Either token accounting is broken, context handling is broken, background consumption is broken, or all three. Alex Albert is this really the experience you want users to pay for? Just watch the video. I tried to be very transparent and clear for your team! I was fan of Claude but just disappointed! And if you want, send me the detailed token accounting for this session and let us inspect it together publicly. Because from where I am standing, this is no longer a small pricing annoyance. It looks like something seriously wrong is happening, and users deserve a real explanation.

Hayrettin Tüzel

26,854 просмотров • 4 месяцев назад

Researchers found a way to make LLMs 8.5x faster! (without compromising accuracy) Speculative decoding is quite an effective way to address the single-token bottleneck in traditional LLM inference. A small "draft" model first generates the next several tokens, then the large model verifies all of them at once in a single forward pass. If a token at any position is wrong, you keep everything before it and restart from there. This never does worse than normal decoding. But current drafters in Speculative decoding still guess one token at a time. That makes the drafting step itself a bottleneck, capping real-world speedups at 2-3x. DFlash is a new technique that swaps the autoregressive drafter with a lightweight block diffusion model that guesses all tokens in one parallel shot. Drafting cost stays flat no matter how many tokens you speculate. On top of that, the drafter is conditioned on hidden features pulled from multiple layers of the target model and injected into every draft layer, so it makes significantly better guesses than a drafter working from scratch. In the side-by-side demo below, vanilla decoding runs at 48.5 tokens/sec. DFlash hits 415 tokens/sec on the same model, with zero quality loss. It's already integrated with vLLM, SGLang, and Transformers, with draft models on HuggingFace for several models like Qwen3, Qwen3.5, Llama 3.1, Kimi-K2.5, gpt-oss, and many more. I have shared the GitHub repo in the replies! KV caching is another must-know technique to boost LLM inference. I recently wrote an article about it. Read it below. 👉 Over to you: What use case are you working on that can benefit from this new technique?
0:26

Researchers found a way to make LLMs 8.5x faster! (without compromising accuracy) Speculative decoding is quite an effective way to address the single-token bottleneck in traditional LLM inference. A small "draft" model first generates the next several tokens, then the large model verifies all of them at once in a single forward pass. If a token at any position is wrong, you keep everything before it and restart from there. This never does worse than normal decoding. But current drafters in Speculative decoding still guess one token at a time. That makes the drafting step itself a bottleneck, capping real-world speedups at 2-3x. DFlash is a new technique that swaps the autoregressive drafter with a lightweight block diffusion model that guesses all tokens in one parallel shot. Drafting cost stays flat no matter how many tokens you speculate. On top of that, the drafter is conditioned on hidden features pulled from multiple layers of the target model and injected into every draft layer, so it makes significantly better guesses than a drafter working from scratch. In the side-by-side demo below, vanilla decoding runs at 48.5 tokens/sec. DFlash hits 415 tokens/sec on the same model, with zero quality loss. It's already integrated with vLLM, SGLang, and Transformers, with draft models on HuggingFace for several models like Qwen3, Qwen3.5, Llama 3.1, Kimi-K2.5, gpt-oss, and many more. I have shared the GitHub repo in the replies! KV caching is another must-know technique to boost LLM inference. I recently wrote an article about it. Read it below. 👉 Over to you: What use case are you working on that can benefit from this new technique?

Avi Chawla

157,390 просмотров • 2 месяцев назад

**Doris Yin - 3/28/25 International GCV Group Speech** Pioneers, I encourage you to read all my articles to grasp the big picture. It is essential to understand that different phases come with different tasks. I find that many questions arise from a lack of understanding of these phases and tasks. Your primary task is to understand this first and then help your community to understand as well. 1. We have completed the first stage of coinage at GCV, and we have already succeeded! The Pi GCV value of $314,159 has been encoded. This stage focuses on building Pi value within our community through the enclosed mainnet. The blockchain records are critical, as they will determine the value of Pi in code. 2. Now we are entering a new stage: the Open Network. This means we need to expand our reach to the world. We want Pi GCV to be accepted by governments and regulatory bodies, large companies, and a broader audience. That is why we are engaging in the exchange market, which serves as a bridge to implement the GCV code. Do you understand? We at GCV are taking charge of the entire Pi Network community! GCV is the champion 🏆. This is why blockchain records are now less important than the exchange market in this new stage. Our battlefield has shifted from the blockchain to the exchange. The exchange markets are where Pi can be accepted as a stable coin according to the GCV! How do we achieve that? 1. Increase the demand for Pi in the exchange market. 2. Control the supply of Pi in the exchange market. 3. Educate Pioneers not to sell, except to GCV. Currently, Pi CT is also working on this by using domain name auctions to enhance the demand for Pi, especially from external companies. The goal is to attract more businesses to our ecosystem and bring in more users. At present, we have 11 million wallets, which means there are 11 million Pioneers holding Pi wallets. This is a small population, and we aim to become a global currency. However, we face challenges with only 11 million Pioneers. After implementing the codes, the mining speed will drop to 0.1 or lower annually. At that time, the price will be pegged to GCV, making it difficult for most people to afford 0.01. But don’t worry about those who do not have Pi wallets if you buy from exchange market, it is easy for CT to provide this kind of wallet for the ecosystem. You will have auction purpose wallet to HOLD your Pi from exchange soon. How can we further develop if we don't have enough companies and users of Pi? This is why CT needs lower prices in the exchange market to attract companies and people. Do you understand? Utilize the three-month marketing strategy for Pi! After three months, all low-priced Pi can be eliminated. Then we will have the power to set the Pi price in the exchange market, solidifying Pi as a stable coin according to the code! CT is not allowed to interfere with or guide the exchange market. Therefore, we, the Pioneers, must take responsibility for helping to expand Pi usage. If you understand this logic, do you still have doubts? Everything is in the hands of CT and our GCV CT! This is our shared strategy. None of you should be passive. You have a responsibility to help your community build confidence so that you can cooperate and promote the Domain Auction, encouraging others to join Web3 blockchain and use Pi as payment! Any complaints stem from a lack of understanding. If you cannot think things through, be humble enough to learn. All doubt comes from arrogance and ignorance. We are on the verge of conquering the world and establishing Pi as a global currency! But we need all Pioneers unified and working hard. If complaining could resolve your financial problems, we would work diligently to complain. But can it? Of course not! It wastes both your time and ours and hinders our progress. I have muted the group so that you can read this message clearly. If you can’t contribute positively, I encourage you to find a job to earn money for yourself. Please don’t come to the group to spread complaints and negativity. I have my own job and responsibilities, and I will never quit my job. If you have nothing to do and no job, you may become desperate. So please find a job and work on improving yourself. Once you stabilize your own situation, you are welcome to join the group as a volunteer to help others. Only when we clearly see our path can we walk steadily and reach our destination quickly and successfully. Have a wonderful day! ❤️ Doris Yin 🪷🪷🪷 PS: I've been very busy lately with tax season work, so I don't have time to write articles. However, I still participate in some group daily.
0:46

**Doris Yin - 3/28/25 International GCV Group Speech** Pioneers, I encourage you to read all my articles to grasp the big picture. It is essential to understand that different phases come with different tasks. I find that many questions arise from a lack of understanding of these phases and tasks. Your primary task is to understand this first and then help your community to understand as well. 1. We have completed the first stage of coinage at GCV, and we have already succeeded! The Pi GCV value of $314,159 has been encoded. This stage focuses on building Pi value within our community through the enclosed mainnet. The blockchain records are critical, as they will determine the value of Pi in code. 2. Now we are entering a new stage: the Open Network. This means we need to expand our reach to the world. We want Pi GCV to be accepted by governments and regulatory bodies, large companies, and a broader audience. That is why we are engaging in the exchange market, which serves as a bridge to implement the GCV code. Do you understand? We at GCV are taking charge of the entire Pi Network community! GCV is the champion 🏆. This is why blockchain records are now less important than the exchange market in this new stage. Our battlefield has shifted from the blockchain to the exchange. The exchange markets are where Pi can be accepted as a stable coin according to the GCV! How do we achieve that? 1. Increase the demand for Pi in the exchange market. 2. Control the supply of Pi in the exchange market. 3. Educate Pioneers not to sell, except to GCV. Currently, Pi CT is also working on this by using domain name auctions to enhance the demand for Pi, especially from external companies. The goal is to attract more businesses to our ecosystem and bring in more users. At present, we have 11 million wallets, which means there are 11 million Pioneers holding Pi wallets. This is a small population, and we aim to become a global currency. However, we face challenges with only 11 million Pioneers. After implementing the codes, the mining speed will drop to 0.1 or lower annually. At that time, the price will be pegged to GCV, making it difficult for most people to afford 0.01. But don’t worry about those who do not have Pi wallets if you buy from exchange market, it is easy for CT to provide this kind of wallet for the ecosystem. You will have auction purpose wallet to HOLD your Pi from exchange soon. How can we further develop if we don't have enough companies and users of Pi? This is why CT needs lower prices in the exchange market to attract companies and people. Do you understand? Utilize the three-month marketing strategy for Pi! After three months, all low-priced Pi can be eliminated. Then we will have the power to set the Pi price in the exchange market, solidifying Pi as a stable coin according to the code! CT is not allowed to interfere with or guide the exchange market. Therefore, we, the Pioneers, must take responsibility for helping to expand Pi usage. If you understand this logic, do you still have doubts? Everything is in the hands of CT and our GCV CT! This is our shared strategy. None of you should be passive. You have a responsibility to help your community build confidence so that you can cooperate and promote the Domain Auction, encouraging others to join Web3 blockchain and use Pi as payment! Any complaints stem from a lack of understanding. If you cannot think things through, be humble enough to learn. All doubt comes from arrogance and ignorance. We are on the verge of conquering the world and establishing Pi as a global currency! But we need all Pioneers unified and working hard. If complaining could resolve your financial problems, we would work diligently to complain. But can it? Of course not! It wastes both your time and ours and hinders our progress. I have muted the group so that you can read this message clearly. If you can’t contribute positively, I encourage you to find a job to earn money for yourself. Please don’t come to the group to spread complaints and negativity. I have my own job and responsibilities, and I will never quit my job. If you have nothing to do and no job, you may become desperate. So please find a job and work on improving yourself. Once you stabilize your own situation, you are welcome to join the group as a volunteer to help others. Only when we clearly see our path can we walk steadily and reach our destination quickly and successfully. Have a wonderful day! ❤️ Doris Yin 🪷🪷🪷 PS: I've been very busy lately with tax season work, so I don't have time to write articles. However, I still participate in some group daily.

Doris Yin 东方紫莲🪷

20,703 просмотров • 1 год назад

GCV Movement: Confidence, Stability, and the Power of Pioneers This is Doris Yin in Toronto, Canada. Today is a beautiful Sunday morning around my condo, and I hope everyone is enjoying the weekend with their family. Our topic today is confidence. We understand what others may want to do with Pi Network. But if you take a moment to read the white paper — just the first few pages — you will understand what Pi Network truly is and how it differs from traditional cryptocurrency. We are doing something completely different. Traditional cryptocurrencies cannot use the blockchain to serve ordinary people. That is exactly why Dr. Nicolas created Pi Network — to build a currency that ordinary people can actually use. If you understand this vision, you also understand Global Consensus Value (GCV). Only GCV can realize Pi as a real currency. And a true currency must have two essential characteristics: 1. Stability – Its value cannot fluctuate wildly like Bitcoin or other cryptocurrencies. 2. Usability – It must be usable in everyday life by ordinary people. Why Stability Matters In traditional cryptocurrency, tokens behave almost like commodities or stocks. People buy and sell for profit when prices go up or down. This speculation creates instability, and instability prevents cryptocurrencies from serving as real money. Pi Network is different. Its mission is to provide a stable, usable currency. That’s why we focus on creating Global Consensus Value (GCV) — the mechanism through which pioneers themselves determine the value of Pi. Who Creates the Value of Pi? Dr. Nicolas has made it clear: the value of Pi is created by pioneers, not the Core Team. The Core Team builds the technology, but we, the pioneers, hold the power to set Pi’s value. Confidence is essential — confidence in the Core Team, yes, but also confidence in ourselves. Some pioneers lack confidence because they only passively mine. But if you participate actively — as a GCV Ambassador, a Pi Warrior, or a contributing pioneer — you will understand the movement and your role in it. Two Steps to a Mature Pi Economy Pi’s development has two key sides: Technical side: Already open. The blockchain infrastructure is connected to the world. Economic side: Not yet open. Real economic activity must wait for GCV implementation to create a mature ecosystem. This is why we focus on ecosystem depth — businesses, services, and real-life use cases that give Pi weight and support its value. Mind + Depth = Stable Value Our mind (pioneers’ consensus) fixes the value. Our depth (ecosystem adoption) supports the value. Together, they make Pi a living, functional currency. Become a Pi Warrior Today, many pioneers and social media supporters are spreading the message of GCV. But many writers and outsiders still don’t understand Pi or GCV. That is why it is our responsibility to share, teach, and invite more pioneers into the movement. We are not only pioneers — we are warriors protecting the Pi Network. The outside world may not yet understand, but we do. And it is our duty to ensure more pioneers learn, support, and actively participate. If you understand GCV, don’t keep it to yourself. Share it, pass it on, and invite others to join. Protect Pi. Build confidence. Build the ecosystem. A Personal Note As I speak this early morning in Toronto, I feel energized from my workouts and yoga. I encourage all pioneers to take care of your health. A strong body supports a positive mindset, and a positive mindset allows us to contribute more to Pi’s mission. Thank you for your attention. Enjoy your day. Stay confident, stay united, and together we will make Pi the world’s first stable, global, people’s currency! 🌍✨ 🔥 Pioneers Create the Value — Warriors Protect the Network! 🔥 Doris Yin 🪷 🪷🪷
6:09

GCV Movement: Confidence, Stability, and the Power of Pioneers This is Doris Yin in Toronto, Canada. Today is a beautiful Sunday morning around my condo, and I hope everyone is enjoying the weekend with their family. Our topic today is confidence. We understand what others may want to do with Pi Network. But if you take a moment to read the white paper — just the first few pages — you will understand what Pi Network truly is and how it differs from traditional cryptocurrency. We are doing something completely different. Traditional cryptocurrencies cannot use the blockchain to serve ordinary people. That is exactly why Dr. Nicolas created Pi Network — to build a currency that ordinary people can actually use. If you understand this vision, you also understand Global Consensus Value (GCV). Only GCV can realize Pi as a real currency. And a true currency must have two essential characteristics: 1. Stability – Its value cannot fluctuate wildly like Bitcoin or other cryptocurrencies. 2. Usability – It must be usable in everyday life by ordinary people. Why Stability Matters In traditional cryptocurrency, tokens behave almost like commodities or stocks. People buy and sell for profit when prices go up or down. This speculation creates instability, and instability prevents cryptocurrencies from serving as real money. Pi Network is different. Its mission is to provide a stable, usable currency. That’s why we focus on creating Global Consensus Value (GCV) — the mechanism through which pioneers themselves determine the value of Pi. Who Creates the Value of Pi? Dr. Nicolas has made it clear: the value of Pi is created by pioneers, not the Core Team. The Core Team builds the technology, but we, the pioneers, hold the power to set Pi’s value. Confidence is essential — confidence in the Core Team, yes, but also confidence in ourselves. Some pioneers lack confidence because they only passively mine. But if you participate actively — as a GCV Ambassador, a Pi Warrior, or a contributing pioneer — you will understand the movement and your role in it. Two Steps to a Mature Pi Economy Pi’s development has two key sides: Technical side: Already open. The blockchain infrastructure is connected to the world. Economic side: Not yet open. Real economic activity must wait for GCV implementation to create a mature ecosystem. This is why we focus on ecosystem depth — businesses, services, and real-life use cases that give Pi weight and support its value. Mind + Depth = Stable Value Our mind (pioneers’ consensus) fixes the value. Our depth (ecosystem adoption) supports the value. Together, they make Pi a living, functional currency. Become a Pi Warrior Today, many pioneers and social media supporters are spreading the message of GCV. But many writers and outsiders still don’t understand Pi or GCV. That is why it is our responsibility to share, teach, and invite more pioneers into the movement. We are not only pioneers — we are warriors protecting the Pi Network. The outside world may not yet understand, but we do. And it is our duty to ensure more pioneers learn, support, and actively participate. If you understand GCV, don’t keep it to yourself. Share it, pass it on, and invite others to join. Protect Pi. Build confidence. Build the ecosystem. A Personal Note As I speak this early morning in Toronto, I feel energized from my workouts and yoga. I encourage all pioneers to take care of your health. A strong body supports a positive mindset, and a positive mindset allows us to contribute more to Pi’s mission. Thank you for your attention. Enjoy your day. Stay confident, stay united, and together we will make Pi the world’s first stable, global, people’s currency! 🌍✨ 🔥 Pioneers Create the Value — Warriors Protect the Network! 🔥 Doris Yin 🪷 🪷🪷

Doris Yin 东方紫莲🪷

10,706 просмотров • 11 месяцев назад

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.
0:23

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

Avi Chawla

269,667 просмотров • 1 месяц назад

What is an Automated Market Maker (AMM)? ************* Notes: The video is attached. Links to the shorts on other platforms are in the second Tweet. Voting on fees using your LP tokens is limited to the 8 biggest LP holders. For auctions, the discount gets you close to 0% but not effectively 0%. The math example couldn’t exist in reality based on G3M. It is just an example to explain the process. ************* Imagine a robot (AMM) that's always ready to help you trade your money (assets) with others on a special online platform (the XRP Ledger's decentralised exchange). This robot doesn’t need to find someone else to take the other side of your trade; it just makes the trade happen using a big pot of money (pool) it manages. How Does it Work? Trading: You can swap one type of money (asset) for another anytime you want, using the robot’s pot of money. The robot uses a special formula to decide the swap rate. Creating a Pool: Anyone can create a new pot of money for two different types of assets if it doesn’t exist yet, or add to an existing one. Rewards for Pool Creators: People who add money to the pot (liquidity providers) get special tokens (LP Tokens) as a thank-you. These tokens can be used to: • Get a share of the money in the pot back, along with some extra (fees collected). • Have a say in changing the robot’s settings, like trading fees. • Bid to get a temporary discount on trading fees. Risks and Rewards: If many people are swapping money and the pot stays balanced, the people who added money to the pot earn some passive income from the fees. But, if the value of the assets changes a lot, they might lose some money. More Technical Bits: Exchange Rate: The robot adjusts the swap rate based on how much each asset has in its pot. If it has a lot of one asset, that asset becomes cheaper to swap. Trading Fees: The robot charges a small fee for each swap, which goes to the people who added money to the pot. Voting on Fees: People with LP Tokens can vote to change the trading fee; the more tokens you have, the more your vote counts. Auction Slot: There’s a special feature where you can bid to get a discount on trading fees for a day. You bid with LP Tokens, and if you win, you (and up to 4 friends) pay no trading fees for 24 hours. LP Tokens: These are special tokens you get for adding money to the pot. They can be traded, used to vote on fees, or redeemed to pull your money out of the pot. Deleting an AMM: If all the money gets pulled out of the pot, the robot (AMM) gets deleted. But, it can be recreated by adding money to the pot again. In a Nutshell: An AMM is like a robot banker that helps people easily swap different types of money using a big shared pot. People who add money to the pot get special tokens and can earn fees from the swaps, but there are some risks if the market changes a lot. They can also vote on settings and bid for fee discounts. If the pot empties, the robot goes away but can be brought back by refilling the pot. Let’s look at an Example: Step 1: Creating a Money Pot with the AMM Robot Alice creates a new money pot (AMM) using the robot. She chooses two types of money: US Dollars (USD) and XRP. She puts in: 1000 USD 10 XRP In this example, we assume an exchange rate of $1 per XRP for easy math. Alice gets special tokens (LP Tokens) from the robot as a thank-you for adding money to the pot. These tokens prove she added money and can be used later to get her money back, plus some extra if the robot earns fees. Step 2: Bob Makes a Swap Bob wants to swap his 100 USD for XRP. He doesn’t have to wait for someone to take his offer; the robot does it instantly using the money in the pot. The robot uses a formula to decide how much XRP Bob gets for his 100 USD, ensuring it's a fair rate based on how much USD and XRP are in the pot. Step 3: Earning Fees The robot charges Bob a small fee, let’s say 1%, for convenience. So, Bob pays 1 USD as a fee, which stays in the pot. Now, the pot has more money than it started with, which is good for Alice because she can earn that extra when she uses her LP Tokens. Step 4: Alice Withdraws Her Money After some time, Alice decides to take her money out of the pot. Thanks to the fees the robot earned from Bob and others, the pot has grown to: 1101 USD 9 XRP Alice uses her LP Tokens to claim her share of the money in the pot. If she puts in 100% of the original money, she gets 100% of what’s in the pot now, including the extra earned from fees. Step 5: Voting and Discounts Alice can also use her LP Tokens to vote on things, like changing the robot’s fee. And, if she wants a discount on fees, she can bid for a special 24-hour discount slot using her LP Tokens. Conclusion In this example, the AMM robot helped Alice earn extra money by providing a convenient way for Bob to swap his USD for XRP. Alice took on some risk by putting her money into the pot, but she earned fees from Bob’s and others’ trades as a reward. Bob enjoyed the convenience of instant, hassle-free trading. And the AMM robot managed it all automatically!
6:08

What is an Automated Market Maker (AMM)? ************* Notes: The video is attached. Links to the shorts on other platforms are in the second Tweet. Voting on fees using your LP tokens is limited to the 8 biggest LP holders. For auctions, the discount gets you close to 0% but not effectively 0%. The math example couldn’t exist in reality based on G3M. It is just an example to explain the process. ************* Imagine a robot (AMM) that's always ready to help you trade your money (assets) with others on a special online platform (the XRP Ledger's decentralised exchange). This robot doesn’t need to find someone else to take the other side of your trade; it just makes the trade happen using a big pot of money (pool) it manages. How Does it Work? Trading: You can swap one type of money (asset) for another anytime you want, using the robot’s pot of money. The robot uses a special formula to decide the swap rate. Creating a Pool: Anyone can create a new pot of money for two different types of assets if it doesn’t exist yet, or add to an existing one. Rewards for Pool Creators: People who add money to the pot (liquidity providers) get special tokens (LP Tokens) as a thank-you. These tokens can be used to: • Get a share of the money in the pot back, along with some extra (fees collected). • Have a say in changing the robot’s settings, like trading fees. • Bid to get a temporary discount on trading fees. Risks and Rewards: If many people are swapping money and the pot stays balanced, the people who added money to the pot earn some passive income from the fees. But, if the value of the assets changes a lot, they might lose some money. More Technical Bits: Exchange Rate: The robot adjusts the swap rate based on how much each asset has in its pot. If it has a lot of one asset, that asset becomes cheaper to swap. Trading Fees: The robot charges a small fee for each swap, which goes to the people who added money to the pot. Voting on Fees: People with LP Tokens can vote to change the trading fee; the more tokens you have, the more your vote counts. Auction Slot: There’s a special feature where you can bid to get a discount on trading fees for a day. You bid with LP Tokens, and if you win, you (and up to 4 friends) pay no trading fees for 24 hours. LP Tokens: These are special tokens you get for adding money to the pot. They can be traded, used to vote on fees, or redeemed to pull your money out of the pot. Deleting an AMM: If all the money gets pulled out of the pot, the robot (AMM) gets deleted. But, it can be recreated by adding money to the pot again. In a Nutshell: An AMM is like a robot banker that helps people easily swap different types of money using a big shared pot. People who add money to the pot get special tokens and can earn fees from the swaps, but there are some risks if the market changes a lot. They can also vote on settings and bid for fee discounts. If the pot empties, the robot goes away but can be brought back by refilling the pot. Let’s look at an Example: Step 1: Creating a Money Pot with the AMM Robot Alice creates a new money pot (AMM) using the robot. She chooses two types of money: US Dollars (USD) and XRP. She puts in: 1000 USD 10 XRP In this example, we assume an exchange rate of $1 per XRP for easy math. Alice gets special tokens (LP Tokens) from the robot as a thank-you for adding money to the pot. These tokens prove she added money and can be used later to get her money back, plus some extra if the robot earns fees. Step 2: Bob Makes a Swap Bob wants to swap his 100 USD for XRP. He doesn’t have to wait for someone to take his offer; the robot does it instantly using the money in the pot. The robot uses a formula to decide how much XRP Bob gets for his 100 USD, ensuring it's a fair rate based on how much USD and XRP are in the pot. Step 3: Earning Fees The robot charges Bob a small fee, let’s say 1%, for convenience. So, Bob pays 1 USD as a fee, which stays in the pot. Now, the pot has more money than it started with, which is good for Alice because she can earn that extra when she uses her LP Tokens. Step 4: Alice Withdraws Her Money After some time, Alice decides to take her money out of the pot. Thanks to the fees the robot earned from Bob and others, the pot has grown to: 1101 USD 9 XRP Alice uses her LP Tokens to claim her share of the money in the pot. If she puts in 100% of the original money, she gets 100% of what’s in the pot now, including the extra earned from fees. Step 5: Voting and Discounts Alice can also use her LP Tokens to vote on things, like changing the robot’s fee. And, if she wants a discount on fees, she can bid for a special 24-hour discount slot using her LP Tokens. Conclusion In this example, the AMM robot helped Alice earn extra money by providing a convenient way for Bob to swap his USD for XRP. Alice took on some risk by putting her money into the pot, but she earned fees from Bob’s and others’ trades as a reward. Bob enjoyed the convenience of instant, hassle-free trading. And the AMM robot managed it all automatically!

Daniel "CEO of the XRPL" Keller

238,486 просмотров • 2 лет назад

Stop paying $20–50/month for your AI agent to browse the web. i built the free version no API keys, no config, no rate limits😳 it's called Hound. An MCP server that does fetch, search, crawl, and screenshot — all keyless and now it ships a native Pi extension, so you get all 6 tools as first-class Pi tools, not through a generic MCP adapter what you get: → web_fetch — anti-bot fetch → clean markdown. PDFs get section maps + auto-OCR. Dead pages auto-recover from the Internet archive (honestly marked, never pretending it's live) → web_search — 10 keyless backends in parallel (DuckDuckGo, Brave, Mojeek, Yandex, Google…) + a local neural reranker → web_crawl — best-first same-domain walk. sitemap mode maps a whole site in one fetch → web_screenshot — anti-bot shots for multimodal models → cache_clear + hound_version what it replaces: -tavily — free tier -firecrawl — $16–50/mo -searXNG + browser + extractor glue — fragile -hound — $0, keyless, forever why it matters: >agent research needs web access, but everyone gates it behind a key or a limit >hound is honest about errors — a 404 says "Page doesn't exist" instead of dumping an error page as if it were content >archive recovery tags source= so the agent knows the content is historical, not live 2-minute setup: -pip install hound-mcp[all] -pi install npm: @houndmcp/hound-mcp-pi -/reload → tools are live. That's it Caveats (read): dataDome, Akamai, Cloudflare Turnstile — no free tool bypasses these hound tries the stealthy browser, then tells you to switch sources the [all] extra is ~100MB (onnxruntime + OCR). Works without it, but full install is recommended not a scraping-at-scale tool. Crawl caps at 100 pages by default — built for agent research Give your agent the web for free
1:12

Stop paying $20–50/month for your AI agent to browse the web. i built the free version no API keys, no config, no rate limits😳 it's called Hound. An MCP server that does fetch, search, crawl, and screenshot — all keyless and now it ships a native Pi extension, so you get all 6 tools as first-class Pi tools, not through a generic MCP adapter what you get: → web_fetch — anti-bot fetch → clean markdown. PDFs get section maps + auto-OCR. Dead pages auto-recover from the Internet archive (honestly marked, never pretending it's live) → web_search — 10 keyless backends in parallel (DuckDuckGo, Brave, Mojeek, Yandex, Google…) + a local neural reranker → web_crawl — best-first same-domain walk. sitemap mode maps a whole site in one fetch → web_screenshot — anti-bot shots for multimodal models → cache_clear + hound_version what it replaces: -tavily — free tier -firecrawl — $16–50/mo -searXNG + browser + extractor glue — fragile -hound — $0, keyless, forever why it matters: >agent research needs web access, but everyone gates it behind a key or a limit >hound is honest about errors — a 404 says "Page doesn't exist" instead of dumping an error page as if it were content >archive recovery tags source= so the agent knows the content is historical, not live 2-minute setup: -pip install hound-mcp[all] -pi install npm: @houndmcp/hound-mcp-pi -/reload → tools are live. That's it Caveats (read): dataDome, Akamai, Cloudflare Turnstile — no free tool bypasses these hound tries the stealthy browser, then tells you to switch sources the [all] extra is ~100MB (onnxruntime + OCR). Works without it, but full install is recommended not a scraping-at-scale tool. Crawl caps at 100 pages by default — built for agent research Give your agent the web for free

Nahid

43,303 просмотров • 10 дней назад