Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

OpenHands crushed Codex by 2.3× on token efficiency! We gave both agents the same task on the same model (Qwen3.5 35B): build an 8-bit Space Invaders in 3 iterations (build, fix, polish). Output: • OpenHands: 219K tokens, • Codex: 513K tokens, OpenHands beats Codex in local running. The difference... show more

atomic.chat

12,243 subscribers

28,088 просмотров • 8 дней назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Live on Cline (v3.67.1) OpenAI 's GPT 5.3 Codex. The speed and token efficiency improvements are real. Here is what's new: > 25% faster than 5.2 Codex > #1 on SWE-Bench Pro (4 different programming languages) > Fewer tokens per task than any prior OpenAI model Runs cost less and finish faster. You can use it in Cline right now without setting up an API key. Select the model and try it on your repo.

Live on Cline (v3.67.1) OpenAI 's GPT 5.3 Codex. The speed and token efficiency improvements are real. Here is what's new: > 25% faster than 5.2 Codex > #1 on SWE-Bench Pro (4 different programming languages) > Fewer tokens per task than any prior OpenAI model Runs cost less and finish faster. You can use it in Cline right now without setting up an API key. Select the model and try it on your repo.

Cline

21,291 просмотров • 5 месяцев назад

Today, we launched browser use inside Codex to further close the build & verify loop for local development! Now, you can ask Codex to build your front end, and test it like a user would by clicking through the app. Codex sees everything a user sees through vision & checks the network/console logs to help debug & fix any issues that it finds. This change brings us closer to fully autonomous coding agents that delivers high quality and tested changes. Watch Codex test my app in the browser, catch & fix a real bug, and doing that loop again with a brand new feature.

James Sun

226,529 просмотров • 3 месяцев назад

Qwen3.5 runs quite well in mlx-lm. Awesome that we have a frontier-level hybrid model. The context gets longer but the inference speed and memory use barely change. Here's the Q4 generating a space invaders game on an M3 Ultra. Generated 4,120 tokens at 37.6 tok/s.

Qwen3.5 runs quite well in mlx-lm. Awesome that we have a frontier-level hybrid model. The context gets longer but the inference speed and memory use barely change. Here's the Q4 generating a space invaders game on an M3 Ultra. Generated 4,120 tokens at 37.6 tok/s.

Awni Hannun

48,673 просмотров • 5 месяцев назад

⚡️ JUST IN: MUSK LAUNCHES GROK 4.5, CALLING IT "OPUS-CLASS" AI MODEL Musk just released Grok 4.5 on Grok Build, Cursor, and the SpaceXAI Console, calling it an "Opus-class" AI model. He says it is faster, more token-efficient and lower cost, with EU access expected by mid-July. Built on a 1.5 TRILLION parameter foundation model, it drops the same day as OpenAI's GPT-5.6. API pricing is $2 per million input tokens and $6 per million output tokens.

⚡️ JUST IN: MUSK LAUNCHES GROK 4.5, CALLING IT "OPUS-CLASS" AI MODEL Musk just released Grok 4.5 on Grok Build, Cursor, and the SpaceXAI Console, calling it an "Opus-class" AI model. He says it is faster, more token-efficient and lower cost, with EU access expected by mid-July. Built on a 1.5 TRILLION parameter foundation model, it drops the same day as OpenAI's GPT-5.6. API pricing is $2 per million input tokens and $6 per million output tokens.

Coin Bureau

50,488 просмотров • 17 дней назад

THE 64 MINUTE OPENAI CODEX MASTERCLASS IS HERE if you've been meaning to learn Codex, this is the episode for you, we cover: 1/ Claude vs Codex (what's better?) 2/How to run Claude Code inside Codex (really cool) 3/Why GPT 5.5 changed everything for browser agents 4/How to create skills (reusable agents you build once and call by name forever) 5/How to connect Notion with surgical permissions (one database, not the whole workspace) 6/How to use Remotion inside Codex to create videos 7/ How to one-shot full mobile apps in Swift 8/The 4 projects to run on day one to get up to speed fast shoutout to Riley Brown for coming onto The Startup Ideas Podcast (SIP) 🧃 for the 5th time. every episode he delivers. this one is the Codex masterclass. if you've been meaning to learn Codex, this is the episode. 100% free i just want to see you build your ideas. watch

THE 64 MINUTE OPENAI CODEX MASTERCLASS IS HERE if you've been meaning to learn Codex, this is the episode for you, we cover: 1/ Claude vs Codex (what's better?) 2/How to run Claude Code inside Codex (really cool) 3/Why GPT 5.5 changed everything for browser agents 4/How to create skills (reusable agents you build once and call by name forever) 5/How to connect Notion with surgical permissions (one database, not the whole workspace) 6/How to use Remotion inside Codex to create videos 7/ How to one-shot full mobile apps in Swift 8/The 4 projects to run on day one to get up to speed fast shoutout to Riley Brown for coming onto The Startup Ideas Podcast (SIP) 🧃 for the 5th time. every episode he delivers. this one is the Codex masterclass. if you've been meaning to learn Codex, this is the episode. 100% free i just want to see you build your ideas. watch

GREG ISENBERG

199,076 просмотров • 2 месяцев назад

1-bit Hy3 running locally is 2.2x faster than its API at the same quality! We gave both models the same task and compared one-shot outputs. 1-bit Hy3 295B GGUF (92GB) ran locally on 4x RTX 5090 with 128GB VRAM against the same Hy3 over cloud API Tasks: - Flappy Bird - Arkanoid - Snake Outputs: Hy3 1-bit local: 76.9K tokens, 15.5 min Hy3 cloud API: 75.1K tokens, 34.3 min The 1-bit games look the same as the API ones. Birds fly through the pipes, bricks break, the snake eats and grows. Nothing froze or crashed. Both models even made the same slip: the snake can cross itself and the game does not end. Getting this quality from 1 bit running locally is wild! Run Hy3 GGUF yourself in Atomic Chat in 2 clicks

1-bit Hy3 running locally is 2.2x faster than its API at the same quality! We gave both models the same task and compared one-shot outputs. 1-bit Hy3 295B GGUF (92GB) ran locally on 4x RTX 5090 with 128GB VRAM against the same Hy3 over cloud API Tasks: - Flappy Bird - Arkanoid - Snake Outputs: Hy3 1-bit local: 76.9K tokens, 15.5 min Hy3 cloud API: 75.1K tokens, 34.3 min The 1-bit games look the same as the API ones. Birds fly through the pipes, bricks break, the snake eats and grows. Nothing froze or crashed. Both models even made the same slip: the snake can cross itself and the game does not end. Getting this quality from 1 bit running locally is wild! Run Hy3 GGUF yourself in Atomic Chat in 2 clicks

atomic.chat

89,709 просмотров • 11 дней назад

OpenAI's chairman Brett Taylor on the new Chinese model Kimi K3. the timeline thinks it's a frontier killer. he says every CEO he talks to asks the same two things: am I getting value from my token spend, AND what's my moat in the age of AI. on token efficiency, he says the frontier models still finish the same task in far fewer tokens. worth a listen.

OpenAI's chairman Brett Taylor on the new Chinese model Kimi K3. the timeline thinks it's a frontier killer. he says every CEO he talks to asks the same two things: am I getting value from my token spend, AND what's my moat in the age of AI. on token efficiency, he says the frontier models still finish the same task in far fewer tokens. worth a listen.

Damian Player

467,291 просмотров • 6 дней назад

🚨 OpenAI just launched Codex, a brand-new autonomous coding agent that can build features and fix bugs on its own. We’ve been using it Every 📧 for a few days, and I’m impressed. I invited Alexander Embiricos (ben davies), a member of the product staff responsible for Codex, to demo Codex and talk about it live on a special edition of AI & I: What Codex is and how it works Codex is designed to be used by senior engineers—it performs coding tasks like adding features or fixing bugs autonomously. It's built to allow you to start many sessions at once, so you can have multiple agents working in parallel. Codex is built to have "taste" OpenAI trained Codex to have the taste of a senior software engineer. It knows how big codebases work, how to write a good PR, and uses clean, minimal code. Why an “abundance mindset” is best for interacting with agents Codex is designed to allow users to delegate many tasks at once without getting caught up in the details. This lets you point an abundance of agents at a specific task like a difficult bug—it’s worth it even if only one of them succeeds. How OpenAI is thinking about agents Codex is one piece of a unified super-assistant OpenAI wants to eventually build—an agent that helps users easily get things done by selecting the right tools for them behind the scenes. OpenAI’s vision for the future of programming In the future developers will probably spend less time writing routine code and more time guiding agents, reviewing their work, and making strategy decisions. Programming will become more social, letting teams easily delegate multiple tasks at once, allowing people to focus on ideas and collaboration instead of routine coding. Watch below!

Dan Shipper 📧

145,487 просмотров • 1 год назад

Following up on our Intelligence Index v4.1 release yesterday, in the video below, Daniel from our team shares a short overview of what's changed: 1. Three upgraded evaluations: Terminal-Bench 2.1, τ³-Bench Banking and GDPval-AA v2 2. Cost, time, and tokens per task: Understand the cost, time, and tokens of tasks across our Index and for individual evals, and how these trade off against Intelligence 3. Cached input token reporting: We now report the amount of cached tokens a particular model uses and how this influences cost

Following up on our Intelligence Index v4.1 release yesterday, in the video below, Daniel from our team shares a short overview of what's changed: 1. Three upgraded evaluations: Terminal-Bench 2.1, τ³-Bench Banking and GDPval-AA v2 2. Cost, time, and tokens per task: Understand the cost, time, and tokens of tasks across our Index and for individual evals, and how these trade off against Intelligence 3. Cached input token reporting: We now report the amount of cached tokens a particular model uses and how this influences cost

Artificial Analysis

12,491 просмотров • 1 месяц назад

New video of Qualcomm CEO Cristiano Amon: AI will require “gazillions” of tokens. Because, Agentic AI will consume dramatically more tokens because it performs autonomous tasks, uses multiple systems, and interacts with tools. AI demand will grow hugely when software starts letting agents act, not just answer. A chatbot spends tokens on language; an agent spends tokens on deciding, checking, calling tools, reading outputs, revising plans, and coordinating with other software. Today a single human-AI exchange may be large, a reasoning task may be much larger, but we are already entering the agentic era, where an autonomous workflow can become exponentially larger still because the model is no longer producing one response. It is running a process. When a SaaS product redesigns itself around human-agent interaction, every task can become a chain of hidden micro-decisions, and each micro-decision consumes context, memory, tool calls, verification, and output tokens. Another point is people hear “more tokens” and think only of bigger data centers, when the deeper change is economic: software usage may stop being measured mainly by clicks, seats, or sessions. It will be measured by how much machine reasoning/tokens gets spent on behalf of each user. ---- From "Reuters" YouTube channel, (link in comment)

New video of Qualcomm CEO Cristiano Amon: AI will require “gazillions” of tokens. Because, Agentic AI will consume dramatically more tokens because it performs autonomous tasks, uses multiple systems, and interacts with tools. AI demand will grow hugely when software starts letting agents act, not just answer. A chatbot spends tokens on language; an agent spends tokens on deciding, checking, calling tools, reading outputs, revising plans, and coordinating with other software. Today a single human-AI exchange may be large, a reasoning task may be much larger, but we are already entering the agentic era, where an autonomous workflow can become exponentially larger still because the model is no longer producing one response. It is running a process. When a SaaS product redesigns itself around human-agent interaction, every task can become a chain of hidden micro-decisions, and each micro-decision consumes context, memory, tool calls, verification, and output tokens. Another point is people hear “more tokens” and think only of bigger data centers, when the deeper change is economic: software usage may stop being measured mainly by clicks, seats, or sessions. It will be measured by how much machine reasoning/tokens gets spent on behalf of each user. ---- From "Reuters" YouTube channel, (link in comment)

Rohan Paul

31,617 просмотров • 1 месяц назад

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

Avi Chawla

269,126 просмотров • 29 дней назад

We talked to Micky 🐸, the founder of Ribbit Capital and early backer of Coinbase, Robinhood, and Bridge, about what’s coming next. "I’m more excited for the next decade than the last." “Everything is a token. We call it the token factory revolution; tokenizing information, money, and power.” AI, payments, and financial services are converging fast, and tokens are becoming the raw material for decision-making, coordination, and access. Micky sees a new stack forming where every company is either building, supplying, or orchestrating tokens. And he bets on a generation of young founders who are native to this shift: “Their brains are just faster. The best ones are already building access tokens, expert tokens, asset tokens… stablecoins were only the first chapter.” From his appearance on the show in July.

We talked to Micky 🐸, the founder of Ribbit Capital and early backer of Coinbase, Robinhood, and Bridge, about what’s coming next. "I’m more excited for the next decade than the last." “Everything is a token. We call it the token factory revolution; tokenizing information, money, and power.” AI, payments, and financial services are converging fast, and tokens are becoming the raw material for decision-making, coordination, and access. Micky sees a new stack forming where every company is either building, supplying, or orchestrating tokens. And he bets on a generation of young founders who are native to this shift: “Their brains are just faster. The best ones are already building access tokens, expert tokens, asset tokens… stablecoins were only the first chapter.” From his appearance on the show in July.

TBPN

106,752 просмотров • 8 месяцев назад

OptimAI Search MCP brings live context to the agents people already use. Claude. Cursor. Copilot. Codex. One integration layer. Everywhere you build. Under the hood, OptimAI can process hundreds to thousands of real-time signals in parallel across the public web, social platforms, and onchain data - all powered by a decentralized network. That is the difference: not another search tool, but infrastructure for agent-native intelligence. Build with it. 💡 -- OptimAI: the #1 AI on BNB Chain DappBay.

OptimAI Search MCP brings live context to the agents people already use. Claude. Cursor. Copilot. Codex. One integration layer. Everywhere you build. Under the hood, OptimAI can process hundreds to thousands of real-time signals in parallel across the public web, social platforms, and onchain data - all powered by a decentralized network. That is the difference: not another search tool, but infrastructure for agent-native intelligence. Build with it. 💡 -- OptimAI: the #1 AI on BNB Chain DappBay.

OptimAI Network

54,529 просмотров • 1 месяц назад

OpenAI's chairman just responded to the Kimi K3 panic 👇 The timeline calls K3 a frontier killer. Bret Taylor's answer on CNBC: every CEO he talks to asks the same 2 questions. 1️⃣ Am I getting value from my token spend? 2️⃣ What's my moat in the age of AI? His counter on cheap Chinese models: frontier models finish the same task in far fewer tokens. Cheap per token means little if you burn 5x more of them The new battleground is tokens per task, and the price war just forced OpenAI to argue efficiency instead of capability

OpenAI's chairman just responded to the Kimi K3 panic 👇 The timeline calls K3 a frontier killer. Bret Taylor's answer on CNBC: every CEO he talks to asks the same 2 questions. 1️⃣ Am I getting value from my token spend? 2️⃣ What's my moat in the age of AI? His counter on cheap Chinese models: frontier models finish the same task in far fewer tokens. Cheap per token means little if you burn 5x more of them The new battleground is tokens per task, and the price war just forced OpenAI to argue efficiency instead of capability

Ruben

76,011 просмотров • 4 дней назад

New Fable 5 beats Opus 4.8 on real world physics simulations We gave both models the same three prompts and asked them to build self contained HTML5 sims with real physics and no libraries: 1. Chaotic double pendulum 2. Galton board 3. Water in a spinning drum (WCSPH) Generation cost Fable 5: $3.35 on 68.7k tokens, time 14m 47s Opus 4.8: $0.93 on 38.9k tokens, time 8m 10s Fable clearly did better on the water simulation, producing a much more solid and continuous body of water. Opus left larger gaps near the walls, scattered particles around the scene, and struggled to keep the fluid stable.

New Fable 5 beats Opus 4.8 on real world physics simulations We gave both models the same three prompts and asked them to build self contained HTML5 sims with real physics and no libraries: 1. Chaotic double pendulum 2. Galton board 3. Water in a spinning drum (WCSPH) Generation cost Fable 5: $3.35 on 68.7k tokens, time 14m 47s Opus 4.8: $0.93 on 38.9k tokens, time 8m 10s Fable clearly did better on the water simulation, producing a much more solid and continuous body of water. Opus left larger gaps near the walls, scattered particles around the scene, and struggled to keep the fluid stable.

atomic.chat

1,587,225 просмотров • 1 месяц назад

I cut Fable 5 token usage 2.5x with just one change! - Before: 5.5 M tokens · 7 errors · $8.94 - After: 2.3 M tokens · 0 errors · $4.17 The final build was the same for both, but the path the agent took wildly differed. In both runs, the agent started with the same thing, i.e., it understood the backend before building anything, like: - Permission policies - Available storage buckets - Auth providers configured - How edge functions are deployed The first run used Firebase, which was built for a human dev using a dashboard. While the dev can read the above state by clicking through tabs, an agent has no dashboard. So it gathered the same info through API calls. And there's no single Firebase call that returned this info. The agent required to query multiple times, and each query over-returned. For instance, when the agent asked how sign-in is configured, Firebase also returned the entire auth surface and every method it supported. This was far more context than what it needed. And it repeated across every part of the backend it inspected. Some states (like which auth providers are active) weren't queryable at all. I provided it myself. Otherwise, the agent would have guessed. Errors further compounded the token usage. When a dev sees "permission denied," they can look at the console and figure out whether it's a rule, a path, or an unauthenticated request. Firebase returned the same string to the agent as well, and it had none of that surrounding context to debug. So it guessed again, picked the most likely cause, and rewrote code, utilizing more tokens. This Firebase setup cost me 5.5M tokens and 7 manual interventions during errors on a full-stack RAG app. But I brought that down to 2.3M tokens and 0 manual interventions by using InsForge as the backend context engineering layer (open-source and self-hostable via Docker). It provides the same primitives as Supabase/Firebase, but structures the entire information layer for agents, instead of dashboards. In one CLI call that consumed ~500 tokens, the agent saw the full backend topology before writing a single line of code. This included auth, database, storage, edge functions, model gateway, micro VMs, and deployment. Also, instead of loading the entire product surface into context on every task, four narrowly scoped skills activated only when relevant to keep cognitive load minimal. And to ensure efficient retries if needed, every CLI operation returned structured JSON with meaningful exit codes, so the agent never guessed what to do next. Here's the InsForge GitHub Repo: (don't forget to star it ⭐) The video below depicts the final build, comparing Firebase and InsForge. To dive deeper, I recently published a full walkthrough building the same RAG app on both backends and inspected them end-to-end. Read it below.

I cut Fable 5 token usage 2.5x with just one change! - Before: 5.5 M tokens · 7 errors · $8.94 - After: 2.3 M tokens · 0 errors · $4.17 The final build was the same for both, but the path the agent took wildly differed. In both runs, the agent started with the same thing, i.e., it understood the backend before building anything, like: - Permission policies - Available storage buckets - Auth providers configured - How edge functions are deployed The first run used Firebase, which was built for a human dev using a dashboard. While the dev can read the above state by clicking through tabs, an agent has no dashboard. So it gathered the same info through API calls. And there's no single Firebase call that returned this info. The agent required to query multiple times, and each query over-returned. For instance, when the agent asked how sign-in is configured, Firebase also returned the entire auth surface and every method it supported. This was far more context than what it needed. And it repeated across every part of the backend it inspected. Some states (like which auth providers are active) weren't queryable at all. I provided it myself. Otherwise, the agent would have guessed. Errors further compounded the token usage. When a dev sees "permission denied," they can look at the console and figure out whether it's a rule, a path, or an unauthenticated request. Firebase returned the same string to the agent as well, and it had none of that surrounding context to debug. So it guessed again, picked the most likely cause, and rewrote code, utilizing more tokens. This Firebase setup cost me 5.5M tokens and 7 manual interventions during errors on a full-stack RAG app. But I brought that down to 2.3M tokens and 0 manual interventions by using InsForge as the backend context engineering layer (open-source and self-hostable via Docker). It provides the same primitives as Supabase/Firebase, but structures the entire information layer for agents, instead of dashboards. In one CLI call that consumed ~500 tokens, the agent saw the full backend topology before writing a single line of code. This included auth, database, storage, edge functions, model gateway, micro VMs, and deployment. Also, instead of loading the entire product surface into context on every task, four narrowly scoped skills activated only when relevant to keep cognitive load minimal. And to ensure efficient retries if needed, every CLI operation returned structured JSON with meaningful exit codes, so the agent never guessed what to do next. Here's the InsForge GitHub Repo: (don't forget to star it ⭐) The video below depicts the final build, comparing Firebase and InsForge. To dive deeper, I recently published a full walkthrough building the same RAG app on both backends and inspected them end-to-end. Read it below.

Avi Chawla

112,879 просмотров • 1 месяц назад

I designed a new test specifically for multimodal models: fill out a paper form. And it's much harder than it sounds. This isn't typing into an electronic field that captures your text. The form is just an image. The model has to place each form element: text, checkmarks — at the correct pixel position on the canvas itself. Results: 🟢 Kimi K2.6 → done in 3:45, 16.7k output tokens 🟡 Step 3.7 Flash → half the fields, 57k output tokens 🔴 Gemini 3.5 Flash → 489k output tokens, never finished. I had to kill it. Gemini burned ~29x more output tokens than Kimi on the exact same task, and Kimi's was the only form that actually looked filled out. The test, a mocked application form, contains some challenging parts, such as one-character-per-box fields. I provided every model the same set of tools: > get canvas size > drop probe markers to find coordinates > add text > add checkmarks > move elements > take a screenshot anytime to check their own work > ... etc So it's vision + spatial reasoning + tool use + long context, all at once. Small models (Qwen, Gemma) can't really complete this test, so I skipped them. What happened: > Kimi nailed name, DOB, ID, gender, marital status, nationality, email, phone, address, postal code — placement slightly loose, but content correct. 15 turns. Clean. > Step got maybe half right — fields dropped, "United States" landed in the email line, data floating outside boxes. Burned 1.24M input tokens doing it (81 turns of re-reading the canvas). > Gemini almost got there visually... then spiraled. By turn 40 it was issuing a delete_elements call wiping element IDs 365–425, basically erasing its own work. 31 minutes, 489k output tokens, still streaming. Terminated. The takeaway isn't "Gemini bad." This test is indeed difficult. But token efficiency is capability now. A model that needs 30x the tokens and still can't converge is going to be 30x the cost in production. Kimi K2.6 just quietly did the thing.

I designed a new test specifically for multimodal models: fill out a paper form. And it's much harder than it sounds. This isn't typing into an electronic field that captures your text. The form is just an image. The model has to place each form element: text, checkmarks — at the correct pixel position on the canvas itself. Results: 🟢 Kimi K2.6 → done in 3:45, 16.7k output tokens 🟡 Step 3.7 Flash → half the fields, 57k output tokens 🔴 Gemini 3.5 Flash → 489k output tokens, never finished. I had to kill it. Gemini burned ~29x more output tokens than Kimi on the exact same task, and Kimi's was the only form that actually looked filled out. The test, a mocked application form, contains some challenging parts, such as one-character-per-box fields. I provided every model the same set of tools: > get canvas size > drop probe markers to find coordinates > add text > add checkmarks > move elements > take a screenshot anytime to check their own work > ... etc So it's vision + spatial reasoning + tool use + long context, all at once. Small models (Qwen, Gemma) can't really complete this test, so I skipped them. What happened: > Kimi nailed name, DOB, ID, gender, marital status, nationality, email, phone, address, postal code — placement slightly loose, but content correct. 15 turns. Clean. > Step got maybe half right — fields dropped, "United States" landed in the email line, data floating outside boxes. Burned 1.24M input tokens doing it (81 turns of re-reading the canvas). > Gemini almost got there visually... then spiraled. By turn 40 it was issuing a delete_elements call wiping element IDs 365–425, basically erasing its own work. 31 minutes, 489k output tokens, still streaming. Terminated. The takeaway isn't "Gemini bad." This test is indeed difficult. But token efficiency is capability now. A model that needs 30x the tokens and still can't converge is going to be 30x the cost in production. Kimi K2.6 just quietly did the thing.

stevibe

25,304 просмотров • 1 месяц назад

Opus 5 crushed Fable 5 at 3D destruction physics for 2x cheaper! We gave four models the same task: build three self-contained HTML scenes with real physics Prompts: - A tornado that sucks in a whole field - A wrecking ball taking down an apartment block - An overloaded truck collapsing a truss bridge Outputs: - Opus 5: 55.9K tokens, $1.40 - Fable 5: 55.1K tokens, $2.82 - Kimi K3: 35.7K tokens, $0.55 - GPT 5.6: 20.1K tokens, $0.31 Opus got all three right unlike the other models. Houses fly up the funnel and out the top, the wall breaks where the ball hits and the rubble piles up, the bridge drops the truck into the river. Fable had almost nothing on the ground for the tornado to pick up, its building collapsed on its own before the ball even touched it and its bridge blew into sticks all at once. GPT is the cheapest here but its ball never reached the building at all and its bridge fell apart in a way nothing falls apart in real life. Kimi K3, the new Chinese frontier model, ended up in the same place as GPT

Opus 5 crushed Fable 5 at 3D destruction physics for 2x cheaper! We gave four models the same task: build three self-contained HTML scenes with real physics Prompts: - A tornado that sucks in a whole field - A wrecking ball taking down an apartment block - An overloaded truck collapsing a truss bridge Outputs: - Opus 5: 55.9K tokens, $1.40 - Fable 5: 55.1K tokens, $2.82 - Kimi K3: 35.7K tokens, $0.55 - GPT 5.6: 20.1K tokens, $0.31 Opus got all three right unlike the other models. Houses fly up the funnel and out the top, the wall breaks where the ball hits and the rubble piles up, the bridge drops the truck into the river. Fable had almost nothing on the ground for the tornado to pick up, its building collapsed on its own before the ball even touched it and its bridge blew into sticks all at once. GPT is the cheapest here but its ball never reached the building at all and its bridge fell apart in a way nothing falls apart in real life. Kimi K3, the new Chinese frontier model, ended up in the same place as GPT

atomic.chat

1,418,720 просмотров • 1 день назад

Liquid AI (Liquid AI) CEO Ramin Hasani (Ramin) says: "We're bringing the cost of tokens to zero." "The axis was maximizing intelligence at all costs." Foundation models need to optimize across 3 axes: 1.) Intelligence and capability 2.) Efficiency and cost 3.) Substrate: where the intelligence actually runs "Efficiency is not an afterthought. Energy is not abundant." "If you maximize intelligence at all costs, that axis alone is not going to get you to the place that you want to go." "Efficiency and cost of intelligence as a first-class citizen, and not an afterthought." "Where does this intelligence system go? AI is majorly getting hosted in data centers, but you could also bring intelligence on phones, on laptops, on airplanes, on cars." "Imagine if it's not tokens that are actually important. It's just the outcome that actually matters." "Not just thinking about foundation models as the token machines that are generating money and revenue for the foundation model companies that are useless tokens, and getting them into the place where they can actually unlock true value for enterprises."

Liquid AI (Liquid AI) CEO Ramin Hasani (Ramin) says: "We're bringing the cost of tokens to zero." "The axis was maximizing intelligence at all costs." Foundation models need to optimize across 3 axes: 1.) Intelligence and capability 2.) Efficiency and cost 3.) Substrate: where the intelligence actually runs "Efficiency is not an afterthought. Energy is not abundant." "If you maximize intelligence at all costs, that axis alone is not going to get you to the place that you want to go." "Efficiency and cost of intelligence as a first-class citizen, and not an afterthought." "Where does this intelligence system go? AI is majorly getting hosted in data centers, but you could also bring intelligence on phones, on laptops, on airplanes, on cars." "Imagine if it's not tokens that are actually important. It's just the outcome that actually matters." "Not just thinking about foundation models as the token machines that are generating money and revenue for the foundation model companies that are useless tokens, and getting them into the place where they can actually unlock true value for enterprises."

Molly O’Shea

12,816 просмотров • 17 дней назад

Three months ago, Codex was trash for knowledge work. Now it's my daily driver. I use it for writing, recruiting, deep engineering work, and everything in between. It even keeps me at inbox 0. I chatted with Every 📧's head of growth Austin Austin Tedesco on Every 📧's AI & I about what changed, and why he now spends 80% of his working time in the Codex desktop app too. We get into: - How Codex went from making Austin feel like an idiot to being the place he goes to get stuff done, including complex tasks like writing go-to-market plans using existing material from Slack, Notion, and meeting transcripts. - Why the Codex’s desktop app, which is faster and more reliable than Claude Desktop/Cowork, is the real differentiator. - How I source candidates with Codex by having it identify career arcs, not keywords—my go-to move is identifying organizations likely to teach the skills Every needs for a role, and then find candidates from that pool who have since gone on to work in AI. This is a must-watch for anyone who's wondering whether it’s finally time to give Codex a try. Watch below! Timestamps How Codex went from a tool for senior engineers to a daily driver for knowledge work: 00:00:57 How Claude Code proved that a great coding agent works for any knowledge work: 00:02:42 Austin's switch to Codex: 00:07:24 How Austin set up Codex with folders, keys, and reviewer agents: 00:13:48 Using Codex to brainstorm automations across Gmail, Slack, and Notion: 00:18:24 How Austin manages the human review step when Codex is drafting communications: 00:22:42 Using Codex to build specialized agents inspired by product executive Claire Vo: 00:28:54 Synthesizing meeting transcripts and Slack threads into a go-to-market plan: 00:31:09 Building a live KPI tracker in Notion that agents can read: 00:40:15 Using Codex for recruiting: 00:44:54

Three months ago, Codex was trash for knowledge work. Now it's my daily driver. I use it for writing, recruiting, deep engineering work, and everything in between. It even keeps me at inbox 0. I chatted with Every 📧's head of growth Austin Austin Tedesco on Every 📧's AI & I about what changed, and why he now spends 80% of his working time in the Codex desktop app too. We get into: - How Codex went from making Austin feel like an idiot to being the place he goes to get stuff done, including complex tasks like writing go-to-market plans using existing material from Slack, Notion, and meeting transcripts. - Why the Codex’s desktop app, which is faster and more reliable than Claude Desktop/Cowork, is the real differentiator. - How I source candidates with Codex by having it identify career arcs, not keywords—my go-to move is identifying organizations likely to teach the skills Every needs for a role, and then find candidates from that pool who have since gone on to work in AI. This is a must-watch for anyone who's wondering whether it’s finally time to give Codex a try. Watch below! Timestamps How Codex went from a tool for senior engineers to a daily driver for knowledge work: 00:00:57 How Claude Code proved that a great coding agent works for any knowledge work: 00:02:42 Austin's switch to Codex: 00:07:24 How Austin set up Codex with folders, keys, and reviewer agents: 00:13:48 Using Codex to brainstorm automations across Gmail, Slack, and Notion: 00:18:24 How Austin manages the human review step when Codex is drafting communications: 00:22:42 Using Codex to build specialized agents inspired by product executive Claire Vo: 00:28:54 Synthesizing meeting transcripts and Slack threads into a go-to-market plan: 00:31:09 Building a live KPI tracker in Notion that agents can read: 00:40:15 Using Codex for recruiting: 00:44:54

Dan Shipper 📧

55,221 просмотров • 2 месяцев назад