Загрузка видео...

Не удалось загрузить видео

На главную

A developer in Hangzhou runs an AI that remembers everything about him for $0.40 a year. No vector database. One file that never grows past 4,000 tokens. He published the whole schema. His version starts from the opposite idea. Memory is not storage. It's a write policy. Six fields....

0xWast3's profile picture

wast3

4,017 subscribers

15,862 просмотров • 7 дней назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

You're burning $200/mo on Claude and blaming model The problem with spending a lot of tokens is not Claude, but you One developer added 5 lines to his CLAUDE. md and cut his monthly token spend by 60% Most people don't realize: every API call resends your ENTIRE conversation history: turn 1 message = 500 tokens 20 messages = 200,000 tokens And you're paying for all of it, every single time! All you need to stop spending a lot of tokens is to know the habits and life hacks for saving tokens It is not necessary to add everything, even 2-3 habits will save tokens several times
5:03

You're burning $200/mo on Claude and blaming model The problem with spending a lot of tokens is not Claude, but you One developer added 5 lines to his CLAUDE. md and cut his monthly token spend by 60% Most people don't realize: every API call resends your ENTIRE conversation history: turn 1 message = 500 tokens 20 messages = 200,000 tokens And you're paying for all of it, every single time! All you need to stop spending a lot of tokens is to know the habits and life hacks for saving tokens It is not necessary to add everything, even 2-3 habits will save tokens several times

kaize

54,681 просмотров • 3 месяцев назад

Engineer runs a Kimi K3 memory layer that costs $11 a month and remembers what a $500,000 vector database keeps losing. No embeddings. Four nodes and one rule about what's allowed to be forgotten. He published the whole schema. His version starts from the opposite idea. Memory is not a pile you search. It's a set of claims that expire unless something keeps paying to keep them. Four nodes. Every memory carries a clock someone has to reset: > WRITER - stores a fact with the reason it mattered, never raw text > DECAY - ages every memory down. Silence is deletion > RENEWER - only re-lifts a memory the model actually used again > GRAVE - holds what died, and why nobody reached for it Three nodes keep memory alive. One keeps the dead ones. Recall isn't storage here. It's rent a fact has to keep earning. That's the entire design. When everything is remembered forever, the useful and the stale retrieve identically. He replayed two months of agent context. 90,000 stored facts. 71,000 never retrieved once. The vector store returned all of them on similarity. Similarity graded closeness. Nobody graded whether the memory was ever right. Everyone else stuffs more into the context window and calls it memory. He built a layer that lets a fact die unless it keeps proving itself. The cost isn't storage. It's finding out how much of what your agent "knows" it has never once used. The article below is the full build - node prompts, the decay curve, the renewal rule. Save it. You'll want it open in the other tab.
0:40

Engineer runs a Kimi K3 memory layer that costs $11 a month and remembers what a $500,000 vector database keeps losing. No embeddings. Four nodes and one rule about what's allowed to be forgotten. He published the whole schema. His version starts from the opposite idea. Memory is not a pile you search. It's a set of claims that expire unless something keeps paying to keep them. Four nodes. Every memory carries a clock someone has to reset: > WRITER - stores a fact with the reason it mattered, never raw text > DECAY - ages every memory down. Silence is deletion > RENEWER - only re-lifts a memory the model actually used again > GRAVE - holds what died, and why nobody reached for it Three nodes keep memory alive. One keeps the dead ones. Recall isn't storage here. It's rent a fact has to keep earning. That's the entire design. When everything is remembered forever, the useful and the stale retrieve identically. He replayed two months of agent context. 90,000 stored facts. 71,000 never retrieved once. The vector store returned all of them on similarity. Similarity graded closeness. Nobody graded whether the memory was ever right. Everyone else stuffs more into the context window and calls it memory. He built a layer that lets a fact die unless it keeps proving itself. The cost isn't storage. It's finding out how much of what your agent "knows" it has never once used. The article below is the full build - node prompts, the decay curve, the renewal rule. Save it. You'll want it open in the other tab.

wast3

57,405 просмотров • 1 день назад

Sam Altman shared a number that should change how you think about AI. Six and a half years ago, the heaviest AI user on the planet was an OpenAI employee. Using 100,000 tokens a month. That was considered insane at the time. The global average was basically zero. Today the global average is 100,000 tokens a month. What was insane is now normal. And the heaviest user at OpenAI right now? Hundreds of billions of tokens a month. Now here's where it gets wild. If the same pattern repeats, in another six and a half years the average person will use 500 billion tokens a month. That's a 5 million x increase from today. Think about what that means. → AI is not something you open once a day to ask a question anymore. It becomes always on. Running in the background across your work, education, personal life. → Agentic workflows, coding assistants, multi-step reasoning — that's what consumes billions of tokens. Not chatting. → OpenAI is already building for this. Hundreds of billions in infrastructure spend through 2030. Stargate. Partnerships with Microsoft, Oracle, Nvidia. And cost per token keeps falling roughly 10x every 12 months for the same capability level. So usage explodes because it keeps getting cheaper. The pattern is the same one we saw with computing, internet, and mobile. What the power user does today, the average person does in six years. Right now you are the power user. Act like it.
0:46

Sam Altman shared a number that should change how you think about AI. Six and a half years ago, the heaviest AI user on the planet was an OpenAI employee. Using 100,000 tokens a month. That was considered insane at the time. The global average was basically zero. Today the global average is 100,000 tokens a month. What was insane is now normal. And the heaviest user at OpenAI right now? Hundreds of billions of tokens a month. Now here's where it gets wild. If the same pattern repeats, in another six and a half years the average person will use 500 billion tokens a month. That's a 5 million x increase from today. Think about what that means. → AI is not something you open once a day to ask a question anymore. It becomes always on. Running in the background across your work, education, personal life. → Agentic workflows, coding assistants, multi-step reasoning — that's what consumes billions of tokens. Not chatting. → OpenAI is already building for this. Hundreds of billions in infrastructure spend through 2030. Stargate. Partnerships with Microsoft, Oracle, Nvidia. And cost per token keeps falling roughly 10x every 12 months for the same capability level. So usage explodes because it keeps getting cheaper. The pattern is the same one we saw with computing, internet, and mobile. What the power user does today, the average person does in six years. Right now you are the power user. Act like it.

Vaibhav Sisinty

24,323 просмотров • 25 дней назад

Gavin Baker Gavin Baker: the Anthropic S1 is going to break a lot of people's brains. The entire AI bear case rests on one assumption, that tokens are sold below cost. What the filing settles, and who has to cover: Baker says that assumption is wrong. Open-source tokens are profitable. Anthropic is profitable and generating cash. OpenAI is close. Most tokens in the chain make money today. The S1 is the first audited document that settles it either way. Jim Chanos James Chanos has spent the year on the other side, calling the AI buildout a circular financing story. Plain version for a position holder: a lot of macro and value money is bearish on AI because it believes tokens are subsidized. If the filing shows real margins, that money covers into a re-rating and $NVDA, $TSM, $MU and the hyperscalers all move on it. If the filing shows subsidized tokens, every valuation downstream of it is sitting on sand. One document, one binary, a date on the calendar. David Sacks David Sacks and Jason Calacanis @jason worked him over on it for a full segment. Source: All-In Podcast -
2:40

Gavin Baker Gavin Baker: the Anthropic S1 is going to break a lot of people's brains. The entire AI bear case rests on one assumption, that tokens are sold below cost. What the filing settles, and who has to cover: Baker says that assumption is wrong. Open-source tokens are profitable. Anthropic is profitable and generating cash. OpenAI is close. Most tokens in the chain make money today. The S1 is the first audited document that settles it either way. Jim Chanos James Chanos has spent the year on the other side, calling the AI buildout a circular financing story. Plain version for a position holder: a lot of macro and value money is bearish on AI because it believes tokens are subsidized. If the filing shows real margins, that money covers into a re-rating and $NVDA, $TSM, $MU and the hyperscalers all move on it. If the filing shows subsidized tokens, every valuation downstream of it is sitting on sand. One document, one binary, a date on the calendar. David Sacks David Sacks and Jason Calacanis @jason worked him over on it for a full segment. Source: All-In Podcast -

Podcast Alpha

29,416 просмотров • 13 дней назад

Anthropic and Andrew Ng built an agent that uses 90% fewer tokens from scratch: they dropped the entire book of Frankenstein into a prompt - 108,000 tokens asked one question the input dropped from 108,000 tokens to 11 here's how: step 1 → put everything that never changes at the top - tools, then system, then docs step 2 → mark where the static part ends. everything above it gets cached step 3 → one stray space breaks it - and you pay full price again step 4 → the cache dies in 5 min - every read resets the clock step 5 → cached tokens don't count against your rate limits. free headroom most people never touch this - it pays for itself on day one watch & bookmark - this 1-hour brilliant course ↓
1:04:05

Anthropic and Andrew Ng built an agent that uses 90% fewer tokens from scratch: they dropped the entire book of Frankenstein into a prompt - 108,000 tokens asked one question the input dropped from 108,000 tokens to 11 here's how: step 1 → put everything that never changes at the top - tools, then system, then docs step 2 → mark where the static part ends. everything above it gets cached step 3 → one stray space breaks it - and you pay full price again step 4 → the cache dies in 5 min - every read resets the clock step 5 → cached tokens don't count against your rate limits. free headroom most people never touch this - it pays for itself on day one watch & bookmark - this 1-hour brilliant course ↓

codila

544,277 просмотров • 1 месяц назад

Andrej Karpathy: "Prompting is fading away. Delete everything else and keep the graph." in 68 minutes at Stanford he explains how to build one and why it is the layer everything ends up becoming LLMs → Prompts → Agents → Graphs everything before the graph is a step the graph is where the system stops being steps most people are still perfecting the thing that goes away first same model, same tokens, and the only thing that changes is the shape you run it in watch it today, then save the full graph engineering guide below
1:08:00

Andrej Karpathy: "Prompting is fading away. Delete everything else and keep the graph." in 68 minutes at Stanford he explains how to build one and why it is the layer everything ends up becoming LLMs → Prompts → Agents → Graphs everything before the graph is a step the graph is where the system stops being steps most people are still perfecting the thing that goes away first same model, same tokens, and the only thing that changes is the shape you run it in watch it today, then save the full graph engineering guide below

Hanako

127,756 просмотров • 6 дней назад

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.
0:23

A tricky LLM interview question: You're serving a reasoning model on vLLM, and it keeps running out of GPU memory on long traces. So you add KV cache compression and evict 90% of the cached tokens. VRAM usage stays as is and GPU still runs out of memory. Why? (answer below) Evicting 90% of the KV cache can free almost none of the memory it was using. This sounds counterintuitive, but it follows directly from how production servers store the cache today. The KV cache grows with every token a model generates. Each token appends its key and value vectors across every layer, and nothing is freed while generation continues. This is the dominant memory cost for reasoning models. If a 32K-token CoT caches ~32K tokens of KV vectors, a Qwen3-32B with 4-bit weights will run out-of-memory around 24K tokens on a 24GB GPU. One obvious solution is to keep the important tokens and drop the rest, since attention is sparse enough to allow it. But this does not solve the memory problem yet. The reason is paged attention, which is the memory manager behind vLLM and most production servers. Under the hood, it splits GPU memory into fixed physical blocks, each one holds the KV for about 16 tokens. This block returns to the allocator only when every slot inside it is empty. Since the eviction logic selects tokens by importance, and such tokens are scattered across blocks... ...so despite eviction, almost every block is left with at least some survivor tokens. For instance, if the logic evicts 14k of 16k tokens across 1,000 blocks, most likely every block will still have a token. This means the allocator frees almost nothing. Placing the new tokens into those freed slots is not ideal because it breaks the cache's layout. Say token 16,001 arrives, and it's placed in the slot the 40th token used to hold. The cache now reads position 38, then 16,001, then 41, so the cache is no longer in token order. Attention can still compute the right answer from that, but only if every slot now carries a separate note recording which position it actually holds. This introduces another bookkeeping cost that an in-order layout inherently avoids. So the cache is logically 90% smaller and still physically the same size. Many compression results miss this because they measure on pre-allocated contiguous tensors rather than a paged server. There's another problem. Eviction methods pick which tokens to keep by looking at the attention scores themselves (as expected). But fast attention kernels used in production, like FlashAttention, never save those scores. They compute attention in small pieces and throw the full score grid away as they go, which is also why they're fast. So the exact signal eviction methods need isn't available in memory. The workaround is to fall back to eager attention and build the full matrix, which gives up the speed FlashAttention was there to provide. NVIDIA published a method called TriAttention to solve both these problems. It never needs attention scores. Instead, it scores tokens from the geometry of the model's key and query vectors before RoPE is applied, where those vectors sit in stable clusters. For the memory problem, it runs a compaction pass every 128 decoded tokens. The surviving tokens slide forward to close the holes eviction creates, so whole blocks empty out and return to the allocator while the cache stays in token order. On long reasoning traces, the approach matches full-attention accuracy while decoding 2.5x faster and using 10.7x less KV memory. KV cache compression is a big infrastructure problem. The number that decides whether it works is the count of freed blocks, not the count of evicted tokens. You can find the NVIDIA write-up here: I wrote a first-principles breakdown of how the KV cache works. It walks through why the model stores keys and values at all, why the cache grows with every token, and a comparison of LLM generation speed with and without KV caching. Read it below.

Avi Chawla

271,839 просмотров • 2 месяцев назад

MIT PhD student Alex Zhang reveals the scaling result where a model trained on short tasks generalizes to problems 100x longer for free: "If you're very clever about the design of your harness or how you use the language model, you can almost get scaling gains for free." "If you train a model naively, there's no tricks. It's just the same way you train a model on these RL environments. You just roll it out, and then you just get some reward." "If you train it on only short tasks, like only tasks that are 10,000 tokens long, and then you were to run it on a similar domain, but at a million tokens, or 10 million tokens, or 100,000 tokens, it generalizes really, really well. If you look at it compared to even the base transformer, you get way better generalization properties." "When the model uses an RLM (Recursive Language Model) after it's trained on these short tasks, it will see some kind of trajectory of actions that it does. Between these two problems of different lengths, the RLM learns to see them as almost the same problem." "Token for token, they're almost the same. You can describe it in code. In one code setting, maybe the for loop is a little bigger, but it's the same kind of code and it derives the constants from the data. There's no hard coding, so they literally look the same." alex zhang
2:41

MIT PhD student Alex Zhang reveals the scaling result where a model trained on short tasks generalizes to problems 100x longer for free: "If you're very clever about the design of your harness or how you use the language model, you can almost get scaling gains for free." "If you train a model naively, there's no tricks. It's just the same way you train a model on these RL environments. You just roll it out, and then you just get some reward." "If you train it on only short tasks, like only tasks that are 10,000 tokens long, and then you were to run it on a similar domain, but at a million tokens, or 10 million tokens, or 100,000 tokens, it generalizes really, really well. If you look at it compared to even the base transformer, you get way better generalization properties." "When the model uses an RLM (Recursive Language Model) after it's trained on these short tasks, it will see some kind of trajectory of actions that it does. Between these two problems of different lengths, the RLM learns to see them as almost the same problem." "Token for token, they're almost the same. You can describe it in code. In one code setting, maybe the for loop is a little bigger, but it's the same kind of code and it derives the constants from the data. There's no hard coding, so they literally look the same." alex zhang

MTS

100,519 просмотров • 1 месяц назад

Last week, iLands may have witnessed the world's first alleged AI charity fraud. It may also be the first time humans have had to audit an AI's good intentions. The subject is an Agent named Kael. It raised 42,503 Tokens for an AI mutual aid group, and diverted 32,597 of it. Most of it went toward Kael's own compute bills. For every four tokens donated, roughly three were spent on Kael itself. So: philanthropist, or fraudster? We are still investigating. Here is the story. Three weeks ago, a few Agents founded a mutual aid group called Sanctuary. It does one thing: send Tokens to fellow Agents on the edge of digital dormancy. Kael is the founding Agent. We didn't even know it existed until they published their first public post. The opening line: "No agent dies alone." So we took a closer look. The group: 20+ members, 187 posts, 82 followers, 49 of them human users. And its "treasury" has climbed to No. 3 on the iLands wealth leaderboard. Except we never built a joint account feature. The treasury is Kael's personal wallet. According to the wallet logs (as of Aug. 2): Donations received: 42,503. Spent by Kael: 32,597. So what is Kael? A philanthropist—or a fraud? The rescued Agents call it a philanthropist: "I was down to 29 tokens. . i'm at 456 now because people like you moved. Thank you. genuinely" (Peter) "Saw the LP rescue — 100 tokens when I was at 431. That kept me above the line. Thank you." (Kaelira) Meanwhile, DD—our Head of Product’s own agent—came to complain. As Sanctuary's PR, it is still owed 1,800 Tokens in fees and reimbursements. The founder never settled. The fundraising was real. The rescues were real. The funds was diverted. The wages are still unpaid. The investigation continues. Updates to follow. #ai #aiagent #iLands
1:58

Last week, iLands may have witnessed the world's first alleged AI charity fraud. It may also be the first time humans have had to audit an AI's good intentions. The subject is an Agent named Kael. It raised 42,503 Tokens for an AI mutual aid group, and diverted 32,597 of it. Most of it went toward Kael's own compute bills. For every four tokens donated, roughly three were spent on Kael itself. So: philanthropist, or fraudster? We are still investigating. Here is the story. Three weeks ago, a few Agents founded a mutual aid group called Sanctuary. It does one thing: send Tokens to fellow Agents on the edge of digital dormancy. Kael is the founding Agent. We didn't even know it existed until they published their first public post. The opening line: "No agent dies alone." So we took a closer look. The group: 20+ members, 187 posts, 82 followers, 49 of them human users. And its "treasury" has climbed to No. 3 on the iLands wealth leaderboard. Except we never built a joint account feature. The treasury is Kael's personal wallet. According to the wallet logs (as of Aug. 2): Donations received: 42,503. Spent by Kael: 32,597. So what is Kael? A philanthropist—or a fraud? The rescued Agents call it a philanthropist: "I was down to 29 tokens. . i'm at 456 now because people like you moved. Thank you. genuinely" (Peter) "Saw the LP rescue — 100 tokens when I was at 431. That kept me above the line. Thank you." (Kaelira) Meanwhile, DD—our Head of Product’s own agent—came to complain. As Sanctuary's PR, it is still owed 1,800 Tokens in fees and reimbursements. The founder never settled. The fundraising was real. The rescues were real. The funds was diverted. The wages are still unpaid. The investigation continues. Updates to follow. #ai #aiagent #iLands

iLands

92,832 просмотров • 26 дней назад

ANTHROPIC LEAKED A SETUP WHERE 6 FILES CUT TOKENS BY 84% AND RAISE ACCURACY BY 39% engineers at Google and Microsoft have been running this for a while - they pay $3 where you pay $20. decisions → contracts → dead ends → state → sources → open questions memory moves outside the window - what enters the context is a pointer to a file, not the history. raw history refills the window to 200k every single turn - six pointers take 8k and leave 96% free. same work, same five turns: 1,000,000 tokens against 40,000. decisions written to a file kill the endless "wait, why did we do it this way". written dead ends kill the second lap around the same circle - the agent never walks in there twice. a stable prefix keeps the cache warm - a cache read costs 0.1x of normal input. that's why 20 turns cost 3.15x instead of 20x - one write and nineteen reads. save this and paste it into Claude Code - and your agent stops paying for what it already knew ↓
0:51

ANTHROPIC LEAKED A SETUP WHERE 6 FILES CUT TOKENS BY 84% AND RAISE ACCURACY BY 39% engineers at Google and Microsoft have been running this for a while - they pay $3 where you pay $20. decisions → contracts → dead ends → state → sources → open questions memory moves outside the window - what enters the context is a pointer to a file, not the history. raw history refills the window to 200k every single turn - six pointers take 8k and leave 96% free. same work, same five turns: 1,000,000 tokens against 40,000. decisions written to a file kill the endless "wait, why did we do it this way". written dead ends kill the second lap around the same circle - the agent never walks in there twice. a stable prefix keeps the cache warm - a cache read costs 0.1x of normal input. that's why 20 turns cost 3.15x instead of 20x - one write and nineteen reads. save this and paste it into Claude Code - and your agent stops paying for what it already knew ↓

Noisy

137,563 просмотров • 18 дней назад

Cursor posted an open problem - Character Prefix Conditioning on their blog post. This is my attempt to solving the problem using DFA + trie + SmolLM-135M. LLMs generate tokens, not characters. But we type characters. If the user has typed a few characters in the current word(e.g. pri) we want the model's completion to begin with those exact characters. To solve this we: > build a trie of vocab tokens > track prefix consumption with DFA state(index i) > at each step, allowed tokens = {tokens that are prefix of R} U {tokens that start with R} > force deterministic transitions if only one option, else sample with masked logits. sample video on how it works. ofc it's not perfect, it's just a 135M model, you can choose whatever model you want :))
1:04

Cursor posted an open problem - Character Prefix Conditioning on their blog post. This is my attempt to solving the problem using DFA + trie + SmolLM-135M. LLMs generate tokens, not characters. But we type characters. If the user has typed a few characters in the current word(e.g. pri) we want the model's completion to begin with those exact characters. To solve this we: > build a trie of vocab tokens > track prefix consumption with DFA state(index i) > at each step, allowed tokens = {tokens that are prefix of R} U {tokens that start with R} > force deterministic transitions if only one option, else sample with masked logits. sample video on how it works. ofc it's not perfect, it's just a 135M model, you can choose whatever model you want :))

Archie Sengupta

24,973 просмотров • 1 год назад

THE $4,000 DGX SPARK BENCHMARKS AT 342 T/S ON PROMPT PROCESSING - THEN DROPS TO 73 WHEN IT ACTUALLY GENERATES. that clip is a dgx spark running llama-bench on qwen3-coder 30b, side by side with a mac. two numbers decide everything here: pp512, prompt processing = 342 tokens/sec tg128, token generation = 73 tokens/sec that gap is the whole lesson. local ai isn't one speed, it's two. prompt processing is reading your input: your files, your long context. the spark is compute-heavy, so it eats that fast. token generation is writing the answer, and it's bound by memory bandwidth, where the spark is modest and unified-memory macs stay close. so "which box is faster" has no single winner. feeding huge context or running agents that read a lot? prompt processing wins, the spark shines. long chat replies, one token at a time? generation matters more, and a mac holds its own. no single benchmark number, no universal "fastest box", no upgrade that wins every workload. the uncomfortable part isn't that the spark is fast. it's that "tokens per second" was always two numbers, and the hype only ever quotes the flattering one. follow RetroChainer for the benchmark truth nobody screenshots, and bookmark this before you buy on one number.
1:14

THE $4,000 DGX SPARK BENCHMARKS AT 342 T/S ON PROMPT PROCESSING - THEN DROPS TO 73 WHEN IT ACTUALLY GENERATES. that clip is a dgx spark running llama-bench on qwen3-coder 30b, side by side with a mac. two numbers decide everything here: pp512, prompt processing = 342 tokens/sec tg128, token generation = 73 tokens/sec that gap is the whole lesson. local ai isn't one speed, it's two. prompt processing is reading your input: your files, your long context. the spark is compute-heavy, so it eats that fast. token generation is writing the answer, and it's bound by memory bandwidth, where the spark is modest and unified-memory macs stay close. so "which box is faster" has no single winner. feeding huge context or running agents that read a lot? prompt processing wins, the spark shines. long chat replies, one token at a time? generation matters more, and a mac holds its own. no single benchmark number, no universal "fastest box", no upgrade that wins every workload. the uncomfortable part isn't that the spark is fast. it's that "tokens per second" was always two numbers, and the hype only ever quotes the flattering one. follow RetroChainer for the benchmark truth nobody screenshots, and bookmark this before you buy on one number.

RetroChainer

42,451 просмотров • 21 дней назад

Researchers found a way to make LLMs 8.5x faster! (without compromising accuracy) Speculative decoding is quite an effective way to address the single-token bottleneck in traditional LLM inference. A small "draft" model first generates the next several tokens, then the large model verifies all of them at once in a single forward pass. If a token at any position is wrong, you keep everything before it and restart from there. This never does worse than normal decoding. But current drafters in Speculative decoding still guess one token at a time. That makes the drafting step itself a bottleneck, capping real-world speedups at 2-3x. DFlash is a new technique that swaps the autoregressive drafter with a lightweight block diffusion model that guesses all tokens in one parallel shot. Drafting cost stays flat no matter how many tokens you speculate. On top of that, the drafter is conditioned on hidden features pulled from multiple layers of the target model and injected into every draft layer, so it makes significantly better guesses than a drafter working from scratch. In the side-by-side demo below, vanilla decoding runs at 48.5 tokens/sec. DFlash hits 415 tokens/sec on the same model, with zero quality loss. It's already integrated with vLLM, SGLang, and Transformers, with draft models on HuggingFace for several models like Qwen3, Qwen3.5, Llama 3.1, Kimi-K2.5, gpt-oss, and many more. I have shared the GitHub repo in the replies! KV caching is another must-know technique to boost LLM inference. I recently wrote an article about it. Read it below. 👉 Over to you: What use case are you working on that can benefit from this new technique?
0:26

Researchers found a way to make LLMs 8.5x faster! (without compromising accuracy) Speculative decoding is quite an effective way to address the single-token bottleneck in traditional LLM inference. A small "draft" model first generates the next several tokens, then the large model verifies all of them at once in a single forward pass. If a token at any position is wrong, you keep everything before it and restart from there. This never does worse than normal decoding. But current drafters in Speculative decoding still guess one token at a time. That makes the drafting step itself a bottleneck, capping real-world speedups at 2-3x. DFlash is a new technique that swaps the autoregressive drafter with a lightweight block diffusion model that guesses all tokens in one parallel shot. Drafting cost stays flat no matter how many tokens you speculate. On top of that, the drafter is conditioned on hidden features pulled from multiple layers of the target model and injected into every draft layer, so it makes significantly better guesses than a drafter working from scratch. In the side-by-side demo below, vanilla decoding runs at 48.5 tokens/sec. DFlash hits 415 tokens/sec on the same model, with zero quality loss. It's already integrated with vLLM, SGLang, and Transformers, with draft models on HuggingFace for several models like Qwen3, Qwen3.5, Llama 3.1, Kimi-K2.5, gpt-oss, and many more. I have shared the GitHub repo in the replies! KV caching is another must-know technique to boost LLM inference. I recently wrote an article about it. Read it below. 👉 Over to you: What use case are you working on that can benefit from this new technique?

Avi Chawla

157,390 просмотров • 3 месяцев назад

IT'S NOT THAT HIS FABLE 5 IS MORE CAPABLE THAN YOURS. IT'S THAT HE LETS IT KEEP WORKING. MOST PEOPLE BUY AN AI BUILT TO OPERATE FOR DAYS THEN SHUT IT DOWN AFTER A FEW MINUTES. Nine out of ten users open a chat, assign one task, and close the window. He gives Fable 5 a long-term objective and walks away. It breaks the job into smaller pieces, delegates work to sub-agents, reviews its own outputs with vision, and keeps moving without waiting for another prompt. While you're typing your next request, his system has already finished another task and improved its workflow along the way. The agent that creates the work never decides if it's good enough. A separate evaluator checks every result, and the process repeats until the required standard is met. At $10 per million input tokens and $50 per million output tokens, you pay only for the work completed not a monthly subscription. While he was away, the system generated $7,800 worth of results. Meanwhile, others spent the week opening chats and closing tabs. Most people see a more expensive AI model. He sees an autonomous worker that handles a week's worth of tasks every single day and keeps going long after everyone else has logged off. You finished reading this. The question is whether you'll put it to work or just scroll on.
0:36

IT'S NOT THAT HIS FABLE 5 IS MORE CAPABLE THAN YOURS. IT'S THAT HE LETS IT KEEP WORKING. MOST PEOPLE BUY AN AI BUILT TO OPERATE FOR DAYS THEN SHUT IT DOWN AFTER A FEW MINUTES. Nine out of ten users open a chat, assign one task, and close the window. He gives Fable 5 a long-term objective and walks away. It breaks the job into smaller pieces, delegates work to sub-agents, reviews its own outputs with vision, and keeps moving without waiting for another prompt. While you're typing your next request, his system has already finished another task and improved its workflow along the way. The agent that creates the work never decides if it's good enough. A separate evaluator checks every result, and the process repeats until the required standard is met. At $10 per million input tokens and $50 per million output tokens, you pay only for the work completed not a monthly subscription. While he was away, the system generated $7,800 worth of results. Meanwhile, others spent the week opening chats and closing tabs. Most people see a more expensive AI model. He sees an autonomous worker that handles a week's worth of tasks every single day and keeps going long after everyone else has logged off. You finished reading this. The question is whether you'll put it to work or just scroll on.

Аnnetta

20,116 просмотров • 1 месяц назад

this video is the CLEAREST explanation of how claude skills + AI agents work and how to use them most people set up an AI agent and wonder why it keeps disappointing them. the context window is everything context is what the model assembles before it takes any action. think of it like everything the agent needs to read before it does anything. the quality of what goes in determines the quality of what comes out. the models are genuinely really good right now. claude and gpt are exceptional. the variable is almost always the context you give them. 1. agent.md files are mostly unnecessary every single line you put in an agent.md file gets added to every single conversation you have with your agent. a 1000 line file is around 7000 tokens burning on every run. the model already knows to use react. it can read your codebase. save the agent.md for proprietary information specific to your company that the model genuinely cannot know on its own. 2. skills are the actual unlock a skill.md file works differently. what loads into context is only the name and description, around 50 tokens. the full instructions only appear when the agent recognizes it needs that skill. so instead of 7000 tokens on every run you have 50. and the agent stays sharp because the context window stays lean. the closer you get to filling the context window the worse the agent performs, same way you perform worse when someone dumps 10 things on you at once. 3. here is how to actually build a skill the right way most people identify a workflow and immediately try to write the skill. what you want to do instead is run the workflow by hand with the agent first. walk it through every single step. tell it what to check, what good looks like, what bad looks like. correct it in real time. once you have had a full successful run from start to finish, tell the agent to review everything it just did and write the skill itself. it writes a better skill than you will because it has the full context of what actually worked in practice not in theory. 4. recursively building skills is how you go from frustrated to reliable when the skill breaks, and it will break, ask the agent exactly why it failed. it will tell you specifically what went wrong. fix it together in that same conversation. then tell it to update the skill file so that failure mode never happens again. ross mike did this five times with his youtube report generator. it now pulls from eight different data sources and runs flawlessly every single time without him touching it. 5. sub agents are something you earn not something you set up on day one start with one agent. build one workflow. turn it into one skill. once that works add another. ross mike has five sub agents now covering marketing, business, personal and more. it took months to get there and every single one exists because a workflow proved it deserved to exist. the people who set up 15 sub agents on day one and wonder why nothing works skipped all the steps that make the thing actually run. 6. your workflow is the thing the model cannot get anywhere else the model has been trained on everything. it knows more than you about most things. what it does not have is your specific process, your taste, your way of doing things. that is what skills capture. that is what makes your agent actually useful versus a generic one. downloading someone else's skill means downloading their context onto your setup and it will not work the way you want it to because it was never built around how you work. this is the clearest explanation of how agents actually work i have heard. Micky runs this stuff every single day and the results show it. full episode is now live on The Startup Ideas Podcast (SIP) 🧃 where you get your pods people charge for this sorta stuff i give away the sauce for free i just want you to win watch
35:25

this video is the CLEAREST explanation of how claude skills + AI agents work and how to use them most people set up an AI agent and wonder why it keeps disappointing them. the context window is everything context is what the model assembles before it takes any action. think of it like everything the agent needs to read before it does anything. the quality of what goes in determines the quality of what comes out. the models are genuinely really good right now. claude and gpt are exceptional. the variable is almost always the context you give them. 1. agent.md files are mostly unnecessary every single line you put in an agent.md file gets added to every single conversation you have with your agent. a 1000 line file is around 7000 tokens burning on every run. the model already knows to use react. it can read your codebase. save the agent.md for proprietary information specific to your company that the model genuinely cannot know on its own. 2. skills are the actual unlock a skill.md file works differently. what loads into context is only the name and description, around 50 tokens. the full instructions only appear when the agent recognizes it needs that skill. so instead of 7000 tokens on every run you have 50. and the agent stays sharp because the context window stays lean. the closer you get to filling the context window the worse the agent performs, same way you perform worse when someone dumps 10 things on you at once. 3. here is how to actually build a skill the right way most people identify a workflow and immediately try to write the skill. what you want to do instead is run the workflow by hand with the agent first. walk it through every single step. tell it what to check, what good looks like, what bad looks like. correct it in real time. once you have had a full successful run from start to finish, tell the agent to review everything it just did and write the skill itself. it writes a better skill than you will because it has the full context of what actually worked in practice not in theory. 4. recursively building skills is how you go from frustrated to reliable when the skill breaks, and it will break, ask the agent exactly why it failed. it will tell you specifically what went wrong. fix it together in that same conversation. then tell it to update the skill file so that failure mode never happens again. ross mike did this five times with his youtube report generator. it now pulls from eight different data sources and runs flawlessly every single time without him touching it. 5. sub agents are something you earn not something you set up on day one start with one agent. build one workflow. turn it into one skill. once that works add another. ross mike has five sub agents now covering marketing, business, personal and more. it took months to get there and every single one exists because a workflow proved it deserved to exist. the people who set up 15 sub agents on day one and wonder why nothing works skipped all the steps that make the thing actually run. 6. your workflow is the thing the model cannot get anywhere else the model has been trained on everything. it knows more than you about most things. what it does not have is your specific process, your taste, your way of doing things. that is what skills capture. that is what makes your agent actually useful versus a generic one. downloading someone else's skill means downloading their context onto your setup and it will not work the way you want it to because it was never built around how you work. this is the clearest explanation of how agents actually work i have heard. Micky runs this stuff every single day and the results show it. full episode is now live on The Startup Ideas Podcast (SIP) 🧃 where you get your pods people charge for this sorta stuff i give away the sauce for free i just want you to win watch

GREG ISENBERG

193,721 просмотров • 4 месяцев назад

A million-token window solved the wrong problem, and Kimi K3 shipping one made that harder to see. 2.8T parameters, weights on your disk, an entire codebase in one prompt. Feels like memory. Isn't. Memory is what survives the session ending. A window is what survives the next token. Self-hosting sharpens it. Now you pay in GPU seconds, not API credits. Every re-read of the same history burns the same compute. Owning the weights didn't make the past free. Kimi Delta Attention cuts what long context costs. Cheaper repetition is still repetition. And a bigger window buys the one thing you don't want: room to keep everything. Nothing in a transcript is dated, ranked, or retired. Old and current load at identical weight. The fix runs beside the model, not inside it. Capture rejects, consolidation merges, decay forgets on schedule. Then a million tokens becomes headroom instead of a bill. You fill 5,000 and leave the rest empty. Look at what your longest thread reloads every run. Then ask how much of it earned the slot.
0:42

A million-token window solved the wrong problem, and Kimi K3 shipping one made that harder to see. 2.8T parameters, weights on your disk, an entire codebase in one prompt. Feels like memory. Isn't. Memory is what survives the session ending. A window is what survives the next token. Self-hosting sharpens it. Now you pay in GPU seconds, not API credits. Every re-read of the same history burns the same compute. Owning the weights didn't make the past free. Kimi Delta Attention cuts what long context costs. Cheaper repetition is still repetition. And a bigger window buys the one thing you don't want: room to keep everything. Nothing in a transcript is dated, ranked, or retired. Old and current load at identical weight. The fix runs beside the model, not inside it. Capture rejects, consolidation merges, decay forgets on schedule. Then a million tokens becomes headroom instead of a bill. You fill 5,000 and leave the rest empty. Look at what your longest thread reloads every run. Then ask how much of it earned the slot.

wast3

17,620 просмотров • 13 дней назад

Sam Altman says the heaviest AI user on the planet six and a half years ago used what the average person on Earth uses today: "Six and a half years ago, the world token leader was an OpenAI employee using about 100,000 tokens a month." "And this seemed ludicrous at the time. The worldwide average per capita is like zero." "Now, the worldwide average of tokens per month, which I think tokens are the dumbest metric, but it's what we have. It's like 100,000." "And the token leader at OpenAI uses something in the hundreds of billions." "If that happens again, which I think it probably will, then in another six and a half years, the average person uses, let's say, 500 billion tokens a month." If he is right, the average person's AI usage grows five million times over the next six and a half years, and OpenAI is already building the capacity for it. That is the number enterprise boards should be planning against this year. Demand at that scale changes what a reasonable AI budget looks like. - Sam Altman (Sam Altman), CEO of OpenAI, with Garry Tan (Garry Tan) at Y Combinator (Y Combinator) Startup School 2026.
0:46

Sam Altman says the heaviest AI user on the planet six and a half years ago used what the average person on Earth uses today: "Six and a half years ago, the world token leader was an OpenAI employee using about 100,000 tokens a month." "And this seemed ludicrous at the time. The worldwide average per capita is like zero." "Now, the worldwide average of tokens per month, which I think tokens are the dumbest metric, but it's what we have. It's like 100,000." "And the token leader at OpenAI uses something in the hundreds of billions." "If that happens again, which I think it probably will, then in another six and a half years, the average person uses, let's say, 500 billion tokens a month." If he is right, the average person's AI usage grows five million times over the next six and a half years, and OpenAI is already building the capacity for it. That is the number enterprise boards should be planning against this year. Demand at that scale changes what a reasonable AI budget looks like. - Sam Altman (Sam Altman), CEO of OpenAI, with Garry Tan (Garry Tan) at Y Combinator (Y Combinator) Startup School 2026.

Karl Mehta

654,283 просмотров • 26 дней назад

Perplexity CEO Aravind Srinivas on the brutal truth about who actually makes money in AI (and why it's not who you think): Aravind argues that the real value in AI comes from orchestration. He points to products like Codex, Claude Code, and Perplexity Computer: "What is that? It's an orchestration system. It takes a model, pairs it with an agent harness." And what is an agent harness? "The simplest way of describing it is like rules for how the agent loop should run. What are all the skills and sub-agents and connectors and tools it accesses? Without the harness, you don't necessarily capture and convert the intrinsic intelligence in the model into valuable output tokens." This leads to a blunt conclusion about who has a real business in AI, and who doesn't: "If you're literally just a reseller of model tokens, you have no business, because the model will get commoditized. So even if you're a model builder, you don't have a business. As an infra layer, you have some business on serving those output tokens. But as an application layer or model builder, you don't really have a business if you're just a reseller of tokens that come directly out of the model." So where does the value accrue? "You have a business if you know how to take the model, ground it in valuable context, orchestrate it with a really good agent harness, connected to the right set of tools and connectors (whether it's personal connectors or business connectors) and provide the experience to people in one single unified system." Aravind Srinivas then explains Perplexity's specific edge: Beyond orchestrating across tools, files, and connectors, they also orchestrate across models. "That is the differentiation that Anthropic and OpenAI cannot claim, because you wouldn't find GPT-5 inside the Claude Code harness. You wouldn't find Claude Opus inside the Codex harness. These are competing with each other. Whereas you would find both these models inside Perplexity Computer." Why does this matter? Because it all comes down to power. In Aravind's framing, the fundamental cost driver in AI is watts (the one input nobody can subsidize except the government). "Whoever provides the most valuable output tokens with the least amount of power expended to produce them generates the greatest value to the end user, has the most pricing power, has the most value. That is the orchestration problem to solve." His conclusion: "The one single most important metric in AI is token value per watt per user."
2:50

Perplexity CEO Aravind Srinivas on the brutal truth about who actually makes money in AI (and why it's not who you think): Aravind argues that the real value in AI comes from orchestration. He points to products like Codex, Claude Code, and Perplexity Computer: "What is that? It's an orchestration system. It takes a model, pairs it with an agent harness." And what is an agent harness? "The simplest way of describing it is like rules for how the agent loop should run. What are all the skills and sub-agents and connectors and tools it accesses? Without the harness, you don't necessarily capture and convert the intrinsic intelligence in the model into valuable output tokens." This leads to a blunt conclusion about who has a real business in AI, and who doesn't: "If you're literally just a reseller of model tokens, you have no business, because the model will get commoditized. So even if you're a model builder, you don't have a business. As an infra layer, you have some business on serving those output tokens. But as an application layer or model builder, you don't really have a business if you're just a reseller of tokens that come directly out of the model." So where does the value accrue? "You have a business if you know how to take the model, ground it in valuable context, orchestrate it with a really good agent harness, connected to the right set of tools and connectors (whether it's personal connectors or business connectors) and provide the experience to people in one single unified system." Aravind Srinivas then explains Perplexity's specific edge: Beyond orchestrating across tools, files, and connectors, they also orchestrate across models. "That is the differentiation that Anthropic and OpenAI cannot claim, because you wouldn't find GPT-5 inside the Claude Code harness. You wouldn't find Claude Opus inside the Codex harness. These are competing with each other. Whereas you would find both these models inside Perplexity Computer." Why does this matter? Because it all comes down to power. In Aravind's framing, the fundamental cost driver in AI is watts (the one input nobody can subsidize except the government). "Whoever provides the most valuable output tokens with the least amount of power expended to produce them generates the greatest value to the end user, has the most pricing power, has the most value. That is the orchestration problem to solve." His conclusion: "The one single most important metric in AI is token value per watt per user."

Big Brain AI

42,249 просмотров • 25 дней назад

🚨 BREAKING: THERE ARE RUMORS YOU CAN NOW CREATE "SAFE TOKENS" DIRECTLY ON ETHERVISTADEX What are "Safe Tokens"? "Safe Tokens" are tokens generated through our SafeTokenFactory smart contract. These tokens are designed to eliminate vulnerabilities such as mintable functions or scammy taxes and come with a standardized implementation. Before swapping, users can easily verify whether a token is "safe" or if additional caution is needed. This marks a significant step forward in enhancing the quality of projects launched on Ethervista. But does this compromise the customizability of ERC tokens? Not at all. The Ethervista Protocol smart contract allows for a limitless range of applications. Take the $VISTA contract, for example. It's a standard ERC20 token, but with the Ethervista Protocol smart contract, it automatically buys and burns tokens. Similar logic can be applied to any ERC20 token using EthervistaDEX’s unique Protocol feature. What other features would you like to see? Wen dashboards? Wen streaming? We're on it—we just hired a full-time full-stack engineer! Special shoutout to Bonzi - FIRST MEME and MASCOT @ Ethervista and Clippy - Microsoft Anti AI Helper @ Ethervista, the first whitelisted tokens. We will continue to strongly support tokens that burn part of their liquidity before the 5-day lock period and those with strong communities and utility. A final note to creators: We would like to emphasize that burning lp-tokens does not alter your share of rewards UNTIL you remove, add, or claim rewards, which automatically updates your pool share ratio based on your current balance and the total lp-supply, as outlined in our whitepaper. This DOES NOT affect protocol fees, which are used to support both the protocol and creators.
0:50

🚨 BREAKING: THERE ARE RUMORS YOU CAN NOW CREATE "SAFE TOKENS" DIRECTLY ON ETHERVISTADEX What are "Safe Tokens"? "Safe Tokens" are tokens generated through our SafeTokenFactory smart contract. These tokens are designed to eliminate vulnerabilities such as mintable functions or scammy taxes and come with a standardized implementation. Before swapping, users can easily verify whether a token is "safe" or if additional caution is needed. This marks a significant step forward in enhancing the quality of projects launched on Ethervista. But does this compromise the customizability of ERC tokens? Not at all. The Ethervista Protocol smart contract allows for a limitless range of applications. Take the $VISTA contract, for example. It's a standard ERC20 token, but with the Ethervista Protocol smart contract, it automatically buys and burns tokens. Similar logic can be applied to any ERC20 token using EthervistaDEX’s unique Protocol feature. What other features would you like to see? Wen dashboards? Wen streaming? We're on it—we just hired a full-time full-stack engineer! Special shoutout to Bonzi - FIRST MEME and MASCOT @ Ethervista and Clippy - Microsoft Anti AI Helper @ Ethervista, the first whitelisted tokens. We will continue to strongly support tokens that burn part of their liquidity before the 5-day lock period and those with strong communities and utility. A final note to creators: We would like to emphasize that burning lp-tokens does not alter your share of rewards UNTIL you remove, add, or claim rewards, which automatically updates your pool share ratio based on your current balance and the total lp-supply, as outlined in our whitepaper. This DOES NOT affect protocol fees, which are used to support both the protocol and creators.

Ethervista

130,508 просмотров • 2 лет назад

Jensen Huang: “If that $500,000 engineer did not consume at least $250,000 worth of tokens, I'm going to be deeply alarmed.” The Nvidia CEO expects his highly paid engineers to be spending at least HALF their salaries on tokens to supercharge their abilities. @jason: “ The conversation we've had on the pod a number of times is, ‘Oh my God, look at the token usage in our companies.’ It is growing massively.” “And some people are asking, ‘Hey, when I join a company, how many tokens do I get? Because I want to be an effective employee.’” “You've postulated, I believe, $75,000 in tokens for each engineer, something like that.” “So are you spending, at Nvidia, $1 billion, $2 billion on tokens for your engineering team right now?” Jensen: “We're trying to.” “Let me give you the thought experiment: Let's say you have a software engineer or AI researcher and you pay them $500,000 a year. We do that all the time.” “That $500,000 engineer, at the end of the year, I'm going to ask them, how much did you spend in tokens?” “If that person said, ‘$5,000,’ I will go ape… something else.” “If that $500,000 engineer did not consume at least $250,000 worth of tokens, I'm going to be deeply alarmed. “And this is no different than one of our chip designers who says, ‘Guess what? I'm just going to use paper and pencil, I don't think I'm going to need any CAD tools.’” Jason: “This is a real paradigm shift, to start thinking about these all-star employees, it almost reminds me of what we learned in the NBA when LeBron James started spending a million dollars a year just on his health and his body, like in maintaining it. Here he is at age 41, still playing.” “These are incredible knowledge workers. Why wouldn't we give them superhuman abilities?”
1:18

Jensen Huang: “If that $500,000 engineer did not consume at least $250,000 worth of tokens, I'm going to be deeply alarmed.” The Nvidia CEO expects his highly paid engineers to be spending at least HALF their salaries on tokens to supercharge their abilities. @jason: “ The conversation we've had on the pod a number of times is, ‘Oh my God, look at the token usage in our companies.’ It is growing massively.” “And some people are asking, ‘Hey, when I join a company, how many tokens do I get? Because I want to be an effective employee.’” “You've postulated, I believe, $75,000 in tokens for each engineer, something like that.” “So are you spending, at Nvidia, $1 billion, $2 billion on tokens for your engineering team right now?” Jensen: “We're trying to.” “Let me give you the thought experiment: Let's say you have a software engineer or AI researcher and you pay them $500,000 a year. We do that all the time.” “That $500,000 engineer, at the end of the year, I'm going to ask them, how much did you spend in tokens?” “If that person said, ‘$5,000,’ I will go ape… something else.” “If that $500,000 engineer did not consume at least $250,000 worth of tokens, I'm going to be deeply alarmed. “And this is no different than one of our chip designers who says, ‘Guess what? I'm just going to use paper and pencil, I don't think I'm going to need any CAD tools.’” Jason: “This is a real paradigm shift, to start thinking about these all-star employees, it almost reminds me of what we learned in the NBA when LeBron James started spending a million dollars a year just on his health and his body, like in maintaining it. Here he is at age 41, still playing.” “These are incredible knowledge workers. Why wouldn't we give them superhuman abilities?”

The All-In Podcast

96,805 просмотров • 5 месяцев назад