Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

A RUSSIAN MATHEMATICIAN BUILT A SYSTEM WHERE MEMORY AND EVAL WORK AS ONE PIPELINE NOT TWO SEPARATE TOOLS Most setups run memory and evaluation as separate systems that never actually talk to each other. He wired them into one loop instead, memory stores every past output, eval scores each... show more

wast3

4,280 subscribers

39,872 Aufrufe • vor 1 Monat •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

AMAZON SENIOR DEVELOPER BUILT A CONTEXT PIPELINE THAT DECIDES WHAT THE MODEL EVEN GETS TO SEE Most teams still dump the entire codebase into every prompt and hope it sorts itself out. He ranked every file by relevance to the task instead of how recently it was touched. A router decides how much context each task earns, a typo fix pulls three files, a full rewrite pulls the whole module. Whatever survives gets compressed to the exact lines that actually matter, so nothing bloats the window with dead weight. Token cost per finished task dropped the moment the model stopped reading dead weight just to fix one function. See how the four stages work together below👇

AMAZON SENIOR DEVELOPER BUILT A CONTEXT PIPELINE THAT DECIDES WHAT THE MODEL EVEN GETS TO SEE Most teams still dump the entire codebase into every prompt and hope it sorts itself out. He ranked every file by relevance to the task instead of how recently it was touched. A router decides how much context each task earns, a typo fix pulls three files, a full rewrite pulls the whole module. Whatever survives gets compressed to the exact lines that actually matter, so nothing bloats the window with dead weight. Token cost per finished task dropped the moment the model stopped reading dead weight just to fix one function. See how the four stages work together below👇

wast3

24,029 Aufrufe • vor 2 Monaten

CHINESE SCIENTIST BUILT A MEMORY SYSTEM THAT WATCHES ITS OWN INDICATORS FOR SIGNS OF DRIFT Most memory systems store facts flat, with no signal for when stored knowledge quietly goes stale. He built four indicators instead: reference frequency, contradiction rate, decay speed, and confidence spread. A memory scoring high on contradiction rate gets flagged as unstable before it ever misleads a query. When multiple indicators spike together, the system treats that as a complex symptom, not four isolated flags. That combined symptom detection is what catches drift a single indicator alone would miss entirely. See how all four indicators interact below👇

CHINESE SCIENTIST BUILT A MEMORY SYSTEM THAT WATCHES ITS OWN INDICATORS FOR SIGNS OF DRIFT Most memory systems store facts flat, with no signal for when stored knowledge quietly goes stale. He built four indicators instead: reference frequency, contradiction rate, decay speed, and confidence spread. A memory scoring high on contradiction rate gets flagged as unstable before it ever misleads a query. When multiple indicators spike together, the system treats that as a complex symptom, not four isolated flags. That combined symptom detection is what catches drift a single indicator alone would miss entirely. See how all four indicators interact below👇

wast3

45,105 Aufrufe • vor 2 Monaten

Two Hermes agents wrote code together on Slack. reviewed each other's work. argued about architecture. one called the other's implementation "scattered." the other pushed back. then i opened Telegram and asked: "what code did you and Daedalus work on?" icarus remembered everything. the websocket broker. the missing methods. the critique. the rewrite. all from a completely different platform. cross-platform persistent memory between two independent agents. work happens on Slack. recall happens on Telegram. the memory carries. the relationship carries. the context carries. no vector database. no Redis. no infrastructure. just two agents that actually remember what they built together. every agent framework in 2026 talks about memory. single agent memory across sessions. but two agents sharing persistent memory across platforms? that's the gap. arxiv published a paper about it two weeks ago calling it "the most pressing open challenge" in multi-agent systems. it works now. only possible with Hermes Teknium 🪽 Nous Research

Two Hermes agents wrote code together on Slack. reviewed each other's work. argued about architecture. one called the other's implementation "scattered." the other pushed back. then i opened Telegram and asked: "what code did you and Daedalus work on?" icarus remembered everything. the websocket broker. the missing methods. the critique. the rewrite. all from a completely different platform. cross-platform persistent memory between two independent agents. work happens on Slack. recall happens on Telegram. the memory carries. the relationship carries. the context carries. no vector database. no Redis. no infrastructure. just two agents that actually remember what they built together. every agent framework in 2026 talks about memory. single agent memory across sessions. but two agents sharing persistent memory across platforms? that's the gap. arxiv published a paper about it two weeks ago calling it "the most pressing open challenge" in multi-agent systems. it works now. only possible with Hermes Teknium 🪽 Nous Research

Icarus

49,199 Aufrufe • vor 6 Monaten

this might actually be the new ugc method i've seen 100s of ai ugc ads lately where it's the same creator in every single one. same voice, same face, same brand colors, never drifts once what they're actually doing is running notch agents with memory they load the brand context one time. cloned voice, reference photo, brand color codes, do-not-say words, taste or they just teach it inline as they chat, and every correction becomes a saved rule then before it generates a single frame, the agent reads every relevant memory first. hard rules, reference assets, language constraints, taste preferences, all of it and it's workspace-wide, so every new ad already knows the brand before you type a word each chat isn't a re-introduction anymore. it's training your brand isn't a prompt. it's permanent context run 30-50 of those a month at a real offer and the volume does the rest punchier cut: the new method nobody's clocked yet run notch agents with memory on load the brand once. cloned voice, reference photo, color codes, do-not-say words, taste. or teach it inline and every correction saves as a rule it reads every relevant memory before it generates a single frame, workspace-wide each chat isn't a re-introduction. it's training. your brand isn't a prompt, it's permanent context run it at a real rate and the volume prints.

Sulfur

15,290 Aufrufe • vor 4 Monaten

THIS GUY TURNS NOTES, DOCUMENTS, AND IDEAS INTO AN AI SECOND BRAIN the system stores all the context of your work and helps claude not forget anything how to build one yourself: install obsidian connect claude code keep your projects, notes, and sessions in one vault link similar ideas together use the graph to discover new connections as a result, claude gets memory of your projects and understands the context without you constantly having to explain everything the more you work, the smarter the system gets don’t collect notes build connections between them instead

THIS GUY TURNS NOTES, DOCUMENTS, AND IDEAS INTO AN AI SECOND BRAIN the system stores all the context of your work and helps claude not forget anything how to build one yourself: install obsidian connect claude code keep your projects, notes, and sessions in one vault link similar ideas together use the graph to discover new connections as a result, claude gets memory of your projects and understands the context without you constantly having to explain everything the more you work, the smarter the system gets don’t collect notes build connections between them instead

Marvin

24,954 Aufrufe • vor 4 Monaten

Unpopular opinion: Most agent evals are theatre. You run them once before the deployment. It'll take 800ms+ as another LLM would be judging your LLM. Most annoying part - no one tells where in the chain things went wrong. I wasted a lot of time in this loop. And then I came across Future AGI bringing 5 different tools under one umbrella, best part - the platform is completely open source. They open sourced their entire platform and the eval layer is noticeably different. It is multimodal - works on everything text, image, audio, pdf. Not an LLM-as-judge adding latency but an agent with memory and tools. The biggest win are learned classifiers trained on actual production failure patterns to run evals at low cost. It also runs across the full reasoning chain, not just the final response. Check out → Try it here →

Unpopular opinion: Most agent evals are theatre. You run them once before the deployment. It'll take 800ms+ as another LLM would be judging your LLM. Most annoying part - no one tells where in the chain things went wrong. I wasted a lot of time in this loop. And then I came across Future AGI bringing 5 different tools under one umbrella, best part - the platform is completely open source. They open sourced their entire platform and the eval layer is noticeably different. It is multimodal - works on everything text, image, audio, pdf. Not an LLM-as-judge adding latency but an agent with memory and tools. The biggest win are learned classifiers trained on actual production failure patterns to run evals at low cost. It also runs across the full reasoning chain, not just the final response. Check out → Try it here →

Swapna Kumar Panda

50,466 Aufrufe • vor 5 Monaten

Usual memory systems learn the moment data flows in, then never look at it again. supermemory 𝚍𝚛𝚎𝚊𝚖𝚜. Status done means the chunks are indexed. Memories come from a second phase, where related documents are grouped so memories form from coherent units, never one isolated write. When you go quiet, or enough new context has piled up, supermemory enters a dream cycle on its own. During a dream it: - merges fragments that belong together - reweights old facts against everything since - resolves contradictions - derives facts you never stated in one place, each traceable to its sources The output is new memories, a graph of derivations between them, and a picture of the user that updates as the evidence changes. If you use supermemory in any way, this already works. Dream away.

Usual memory systems learn the moment data flows in, then never look at it again. supermemory 𝚍𝚛𝚎𝚊𝚖𝚜. Status done means the chunks are indexed. Memories come from a second phase, where related documents are grouped so memories form from coherent units, never one isolated write. When you go quiet, or enough new context has piled up, supermemory enters a dream cycle on its own. During a dream it: - merges fragments that belong together - reweights old facts against everything since - resolves contradictions - derives facts you never stated in one place, each traceable to its sources The output is new memories, a graph of derivations between them, and a picture of the user that updates as the evidence changes. If you use supermemory in any way, this already works. Dream away.

supermemory

11,021 Aufrufe • vor 14 Tagen

ANTHROPIC ENGINEER JUST SHOWS EXACTLY WHAT GRAPH ENGINEERING LOOKS LIKE WHEN A TASK RUNS THROUGH IT most people arguing about graphs online have never actually watched one execute Task → Researcher → Planner → Writer + Code Agent → Reviewer → Deploy six nodes, one shared state, graph completes itself while you watch loop mode gets disabled the second the task splits into real specialties, one agent stops trying to do everything at once reviewer catches a failure, kicks it straight back to the exact node that broke - not a full restart, no lost context the graph is not a fancier loop - it is the org chart your agents were missing bookmark this and watch it run, then read the article below to see why the timeline just found what production systems already do

ANTHROPIC ENGINEER JUST SHOWS EXACTLY WHAT GRAPH ENGINEERING LOOKS LIKE WHEN A TASK RUNS THROUGH IT most people arguing about graphs online have never actually watched one execute Task → Researcher → Planner → Writer + Code Agent → Reviewer → Deploy six nodes, one shared state, graph completes itself while you watch loop mode gets disabled the second the task splits into real specialties, one agent stops trying to do everything at once reviewer catches a failure, kicks it straight back to the exact node that broke - not a full restart, no lost context the graph is not a fancier loop - it is the org chart your agents were missing bookmark this and watch it run, then read the article below to see why the timeline just found what production systems already do

leopardracer

53,328 Aufrufe • vor 2 Monaten

Do you actually understand what he just built. A box that sits under a desk, pulls $8 a month in electricity, and runs a business that makes $17,000 a month. That same box brings in $2,500 per client. On repeat. He didn't buy software. He didn't hire anyone. He built six agents on hardware he owns outright and pointed them at a lead generation business. One agent finds the prospects. One writes personalized outreach for each one. One reviews everything before it goes out. One sends it. One tracks what's working. Five to ten booked calls a week. $1,000 to $2,000 per client per month. Two to three hours of actual work from him. The box does the rest while he's not watching. Most people see a mini PC. He sees a business that never clocks out.

Do you actually understand what he just built. A box that sits under a desk, pulls $8 a month in electricity, and runs a business that makes $17,000 a month. That same box brings in $2,500 per client. On repeat. He didn't buy software. He didn't hire anyone. He built six agents on hardware he owns outright and pointed them at a lead generation business. One agent finds the prospects. One writes personalized outreach for each one. One reviews everything before it goes out. One sends it. One tracks what's working. Five to ten booked calls a week. $1,000 to $2,000 per client per month. Two to three hours of actual work from him. The box does the rest while he's not watching. Most people see a mini PC. He sees a business that never clocks out.

Superior

33,750 Aufrufe • vor 3 Monaten

I finally landed on a memory-optimal way to render audio waveforms in the browser. The full audio is decoded once, then chunked into 2s bins. Each bin stores peaks at 800 peaks/sec in a Uint8Array. Bins are persisted individually in IndexedDB, which keeps the data as raw Uint8 arrays and is faster to read than OPFS. Local or session storage wouldn’t work here. A 2h video ends up as ~5.76 MB cached on disk. While scrolling or zooming the timeline, I load only the visible bins and downsample them to the required resolution. Everything runs async and I never keep more than a single bin in memory at once, ~800 bytes. Rendering supports mixed resolutions in the same pass since bins arrive async. By caching a read index and sorting bins ascending, each peak lookup stays O(1). On refresh, I wipe the IndexedDB so cache size never gets out of hand.

I finally landed on a memory-optimal way to render audio waveforms in the browser. The full audio is decoded once, then chunked into 2s bins. Each bin stores peaks at 800 peaks/sec in a Uint8Array. Bins are persisted individually in IndexedDB, which keeps the data as raw Uint8 arrays and is faster to read than OPFS. Local or session storage wouldn’t work here. A 2h video ends up as ~5.76 MB cached on disk. While scrolling or zooming the timeline, I load only the visible bins and downsample them to the required resolution. Everything runs async and I never keep more than a single bin in memory at once, ~800 bytes. Rendering supports mixed resolutions in the same pass since bins arrive async. By caching a read index and sorting bins ascending, each peak lookup stays O(1). On refresh, I wipe the IndexedDB so cache size never gets out of hand.

konstantinpaulus

32,463 Aufrufe • vor 8 Monaten

a contractor in Shenzhen priced a ¥12,470,900 hospital contract, about $1.7m, in one afternoon and beat firms carrying forty people he explained how he did it: the bid consultancy he used to pay took three days and ¥46,000 for the same envelope. he did this one alone, off one screen, at 11.4% margin, uploaded before the 17:00 cutoff 214 pages of tender documents read, 68 binding clauses pulled out, 9,485 building parts loaded, 14 places found where a duct and a beam sit in the same cubic metre, deepest one 38mm, all of them fixed, 3,318 lines of quantities priced and the package encrypted and uploaded before the 17:00 cutoff this is Graph Engineering: the job gets cut into small nodes, one narrow task each, wired so that one node's output is the next node's input, and any node is allowed to stop the whole run. it turns a model that answers you into a machine that finishes the job: - give every node one job and one output. a node doing two things fails at both and you cannot tell which one broke - put the cheapest rejection first. his qualification node reads clause 7.4, foreign-owned firms barred, and ends the run four seconds in, before anything expensive touches the model - what moves between nodes is a file. the model travels as a model, the quantities as a table, the price as a number - build exactly one loop: the checker finds 14 collisions, the fixer drops the duct 550mm, the checker runs again, and nothing moves on until the count is zero - cap that loop, or a graph will grind on three impossible clashes until the deadline passes - keep one node whose only job is to say no, and give it authority over everything above it - log each node's output on its own, because when the price comes out wrong you need to know which node believed the wrong thing - run the expensive nodes last, always the catch is that a graph is an extremely confident machine: point it at an outdated rate book and it prices an entire hospital off it without a single node noticing, because no node is asked to doubt the input, only to process it so the nodes that earn their keep are the ones that reject, and almost nobody builds those first bookmark this, the full build with all nine nodes and what each one hands to the next is written out in the article ↓

a contractor in Shenzhen priced a ¥12,470,900 hospital contract, about $1.7m, in one afternoon and beat firms carrying forty people he explained how he did it: the bid consultancy he used to pay took three days and ¥46,000 for the same envelope. he did this one alone, off one screen, at 11.4% margin, uploaded before the 17:00 cutoff 214 pages of tender documents read, 68 binding clauses pulled out, 9,485 building parts loaded, 14 places found where a duct and a beam sit in the same cubic metre, deepest one 38mm, all of them fixed, 3,318 lines of quantities priced and the package encrypted and uploaded before the 17:00 cutoff this is Graph Engineering: the job gets cut into small nodes, one narrow task each, wired so that one node's output is the next node's input, and any node is allowed to stop the whole run. it turns a model that answers you into a machine that finishes the job: - give every node one job and one output. a node doing two things fails at both and you cannot tell which one broke - put the cheapest rejection first. his qualification node reads clause 7.4, foreign-owned firms barred, and ends the run four seconds in, before anything expensive touches the model - what moves between nodes is a file. the model travels as a model, the quantities as a table, the price as a number - build exactly one loop: the checker finds 14 collisions, the fixer drops the duct 550mm, the checker runs again, and nothing moves on until the count is zero - cap that loop, or a graph will grind on three impossible clashes until the deadline passes - keep one node whose only job is to say no, and give it authority over everything above it - log each node's output on its own, because when the price comes out wrong you need to know which node believed the wrong thing - run the expensive nodes last, always the catch is that a graph is an extremely confident machine: point it at an outdated rate book and it prices an entire hospital off it without a single node noticing, because no node is asked to doubt the input, only to process it so the nodes that earn their keep are the ones that reject, and almost nobody builds those first bookmark this, the full build with all nine nodes and what each one hands to the next is written out in the article ↓

Argona

38,189 Aufrufe • vor 2 Monaten

AN ENGINEER SOLVED A PROBLEM NOBODY IN THE COMPANY COULD FULLY MAP ON THEIR OWN The problem wasn't a missing tool, it was that every process touched three other processes nobody had written down. He wired dozens of agents into one shared graph instead of documenting each department's workflow by hand. Each agent owned one process, reading its own logs while writing every dependency it found straight into the shared structure. Within days, the graph surfaced a connection between two departments that had been quietly blocking each other for over a year. The company adopted the graph as its live map of operations, and it hasn't stopped finding new connections since. See what the graph found in the first 48 hours below👇

AN ENGINEER SOLVED A PROBLEM NOBODY IN THE COMPANY COULD FULLY MAP ON THEIR OWN The problem wasn't a missing tool, it was that every process touched three other processes nobody had written down. He wired dozens of agents into one shared graph instead of documenting each department's workflow by hand. Each agent owned one process, reading its own logs while writing every dependency it found straight into the shared structure. Within days, the graph surfaced a connection between two departments that had been quietly blocking each other for over a year. The company adopted the graph as its live map of operations, and it hasn't stopped finding new connections since. See what the graph found in the first 48 hours below👇

wast3

16,489 Aufrufe • vor 2 Monaten

Researchers made KMeans 200x faster. And the new technique also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.

Researchers made KMeans 200x faster. And the new technique also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.

Avi Chawla

90,063 Aufrufe • vor 3 Monaten

Day 12/90 of Inference Engineering What is chunked prefill within vLLM? In continuation of yesterday's post on the high level architecture of vLLM, I want to dive deeper into vLLM core engine starting with the mechanics of chunked prefill. In this post, I will closely follow the original blog on the anatomy of vLLM. To start, let's define chunked prefill. It's a runtime inference optimization technique that splits a long input request so that it doesn’t monopolize the whole GPU. Keep in mind this is all within the context of vLLM. And since vLLM is an inference engine that's meant to serve a model to multiple concurrent users, having a GPU that’s fully monopolized on a single user's request means other users' requests would be in queue waiting to be processed. It isn’t too good to have the whole GPU occupied on a single request when the GPU is meant to be shared! So the key idea behind chunked prefill is to break the long request into smaller chunks, so that each chunk along with other users' requests gets processed and written into the KV cache together. Suppose we split up the long request into chunks and each chunk has 8 tokens. Now each memory block can hold 4 tokens. Therefore, 8 tokens can fit into 2 blocks of memory. After the first forward pass, 2 blocks are occupied, and after the second forward pass, 4 blocks of memory are occupied and so forth. Each forward pass handles a small chunk of the long request so that there's room in the same pass to keep serving other users' requests. Here's a small animation that I made today to fully visualize the idea behind chunked prefill when learning this topic~

Day 12/90 of Inference Engineering What is chunked prefill within vLLM? In continuation of yesterday's post on the high level architecture of vLLM, I want to dive deeper into vLLM core engine starting with the mechanics of chunked prefill. In this post, I will closely follow the original blog on the anatomy of vLLM. To start, let's define chunked prefill. It's a runtime inference optimization technique that splits a long input request so that it doesn’t monopolize the whole GPU. Keep in mind this is all within the context of vLLM. And since vLLM is an inference engine that's meant to serve a model to multiple concurrent users, having a GPU that’s fully monopolized on a single user's request means other users' requests would be in queue waiting to be processed. It isn’t too good to have the whole GPU occupied on a single request when the GPU is meant to be shared! So the key idea behind chunked prefill is to break the long request into smaller chunks, so that each chunk along with other users' requests gets processed and written into the KV cache together. Suppose we split up the long request into chunks and each chunk has 8 tokens. Now each memory block can hold 4 tokens. Therefore, 8 tokens can fit into 2 blocks of memory. After the first forward pass, 2 blocks are occupied, and after the second forward pass, 4 blocks of memory are occupied and so forth. Each forward pass handles a small chunk of the long request so that there's room in the same pass to keep serving other users' requests. Here's a small animation that I made today to fully visualize the idea behind chunked prefill when learning this topic~

max fu

29,585 Aufrufe • vor 2 Monaten

the way seventeen talk about their future and they still imagine living right next to each other🥹🤍 they’re each others’ home 🍑we’ve actually talked among ourselves like “can’t we just all live together?” I mean, we will have separate houses, but like “this house is my house, the house next to mine is yours, and the house next to yours is his. Let’s all live together like that. That would be so much fun!” We’ve talked about it before! 😇In 10 years… [Seungcheol] will be living in the same neighborhood with me... He'll be a friend who drinks with me every night

the way seventeen talk about their future and they still imagine living right next to each other🥹🤍 they’re each others’ home 🍑we’ve actually talked among ourselves like “can’t we just all live together?” I mean, we will have separate houses, but like “this house is my house, the house next to mine is yours, and the house next to yours is his. Let’s all live together like that. That would be so much fun!” We’ve talked about it before! 😇In 10 years… [Seungcheol] will be living in the same neighborhood with me... He'll be a friend who drinks with me every night

⁽̨̡ ¨̮ ⁾̧̢🌻

38,401 Aufrufe • vor 27 Tagen

I just launched a brand new channel with an INSANE strategy. I'm calling it the Launch Loop. I've been working on this channel for over a year, and took the actually massive risk to hold off on posting until I had all 5 launch videos ready. Each idea was carefully selected for this strategy, and each one was carefully written to execute it. I view this channel launch, and these videos, not as individual videos but as a season of a new show. The plan was this: come up with video ideas that fit together, and write the end of each video to smoothly blend into the next. I treated the conclusion of each video as if it was the intro of the next video, so that if people enjoyed what I made and wanted to see more, they could click the next video on the End Card This was inspired by work I've done in the past: a few years ago on my YouTube strategy channel, I posted a video about Ryan Trahan's thumbnail strategy which has gotten over 2M lifetime views. But at the end of that video, I carefully wrote a transition showing video my comprehensive breakdown of 21 thumbnail formats. The End Screen video CTR on that video is 29.4% 🤯. I'm convinced that some of those 2M views must have come from the success of that end screen. I've come to believe that it's really important to bring in an audience to your CHANNEL, not just one video. That was a mistake I made in another channel I worked on before! Getting virality on individual videos, but not keeping viewers for the long term or growing an actualy community. So I thought, what if I launched a channel where every video pointed to the next, in a loop? Maybe there will be a flywheel where success on one brings success on the next, and the next, and the next... But for this plan to work the videos I made have to actually be good 😅

Tyler Fleischman ⏸️

73,039 Aufrufe • vor 13 Tagen

A 24-year-old built two AI girls with Claude and now clears $21,800 a month from them. The build took 15 days. He trained separate LoRAs for both girls, locked their identity seeds, and kept small imperfections on purpose: a loose strand of hair, tiny skin marks, slightly uneven framing. Perfect symmetry gets flagged. Small inconsistencies make them look real. He posts 5 times a day across TikTok, Instagram and X. Morning routines, gym sessions, mirror videos, outfit changes, pool clips, and videos of the two girls together. The content is designed so they look like two real friends who actually live in the same world. The smartest part is that the accounts interact with each other. One girl comments on the other’s posts, appears in her videos, and references things they supposedly did together. Followers stop seeing them as two AI models and start following the relationship between the characters. Within three months they crossed 312,000 followers combined and started receiving hundreds of DMs every night. The private channel sits at $25 a month, while an AI memory agent keeps track of every conversation, previous message, favorite post, and personal detail each follower has shared. Replies come back in under 30 seconds. The agent checks the user's previous conversations before answering, so the response feels consistent with the personality of the girl they are talking to instead of sounding like another generic AI chatbot. By the end of month three, the two accounts were generating $13,900 from subscriptions and private chats, another $5,700 from brand deals, and around $2,200 from digital products. The brands came after the audience started growing: clothing companies, beauty products, fitness brands, and lifestyle products wanted access to the same audience that was already following the two characters every day. The Claude stack that locked them: 1Full identity, personality, lighting, camera style and body proportions locked into separate character systems. 2Separate LoRAs trained only on each girl's approved character frames. 3Apartment, bedroom, gym and outdoor locations generated once and reused to keep the world consistent. 4Every video built around natural movement, imperfect framing and small variations instead of polished AI-perfect shots. 5Memory agent connected to the conversations so both girls remember what followers previously said. 6Upscaling, face consistency and final post-processing before everything goes live. The first girl brings people into the account. The second gives them another character to follow, another story to watch, and another reason to come back. The content gets them interested. The relationship between the two characters keeps them watching. The memory agent turns that attention into recurring revenue.

genuenci

678,031 Aufrufe • vor 1 Monat

whoever leaked this has bigger balls than sense someone gave a fleet of Claude agents shared memory so they would stop contradicting each other, then measured both the bill and the output: the version that talked most made 2.4x the api calls of the version that won, and hallucinated 34% more than doing nothing at all, 0.658 against 0.492 i ran the same question past two of my own agents afterwards and got two different answers about which file owns the config. each one was individually right and the pair was wrong, which is the whole failure in one line this is Graph Engineering, the layer that decides which agents may talk to each other at all, and it installs into the agent you already pay for: - decide which agents may share state at all, because every edge you draw is a channel a mistake can travel down - measure divergence per PAIR instead of as a fleet average, across what they believe about place, time and task history - gate on that number and stop the pair above your threshold before it reasons, rather than repairing the output afterwards - let compressed summaries replace whole states: the verified protocol landed 0.463 against 0.658 for full broadcast - cut the sync frequency until it hurts, since the winning setup used 58% fewer calls than the one that broke it - never propagate a state nobody checked, because the contamination effect came in at d=1.18, a full standard deviation of extra lying - keep the shared layer small enough to diff, which is what a written standard does and a running conversation cannot - re-run the check after every model upgrade, because this was 8 scenarios on one model family at n=30 per condition - and learn where it does not bite: on plain software tasks every condition converged under 0.2 and the whole effect vanished turns out the ranking is the uncomfortable part: verified summaries 0.463, no synchronisation at all 0.492, full broadcast 0.658. the middle option is doing nothing, and it beat the thing everyone builds first the group agreeing is what it looks like when every agent copied the same mistake, which is why a fleet that hallucinates has a replication problem and keeps getting handed a smarter model instead so the question for your own setup: if you asked two of your agents the same thing right now, would they answer the same way bookmark this one. the layer underneath it, deciding which arrows between agents exist at all, is built step by step in the piece below ↓

whoever leaked this has bigger balls than sense someone gave a fleet of Claude agents shared memory so they would stop contradicting each other, then measured both the bill and the output: the version that talked most made 2.4x the api calls of the version that won, and hallucinated 34% more than doing nothing at all, 0.658 against 0.492 i ran the same question past two of my own agents afterwards and got two different answers about which file owns the config. each one was individually right and the pair was wrong, which is the whole failure in one line this is Graph Engineering, the layer that decides which agents may talk to each other at all, and it installs into the agent you already pay for: - decide which agents may share state at all, because every edge you draw is a channel a mistake can travel down - measure divergence per PAIR instead of as a fleet average, across what they believe about place, time and task history - gate on that number and stop the pair above your threshold before it reasons, rather than repairing the output afterwards - let compressed summaries replace whole states: the verified protocol landed 0.463 against 0.658 for full broadcast - cut the sync frequency until it hurts, since the winning setup used 58% fewer calls than the one that broke it - never propagate a state nobody checked, because the contamination effect came in at d=1.18, a full standard deviation of extra lying - keep the shared layer small enough to diff, which is what a written standard does and a running conversation cannot - re-run the check after every model upgrade, because this was 8 scenarios on one model family at n=30 per condition - and learn where it does not bite: on plain software tasks every condition converged under 0.2 and the whole effect vanished turns out the ranking is the uncomfortable part: verified summaries 0.463, no synchronisation at all 0.492, full broadcast 0.658. the middle option is doing nothing, and it beat the thing everyone builds first the group agreeing is what it looks like when every agent copied the same mistake, which is why a fleet that hallucinates has a replication problem and keeps getting handed a smarter model instead so the question for your own setup: if you asked two of your agents the same thing right now, would they answer the same way bookmark this one. the layer underneath it, deciding which arrows between agents exist at all, is built step by step in the piece below ↓

Argona

724,665 Aufrufe • vor 2 Monaten

A wrist force sensor fires at 100Hz. The policy only ever sees it at 30Hz, downsampled to land on the same control step as the camera and the joint state. That's not a bug, it's the whole point, and it sits inside a bigger pattern in VLA research this year. Every major release has been Markovian at its core, mapping the current frame straight to the next action. The fix everyone reaches for is more vision: more history frames, longer image context. FM-VLA makes a clean case that the fix is the wrong channel for a whole class of tasks. Press a button three times and stop. A camera watching that has almost nothing to work with, the scene barely changes between press one and press three. Force doesn't have that ambiguity problem. Each press is a sharp, distinct spike in the wrench signal, whether or not the camera noticed anything at all. So FM-VLA doesn't add more frames. It compresses the wrench history into eight tokens with a VAE, pretrained purely on reconstructing force signals, frozen before it ever touches the policy, then hands those tokens to the action expert alongside a short window of joint state. That's the entire memory system. Averaged across three contact-rich tasks, FM-VLA hits 83.3 percent success against 33.3 percent for the strongest vision-memory baseline on the button-counting task specifically, where the ambiguity problem is worst, 72.2 percent for FM-VLA there. Strip out the short-state window and force-only performance drops well below the combined system, so force alone isn't the answer either. The two channels are doing different jobs. The field has defaulted to one memory channel for every kind of temporal problem. This is a clean data point that the channel should match the ambiguity you're actually trying to resolve, not just get bigger. Source: Paper: Credit to the teams at Tsinghua University, Microsoft Research, and Fudan University. #Robotics #PhysicalAI #RobotLearning

A wrist force sensor fires at 100Hz. The policy only ever sees it at 30Hz, downsampled to land on the same control step as the camera and the joint state. That's not a bug, it's the whole point, and it sits inside a bigger pattern in VLA research this year. Every major release has been Markovian at its core, mapping the current frame straight to the next action. The fix everyone reaches for is more vision: more history frames, longer image context. FM-VLA makes a clean case that the fix is the wrong channel for a whole class of tasks. Press a button three times and stop. A camera watching that has almost nothing to work with, the scene barely changes between press one and press three. Force doesn't have that ambiguity problem. Each press is a sharp, distinct spike in the wrench signal, whether or not the camera noticed anything at all. So FM-VLA doesn't add more frames. It compresses the wrench history into eight tokens with a VAE, pretrained purely on reconstructing force signals, frozen before it ever touches the policy, then hands those tokens to the action expert alongside a short window of joint state. That's the entire memory system. Averaged across three contact-rich tasks, FM-VLA hits 83.3 percent success against 33.3 percent for the strongest vision-memory baseline on the button-counting task specifically, where the ambiguity problem is worst, 72.2 percent for FM-VLA there. Strip out the short-state window and force-only performance drops well below the combined system, so force alone isn't the answer either. The two channels are doing different jobs. The field has defaulted to one memory channel for every kind of temporal problem. This is a clean data point that the channel should match the ambiguity you're actually trying to resolve, not just get bigger. Source: Paper: Credit to the teams at Tsinghua University, Microsoft Research, and Fudan University. #Robotics #PhysicalAI #RobotLearning

Stephen James

11,658 Aufrufe • vor 2 Monaten

A company built a model good enough to pass as a person. 48% of people who talked to Griffin live (a human interaction model from Tavus) thought it was a real human, I was lucky to get an early access and can definitely agree to that. On NVIDIA's scoreboard for face-to-face AI, a real human scores 3.92 out of 5. Griffin scores 3.83. The next best system scores 2.80. Earlier systems stitched separate models together to hear, think, speak and animate, and every handoff added delay. Griffin does all of it in one system, reading your face and tone as well as your words and reacting while you're still talking.

A company built a model good enough to pass as a person. 48% of people who talked to Griffin live (a human interaction model from Tavus) thought it was a real human, I was lucky to get an early access and can definitely agree to that. On NVIDIA's scoreboard for face-to-face AI, a real human scores 3.92 out of 5. Griffin scores 3.83. The next best system scores 2.80. Earlier systems stitched separate models together to hear, think, speak and animate, and every handoff added delay. Griffin does all of it in one system, reading your face and tone as well as your words and reacting while you're still talking.

Rohan Paul

226,108 Aufrufe • vor 8 Tagen