Jay Hack
@mathemagic1an • 78,638 subscribers
Head of AI @clickup. Tweets about AI, computing and their impacts on society. Previously founder @codegen / ML @palantir / startups. Not a pseudonym.
Shorts
Videos
Interesting approach to long context/continual learning here from Baseten at Cursor Compact long trajectories by compressing a prefix of the KV cache using an MLP/autoencoder. You can train this "compactor" MLP by learning to reconstruct activations that the original KV cache would produce on subsequent tokens. This maximally reconstructs information from long context that's useful for subsequent outputs. If you run an agent over extremely long context and run this compaction recursively, the activations of this compressed KV cache become like trained weights. Similar to task-specific LoRA or "cartridges". Would not be surprised if OpenAI is running a similar algorithm for their blackbox compaction. Clear benefits here if you can avoid busting the cache, as e.g. compacting via writing to a text file would. Seeing many emerging approaches to fixed size latent states for LLMs that seem promising. If building a task-specific KV cache compression ends up being more sample efficient than running backprop, getting this to "work" feels like one of the 2-3 remaining breakthroughs on the path to true AGI
Jay Hack30,031 次观看 • 3 个月前
What happens when you give a frontier agent access to a full company's history of docs, messages, tasks and more? As I'm onboarding to ClickUp, this feels like an absolute cheat code. "Cursor for your entire job." Full history of the company's thinking, decision making and operations are natively accessible w/ first-party tools and data access. No exaggeration this has reduced the amount of overhead to get ramped up (# of messages sent) by >60%. At Codegen we built an agent that straddles multiple applications - Slack, Linear, Notion, etc. - with many obvious limitations. Agents cannot search Slack historical conversations, for example, nor can you index Slack data. Any question concerning a historical decision is just out of reach. There are clear constraints of fragmentation preventing otherwise extremely capable models from performing productive tasks. Seems the biggest bottleneck to agent productivity in early 2026 is not capabilities or knowledge, it's the politics and incentives around SAAS fragmentation. Very bullish for all-in-one horizontal SAAS and what that unlocks
Jay Hack17,395 次观看 • 9 个月前
"PR review with an agent" from Graphite (we've moved to @graphite) Just got preview access. IMO this is the leading agentic PR review UI. Allows you to chat inline, make modifications etc. Honestly this should have been baked into Github in late 2023. Excited to see it go live!
Jay Hack11,976 次观看 • 1 年前
没有更多内容可加载