We used RL to train models that create curated context from long documents for downstream use by…
@appliedcompute8346 位关注者
37,368 次观看
8 条评论
评论来自 X (Twitter) 上的原帖

@appliedcompute
Neural cheat-sheets approach the performance of learned KV-caches — which are several orders of magnitude larger and not human-readable — while preserving the auditability of natural language summaries that enterprise workflows require. One can think of a neural-cheat sheet as a learned cache in token space.
17 个赞

@appliedcompute
At deployment time, the trained model takes in a new knowledge artifact: an enterprise SOP, a meeting transcript, a coding agent trace, or even a novella, and outputs a dense set of notes that captures fine-grained detail and amortizes reasoning. This approach is used to build memory for agents, instantiated at Applied Compute as a ContextBase.
11 个赞

@appliedcompute
Read the full research report on our blog:
15 个赞

@themathgeek13
Aligns with extraction of domain-specific ontologies and schemas from unstructured text documents, something for which we fine-tuned (with SFT/LoRA) specialized triplet extraction models for finance, chip design, legal and healthcare a couple of years ago!
3 个赞

@uzpg_
nice - @KaivuHariharan @atticuswzf cool to see someone do this
3 个赞

@jake_010202
this should also be used for compaction
1 个赞

@LegalPrimes
Neat
1 个赞

@sebuzdugan
watched shorthand help agents until one abbreviation collision poisoned the next retrieval step