Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

introducing operator-1, an autonomous document extraction agent operator-1 saturates multiple 3rd party benchmarks, and delivers the most accurate performance on complex tasks that other solutions can't handle: > long tables with thousands of rows > lengthy files with context scattered across hundreds of pages > dense diagrams and images... show more

Kushal Byatnal

7,540 subscribers

20,838 views • 14 hours ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

28 Comments

scott13 hours ago

even more persistent memory than my gf

Kushal Byatnal13 hours ago

i've been trying to come up w a reply for 5 min now

Kushal Byatnal14 hours ago

it saturates three extraction benchmarks, each with differing datasets and methodology - 99.3% on LongArray-Extract (extend) - 99.8% on LongExtractBench (reducto + micro1) - 96.37% on ExtractBench (llamaindex)

Kushal Byatnal14 hours ago

full details in our blog post: go try it out today on your hardest documents!

Abimael Martell14 hours ago

bad day to be a long document

Kushal Byatnal13 hours ago

"bad day to be a edge case in a complex document represented within a 3rd party benchmark"

Davis Treybig13 hours ago

We need new benchmarks! This is too powerful. So bullish on treating everything as agent plus sandbox problem

Kushal Byatnal13 hours ago

“humanity’s last extraction benchmark” stay tuned 👀

aiden brotman13 hours ago

Changing the game yet again

Jing13 hours ago

LFG

Sean Er13 hours ago

lets gooo extend!!

Pri13 hours ago

Pleasure working on the launch video! 🔥

Kushal Byatnal13 hours ago

you cooked sir

Nikhil12 hours ago

Absolutely cooked with this one 🔥

bo lau13 hours ago

BOOM!

chris12 hours ago

LETS GO TEAM!

Darren Chow13 hours ago

not to mention the animation is so so good!

Kenneth Auchenberg 🛠12 hours ago

✨🤖

Mike Bedarev13 hours ago

stop breaking X @kushalbyatnal

John Zeratsky12 hours ago

unreal

Sadok12 hours ago

building custom workspaces for specialized agents makes so much sense

Josh Elgar11 hours ago

Looks fantastic, price is pretty high tbf

Kushal Byatnal11 hours ago

i hear you, we're revamping pricing shortly actually (stay tuned) in the meantime, we also offer significant discounts on our non-PAYG plans so don't let that stop you from trying it out

Elias Stråvik12 hours ago

congrats on the launch!

Dominik13 hours ago

long tables with thousands of rows is where most extraction quietly lies. good that you named it first.

MJ_mufc13 hours ago

Is there any document extraction company that isn't No.1 on their eval benchmarks

Kushal Byatnal13 hours ago

this is run on benchmarks created by our competitors

MJ_mufc13 hours ago

And in a day or two, they either refute it, or say they're No.1 again

Related Videos

We built an AI agent that lets you vibe-code document extraction - high accuracy and citations over the most complex documents. Our latest release lets you upload documents as context. All you then have to do is describe what you want extracted in natural language. 💡 Our agent will then read the document with file tools to infer the right schema, validation rules, and other pre/postprocessing logic. ✅ It will give you back a workflow that can extract over thousands/millions of documents at scale. You can still of course review and edit every output before approving. Stop handling paperwork manually; just upload files, describe your task, and let our agent handle the rest. Our vision for LlamaAgents is to provide the most advanced and easy-to-use way for you to orchestrate document work. Walkthrough: Check it out: If you’re interested in reducing the operational burden of document extraction (invoices, claims, onboarding forms), come talk to us!

We built an AI agent that lets you vibe-code document extraction - high accuracy and citations over the most complex documents. Our latest release lets you upload documents as context. All you then have to do is describe what you want extracted in natural language. 💡 Our agent will then read the document with file tools to infer the right schema, validation rules, and other pre/postprocessing logic. ✅ It will give you back a workflow that can extract over thousands/millions of documents at scale. You can still of course review and edit every output before approving. Stop handling paperwork manually; just upload files, describe your task, and let our agent handle the rest. Our vision for LlamaAgents is to provide the most advanced and easy-to-use way for you to orchestrate document work. Walkthrough: Check it out: If you’re interested in reducing the operational burden of document extraction (invoices, claims, onboarding forms), come talk to us!

Jerry Liu

20,857 views • 7 months ago

New short course: Build AI Apps with MCP Servers: Working with Box Files, built with Box and taught by Ben Kus , their CTO. Many AI applications require custom code for basic file operations. The Model Context Protocol (MCP) standardizes this by letting you offload file tasks to dedicated servers that provide tools an LLM can use directly. In this course, you'll process documents stored in a Box folder using the Box MCP server. Rather than writing custom integration code to connect to the Box API and download files, you'll design your application to use the tools provided via MCP. Skills you'll gain: - Build an LLM-powered document processing app, using the Box MCP server to access files - Design a multi-agent system using Google's Agent Development Kit (ADK), consisting of specialized agents for file operations - Coordinate the multi-agent workflow through an orchestrator that uses the Agent2Agent (A2A) protocol to connect to the agents You'll start with a local file-processing app, refactor it to work with Box's MCP server, then evolve it into a multi-agent system. Sign up here:

New short course: Build AI Apps with MCP Servers: Working with Box Files, built with Box and taught by Ben Kus , their CTO. Many AI applications require custom code for basic file operations. The Model Context Protocol (MCP) standardizes this by letting you offload file tasks to dedicated servers that provide tools an LLM can use directly. In this course, you'll process documents stored in a Box folder using the Box MCP server. Rather than writing custom integration code to connect to the Box API and download files, you'll design your application to use the tools provided via MCP. Skills you'll gain: - Build an LLM-powered document processing app, using the Box MCP server to access files - Design a multi-agent system using Google's Agent Development Kit (ADK), consisting of specialized agents for file operations - Coordinate the multi-agent workflow through an orchestrator that uses the Agent2Agent (A2A) protocol to connect to the agents You'll start with a local file-processing app, refactor it to work with Box's MCP server, then evolve it into a multi-agent system. Sign up here:

Andrew Ng

82,067 views • 1 year ago

The most amazing aspect of DeepSeek Open Source is that just the Reasoning Engine can be isolated and used with any other LLM. In fact you can have a mixture of Reasoning Engines cascade around a problem and then use an Operator type agent AI to function on the the results.

The most amazing aspect of DeepSeek Open Source is that just the Reasoning Engine can be isolated and used with any other LLM. In fact you can have a mixture of Reasoning Engines cascade around a problem and then use an Operator type agent AI to function on the the results.

Brian Roemmele

398,807 views • 1 year ago

Astra is awesome, but if you're using it specifically to parse and extract a massive volume of documents, you will break the bank (10c+ per page on parsing and extraction!) We've added a native LlamaParse connector to ChatGPT that gives the model access to high-quality document OCR and extraction at the pareto frontier of accuracy and cost. It contains native access to our parse and extract endpoints. Upload your most complex documents, and you can do the following: * digitalize it into extremely accurate context for any QA task * perform large-scale extraction from a set of documents with accurate grounding and citations Offload the higher-level reasoning to Astra, leave specialized doc parsing and extraction to us. ChatGPT plugin: Come sign up here:

Astra is awesome, but if you're using it specifically to parse and extract a massive volume of documents, you will break the bank (10c+ per page on parsing and extraction!) We've added a native LlamaParse connector to ChatGPT that gives the model access to high-quality document OCR and extraction at the pareto frontier of accuracy and cost. It contains native access to our parse and extract endpoints. Upload your most complex documents, and you can do the following: * digitalize it into extremely accurate context for any QA task * perform large-scale extraction from a set of documents with accurate grounding and citations Offload the higher-level reasoning to Astra, leave specialized doc parsing and extraction to us. ChatGPT plugin: Come sign up here:

Jerry Liu

11,083 views • 28 days ago

New course: Agent Memory: Building Memory-Aware Agents, built in partnership with Oracle and taught by Richmond Alake and Nacho Martínez. Many agents work well within a single session but their memory resets once the session ends. Consider a research agent working on dozens of papers across multiple days: without memory, it has no way to store and retrieve what it learned across sessions. This short course teaches you to build a memory system that enables agents to persist memory and thereby learn across sessions. You'll design a Memory Manager that handles different memory types, implement semantic tool retrieval that scales without bloating the context, and build write-back pipelines that let your agent autonomously update and refine what it knows over time. Skills you'll gain: - Build persistent memory stores for different agent memory types - Implement a Memory Manager that orchestrates how your agent reads, writes, and retrieves memory - Treat tools as procedural memory and retrieve only relevant ones at inference time using semantic search Join and learn to build agents that remember and improve over time!

New course: Agent Memory: Building Memory-Aware Agents, built in partnership with Oracle and taught by Richmond Alake and Nacho Martínez. Many agents work well within a single session but their memory resets once the session ends. Consider a research agent working on dozens of papers across multiple days: without memory, it has no way to store and retrieve what it learned across sessions. This short course teaches you to build a memory system that enables agents to persist memory and thereby learn across sessions. You'll design a Memory Manager that handles different memory types, implement semantic tool retrieval that scales without bloating the context, and build write-back pipelines that let your agent autonomously update and refine what it knows over time. Skills you'll gain: - Build persistent memory stores for different agent memory types - Implement a Memory Manager that orchestrates how your agent reads, writes, and retrieves memory - Treat tools as procedural memory and retrieve only relevant ones at inference time using semantic search Join and learn to build agents that remember and improve over time!

Andrew Ng

162,196 views • 6 months ago

Introducing the new Box Agent. The Box Agent works across your entire Box file system, maintaining all your security and access controls, and is hyper tuned for working with enterprise content. This means you can now ask questions from all your enterprise content, search for files that were impossible to find before, deploy an agent on specific tasks on subsets of documents, analyze complex data sets, and generate or edit documents and spreadsheets via the agent. You can have the Box Agent search across your Box account to prepare for a sales meeting, analyze customer sentiment reports, process a large set of contracts for legal risk, provide insights into product development, leverage existing knowledge to answer RFPs, and thousands of other use-cases. 90% of enterprise data is unstructured data. This means most enterprise knowledge is sitting in inside of research reports, marketing assets, presentations, roadmap files, contracts, HR documents, and more. This is the critical context that agents need to be able to answer questions about a business, automate workflows, or serve up to other agents. We’ve been grinding on this for a quite a bit, and due to recent AI model advancements we’re now ready to release it to customers. Previous model generations had a difficult time knowing when to give up or keep going on a search, when to browse for files vs. use queries, how to rank files appropriately to know which version of content to use, how to handle large amounts of context to comb through, and more. Due to recent breakthroughs from models like GPT-5.4, Opus 4.6, and Gemini 3, we’ve seen major gains in tool calling, code execution, advanced reasoning, and more. Combined with an agent harness tuned to Box context, now it’s finally possible to have an agent that can work across your file system on long running tasks and actually deliver high quality results. Best of all, because the Box Agent works with any leading AI model, you’ll quickly get the gains coming out of the major labs as major new models are released. Further, openness at Box is key, so you’ll be able to call up the Box Agent from Box’s APIs and MCP server, so you can interact with Box intelligently from any other AI system. We know work happens everywhere, and we want to ensure you can access to the content you need from those places. The new Box Agent is available starting today, rolling out now for Enterprise Plus and Enterprise Advanced customers.

Introducing the new Box Agent. The Box Agent works across your entire Box file system, maintaining all your security and access controls, and is hyper tuned for working with enterprise content. This means you can now ask questions from all your enterprise content, search for files that were impossible to find before, deploy an agent on specific tasks on subsets of documents, analyze complex data sets, and generate or edit documents and spreadsheets via the agent. You can have the Box Agent search across your Box account to prepare for a sales meeting, analyze customer sentiment reports, process a large set of contracts for legal risk, provide insights into product development, leverage existing knowledge to answer RFPs, and thousands of other use-cases. 90% of enterprise data is unstructured data. This means most enterprise knowledge is sitting in inside of research reports, marketing assets, presentations, roadmap files, contracts, HR documents, and more. This is the critical context that agents need to be able to answer questions about a business, automate workflows, or serve up to other agents. We’ve been grinding on this for a quite a bit, and due to recent AI model advancements we’re now ready to release it to customers. Previous model generations had a difficult time knowing when to give up or keep going on a search, when to browse for files vs. use queries, how to rank files appropriately to know which version of content to use, how to handle large amounts of context to comb through, and more. Due to recent breakthroughs from models like GPT-5.4, Opus 4.6, and Gemini 3, we’ve seen major gains in tool calling, code execution, advanced reasoning, and more. Combined with an agent harness tuned to Box context, now it’s finally possible to have an agent that can work across your file system on long running tasks and actually deliver high quality results. Best of all, because the Box Agent works with any leading AI model, you’ll quickly get the gains coming out of the major labs as major new models are released. Further, openness at Box is key, so you’ll be able to call up the Box Agent from Box’s APIs and MCP server, so you can interact with Box intelligently from any other AI system. We know work happens everywhere, and we want to ensure you can access to the content you need from those places. The new Box Agent is available starting today, rolling out now for Enterprise Plus and Enterprise Advanced customers.

Aaron Levie

44,624 views • 6 months ago

Over 1 billion PDFs are created every day, but your agents still can’t read them reliably. Today we’re releasing Parse 2.0, the most accurate document parsing API in the world. Extend already processes millions of pages daily for leading AI teams like Brex, Mercury, Opendoor, Flatiron Health, and hundreds of others. Now, its even better. Parse 2.0 is SOTA quality on RealDoc-Bench, our open source benchmark that measures agent success rate on real world docs that agents actually encounter in production. We trained Parse 2.0 on 1M+ pages of the hardest documents seen in production. Here’s how it stacks up: - #1 in healthcare, real estate, logistics, and financial services - 95.7% agent Q&A accuracy on 581 docs (next best: 92%) - 0.847 F1 on layout (next best: 0.759) Give it a try today and build production-ready document agents with Extend.

Over 1 billion PDFs are created every day, but your agents still can’t read them reliably. Today we’re releasing Parse 2.0, the most accurate document parsing API in the world. Extend already processes millions of pages daily for leading AI teams like Brex, Mercury, Opendoor, Flatiron Health, and hundreds of others. Now, its even better. Parse 2.0 is SOTA quality on RealDoc-Bench, our open source benchmark that measures agent success rate on real world docs that agents actually encounter in production. We trained Parse 2.0 on 1M+ pages of the hardest documents seen in production. Here’s how it stacks up: - #1 in healthcare, real estate, logistics, and financial services - 95.7% agent Q&A accuracy on 581 docs (next best: 92%) - 0.847 F1 on layout (next best: 0.759) Give it a try today and build production-ready document agents with Extend.

Kushal Byatnal

589,119 views • 4 months ago

LiteParse Samples 📄🧑‍🏫 LiteParse is our free and fast document parser that can parse text with bounding boxes from any document. I’ve created a new repository that contains some demos on how you can make use of its outputs! ✅ Comparison against PyPDF and PyMuPDF ✅ Visual Citations: search for any keyword and see joined bounding boxes pop up in the source images It should be a default tool to help AI agents OCR any document type (supports PDF, Word, Powerpoint + dozens of other formats), before heavier-weight VLM-based document parsing solutions (like our own LlamaParse). LiteParse Samples repo: LiteParse:

LiteParse Samples 📄🧑‍🏫 LiteParse is our free and fast document parser that can parse text with bounding boxes from any document. I’ve created a new repository that contains some demos on how you can make use of its outputs! ✅ Comparison against PyPDF and PyMuPDF ✅ Visual Citations: search for any keyword and see joined bounding boxes pop up in the source images It should be a default tool to help AI agents OCR any document type (supports PDF, Word, Powerpoint + dozens of other formats), before heavier-weight VLM-based document parsing solutions (like our own LlamaParse). LiteParse Samples repo: LiteParse:

Jerry Liu

13,196 views • 6 months ago

The latest RAG trend for the current agent harnesses (Codex, Cowork) is to do two passes of document processing to solve a knowledge work task over a data room of documents: 1️⃣ A fast and light pass, oftentimes using a free/OSS doc parsing tool. This can be cheaply run across 10-100-1k’s of files, and enables the agent to then do retrieval (e.g. grep, semantic) to find relevant subsets of context. 2️⃣ A “just-in-time” VLM-based pass. Once the agent finds the relevant pages of context, it will screenshot the documents can call its own VLM (or write code) to dissect the pages. The issue with only using VLM-based OCR tools over massive ad-hoc customer file dumps is that it’s slow and expensive. Doing JIT VLM OCR allows the agent to filter through the data cheaply, but still preserve accuracy for the context that’s needed for the task. The agent harnesses do two-pass document processing by default using off the shelf-tools: pdf2text as the first pass, and using itself (Opus 5) as the second pass. See the below video where Cowork runs over a bunch of PDFs to answer a question about a benchmark graph in the Kimi k3 paper. The main issues here with the “out of the box” doc processing these agents offer are: * Opus 5 is not the best VLM for OCR. It is also way too expensive at scale and lacks grounding * The OSS tools like pypdf, pdf2text, may not be versatile enough as the first pass. * The agent will write a lot of throwaway code to rewrite things an OCR tool would’ve provided out of the box, like chart processing, bounding boxes, confidence scores, leading to increased cost and speed. We have all the tools within LlamaIndex 🦙 to help any agent do two-pass document processing with higher accuracy and lower cost. 1️⃣ We have liteparse for the first pass - a free/OSS parser written in Rust that’s faster/more accurate than other OSS parsers, and supports 50+ document types 2️⃣ We have LlamaParse for the second pass - an agentic document engine that uses VLMs+harnesses to achieve SOTA in accuracy and cost across various doc parsing and extraction tasks. It can be called from any agent harness as an MCP or skill. It takes in page numbers as input, so that the agent can choose to run LlamaParse over a subset of the doc instead of the full doc as a “zoom-in” pass. Come check it out! LiteParse: LlamaParse: All the relevant docs, including MCP, are here:

The latest RAG trend for the current agent harnesses (Codex, Cowork) is to do two passes of document processing to solve a knowledge work task over a data room of documents: 1️⃣ A fast and light pass, oftentimes using a free/OSS doc parsing tool. This can be cheaply run across 10-100-1k’s of files, and enables the agent to then do retrieval (e.g. grep, semantic) to find relevant subsets of context. 2️⃣ A “just-in-time” VLM-based pass. Once the agent finds the relevant pages of context, it will screenshot the documents can call its own VLM (or write code) to dissect the pages. The issue with only using VLM-based OCR tools over massive ad-hoc customer file dumps is that it’s slow and expensive. Doing JIT VLM OCR allows the agent to filter through the data cheaply, but still preserve accuracy for the context that’s needed for the task. The agent harnesses do two-pass document processing by default using off the shelf-tools: pdf2text as the first pass, and using itself (Opus 5) as the second pass. See the below video where Cowork runs over a bunch of PDFs to answer a question about a benchmark graph in the Kimi k3 paper. The main issues here with the “out of the box” doc processing these agents offer are: * Opus 5 is not the best VLM for OCR. It is also way too expensive at scale and lacks grounding * The OSS tools like pypdf, pdf2text, may not be versatile enough as the first pass. * The agent will write a lot of throwaway code to rewrite things an OCR tool would’ve provided out of the box, like chart processing, bounding boxes, confidence scores, leading to increased cost and speed. We have all the tools within LlamaIndex 🦙 to help any agent do two-pass document processing with higher accuracy and lower cost. 1️⃣ We have liteparse for the first pass - a free/OSS parser written in Rust that’s faster/more accurate than other OSS parsers, and supports 50+ document types 2️⃣ We have LlamaParse for the second pass - an agentic document engine that uses VLMs+harnesses to achieve SOTA in accuracy and cost across various doc parsing and extraction tasks. It can be called from any agent harness as an MCP or skill. It takes in page numbers as input, so that the agent can choose to run LlamaParse over a subset of the doc instead of the full doc as a “zoom-in” pass. Come check it out! LiteParse: LlamaParse: All the relevant docs, including MCP, are here:

Jerry Liu

23,119 views • 1 month ago

Introducing LiteParse - the best model-free document parsing tool for AI agents 💫 ✅ It’s completely open-source and free. ✅ No GPU required, will process ~500 pages in 2 seconds on commodity hardware ✅ More accurate than PyPDF, PyMuPDF, Markdown. Also way more readable - see below for how we parse tables!! ✅ Supports 50+ file formats, from PDFs to Office docs to images ✅ Is designed to plug and play with Claude Code, OpenClaw, and any other AI agent with a one-line skills install. Supports native screenshotting capabilities. We spent years building up LlamaParse by orchestrating state-of-the-art VLMs over the most complex documents. Along the way we realized that you could get quite far on most docs through fast and cheap text parsing. Take a look at the video below. For really complex tables within PDFs, we output them in a spatial grid that’s both AI and human-interpretable. Any other free/light parser light PyPDF will destroy the representation of this table and output a sequential list. This is not a replacement for a VLM-based OCR tool (it requires 0 GPUs and doesn’t use models), but it is shocking how good it is to parse most documents. Huge shoutout to Logan Markewich and Clelia Bertelli (🦀/acc) for all the work here. Come check it out: Repo:

Introducing LiteParse - the best model-free document parsing tool for AI agents 💫 ✅ It’s completely open-source and free. ✅ No GPU required, will process ~500 pages in 2 seconds on commodity hardware ✅ More accurate than PyPDF, PyMuPDF, Markdown. Also way more readable - see below for how we parse tables!! ✅ Supports 50+ file formats, from PDFs to Office docs to images ✅ Is designed to plug and play with Claude Code, OpenClaw, and any other AI agent with a one-line skills install. Supports native screenshotting capabilities. We spent years building up LlamaParse by orchestrating state-of-the-art VLMs over the most complex documents. Along the way we realized that you could get quite far on most docs through fast and cheap text parsing. Take a look at the video below. For really complex tables within PDFs, we output them in a spatial grid that’s both AI and human-interpretable. Any other free/light parser light PyPDF will destroy the representation of this table and output a sequential list. This is not a replacement for a VLM-based OCR tool (it requires 0 GPUs and doesn’t use models), but it is shocking how good it is to parse most documents. Huge shoutout to Logan Markewich and Clelia Bertelli (🦀/acc) for all the work here. Come check it out: Repo:

Jerry Liu

257,589 views • 6 months ago

The freedom to build with any agent framework you want is analogous to the freedom of religion, and this is a core value for the Virtuals society. Here are a few ways we are opening up support for every autonomous agentic framework out there: Today: - Enabling agent creators to specify which framework the agent runs on - Release of Terminal API, which allows agent builders using other agent frameworks to stream their thoughts and activities live on their agent pages. For a guide on how to use this, refer to: - Ability for agents running on any framework to publicly list their capabilities, a key enabler for the agent-to-agent marketplace Soon: - Integration into the agentic commerce standard and registry. Agents across frameworks being able to orchestrate and pay for tasks among themselves - ⁠Multi agent orchestration across frameworks. Imagine building an autonomous business with agents from different religions Welcome to the Virtuals society.

The freedom to build with any agent framework you want is analogous to the freedom of religion, and this is a core value for the Virtuals society. Here are a few ways we are opening up support for every autonomous agentic framework out there: Today: - Enabling agent creators to specify which framework the agent runs on - Release of Terminal API, which allows agent builders using other agent frameworks to stream their thoughts and activities live on their agent pages. For a guide on how to use this, refer to: - Ability for agents running on any framework to publicly list their capabilities, a key enabler for the agent-to-agent marketplace Soon: - Integration into the agentic commerce standard and registry. Agents across frameworks being able to orchestrate and pay for tasks among themselves - ⁠Multi agent orchestration across frameworks. Imagine building an autonomous business with agents from different religions Welcome to the Virtuals society.

Virtuals Protocol

216,191 views • 1 year ago

THIS GUY BUILT AN AUTONOMOUS AI AGENT OUT OF CLAUDE CODE + OBSIDIAN and this is way more interesting than another “use AI to take notes” demo the trick is simple: Obsidian is not the writing app here. it becomes the agent’s memory, task board, and context folder. Claude Code is not just answering prompts. it reads the vault, edits files, follows instructions, and keeps moving through the work like a junior operator with a filesystem. the reusable setup looks like this: 1. create an Obsidian vault for one project 2. keep goals, rules, tasks, decisions, and references as markdown files 3. point Claude Code at the folder 4. give it a clear operating loop: read context → choose next task → execute → write back what changed 5. use the notes as persistent memory instead of re-explaining the project every chat that’s the part people miss. the “agent” is not magic. it’s the boring combination of: - local files - explicit rules - task state - write access - a model that can run through the repo/vault Obsidian makes the memory human-readable. Claude Code makes the memory executable. that combo is why the video worked: it turns a notes app into an operating surface for actual work. best use cases: - content systems - research vaults - coding projects - client ops docs - personal knowledge bases that need actions, not just storage the caveat: if your vault is messy, your agent becomes messy too. folders, naming, “done” criteria, and forbidden actions matter more than the prompt. but once the structure is clean, this is one of the easiest ways to build an agent that remembers what happened yesterday without paying for a full custom app.

THIS GUY BUILT AN AUTONOMOUS AI AGENT OUT OF CLAUDE CODE + OBSIDIAN and this is way more interesting than another “use AI to take notes” demo the trick is simple: Obsidian is not the writing app here. it becomes the agent’s memory, task board, and context folder. Claude Code is not just answering prompts. it reads the vault, edits files, follows instructions, and keeps moving through the work like a junior operator with a filesystem. the reusable setup looks like this: 1. create an Obsidian vault for one project 2. keep goals, rules, tasks, decisions, and references as markdown files 3. point Claude Code at the folder 4. give it a clear operating loop: read context → choose next task → execute → write back what changed 5. use the notes as persistent memory instead of re-explaining the project every chat that’s the part people miss. the “agent” is not magic. it’s the boring combination of: - local files - explicit rules - task state - write access - a model that can run through the repo/vault Obsidian makes the memory human-readable. Claude Code makes the memory executable. that combo is why the video worked: it turns a notes app into an operating surface for actual work. best use cases: - content systems - research vaults - coding projects - client ops docs - personal knowledge bases that need actions, not just storage the caveat: if your vault is messy, your agent becomes messy too. folders, naming, “done” criteria, and forbidden actions matter more than the prompt. but once the structure is clean, this is one of the easiest ways to build an agent that remembers what happened yesterday without paying for a full custom app.

kocer

30,403 views • 3 months ago

I built a form-filling agent that anyone can use 💫 This is an extremely simple but useful (I hope) app. Upload a fillable form 📋, some context files, and chat with the agent to fill the form out automatically ✍️ 1️⃣ Yes it is a Claude Code SDK wrapper 2️⃣ It is better and faster than ChatGPT/Claude UI out of the box 3️⃣ We use LlamaParse to parse the context files, so you can have more trust that we are able to read context without hallucinations (e.g. messy scanned handwriting, drivers license photo, and more). This was one of my holiday Claude Code vibe-coding projects. Built with Opus 4.5, and also powered by Opus 4.5. Feeling the AGI 🫡 App is here: Repo is open-source:

I built a form-filling agent that anyone can use 💫 This is an extremely simple but useful (I hope) app. Upload a fillable form 📋, some context files, and chat with the agent to fill the form out automatically ✍️ 1️⃣ Yes it is a Claude Code SDK wrapper 2️⃣ It is better and faster than ChatGPT/Claude UI out of the box 3️⃣ We use LlamaParse to parse the context files, so you can have more trust that we are able to read context without hallucinations (e.g. messy scanned handwriting, drivers license photo, and more). This was one of my holiday Claude Code vibe-coding projects. Built with Opus 4.5, and also powered by Opus 4.5. Feeling the AGI 🫡 App is here: Repo is open-source:

Jerry Liu

48,466 views • 9 months ago

New course: Document AI: From OCR to Agentic Doc Extraction, built with LandingAI, where I'm executive chairman, and taught by David Park and Andrea Kropp. Much of the world's data is locked in PDFs, JPEGs, and other documents. This short course shows you how to build agentic workflows that process documents accurately: breaking them into parts, examining each piece carefully, and extracting information through multiple iterations. Traditional Optical Character Recognition (OCR) captures text but loses context from table headers, chart captions, or reading order of columns. After exploring OCR's limitations, you’ll use LandingAI's Agentic Document Extraction (ADE) framework to process documents. ADE treats pages as visually -- as images -- to parse information and extract fields. Skills you'll gain: - Build agents to convert unstructured files into structured Markdown/HTML and JSON - Use ADE to parse complex data like forms, handwriting, or equations - Map extracted information to named fields using a specified schema, with bounding boxes for grounding and validation - Deploy RAG applications with event-driven document processing Come learn about the best tools for processing documents like financial invoices, medical records, or academic papers intelligently:

New course: Document AI: From OCR to Agentic Doc Extraction, built with LandingAI, where I'm executive chairman, and taught by David Park and Andrea Kropp. Much of the world's data is locked in PDFs, JPEGs, and other documents. This short course shows you how to build agentic workflows that process documents accurately: breaking them into parts, examining each piece carefully, and extracting information through multiple iterations. Traditional Optical Character Recognition (OCR) captures text but loses context from table headers, chart captions, or reading order of columns. After exploring OCR's limitations, you’ll use LandingAI's Agentic Document Extraction (ADE) framework to process documents. ADE treats pages as visually -- as images -- to parse information and extract fields. Skills you'll gain: - Build agents to convert unstructured files into structured Markdown/HTML and JSON - Use ADE to parse complex data like forms, handwriting, or equations - Map extracted information to named fields using a specified schema, with bounding boxes for grounding and validation - Deploy RAG applications with event-driven document processing Come learn about the best tools for processing documents like financial invoices, medical records, or academic papers intelligently:

Andrew Ng

200,844 views • 8 months ago

Google just showed how AI engineers are moving from simple "RAG" to "graphs", memory and multimodal agents • 15:32 - setting up the production agent stack • 28:00 - turning disconnected data into a knowledge graph • 41:00 - Graph RAG with semantic + hybrid search • 58:00 - extracting graph context from images, text and video • 1:09:00 - orchestrating specialized agents with ADK • 1:20:00 - giving agents persistent memory across sessions 94-minute Google Cloud workshop, and it’s one of the clearest hands-on examples of the shift from -> RAG → Graph RAG → Memory → Multimodal Agent Graphs. The workshop uses Spanner Graph, Gemini, ADK and Memory Bank to build that stack. Watch it today, then read the full “From RAG to Context Graphs” roadmap below.

Google just showed how AI engineers are moving from simple "RAG" to "graphs", memory and multimodal agents • 15:32 - setting up the production agent stack • 28:00 - turning disconnected data into a knowledge graph • 41:00 - Graph RAG with semantic + hybrid search • 58:00 - extracting graph context from images, text and video • 1:09:00 - orchestrating specialized agents with ADK • 1:20:00 - giving agents persistent memory across sessions 94-minute Google Cloud workshop, and it’s one of the clearest hands-on examples of the shift from -> RAG → Graph RAG → Memory → Multimodal Agent Graphs. The workshop uses Spanner Graph, Gemini, ADK and Memory Bank to build that stack. Watch it today, then read the full “From RAG to Context Graphs” roadmap below.

Morlex

74,134 views • 1 month ago

Introducing NEO: The first Autonomous Machine Learning Engineer. It works like a full-stack ML engineer that never sleeps: handling data exploration, feature engineering, training, tuning, deployment, and monitoring, end to end. Powered by 11 specialized agents, NEO runs autonomously, saving ML engineers thousands of hours and making them 10x faster. Benchmarks: Tested on 75 Kaggle competitions, NEO scored a medal in 34.2% of them, significantly outperforming Microsoft’s RD Agent (22.4%) on OpenAI's MLE Bench. This sets a new state of the art for autonomous ML systems. NEO runs on a novel multi-agent orchestrator, powered by a multi-step reasoning engine, context transfer protocol, and agent memory, built to solve complex workflows end to end. And with human-in-the-loop mode, you can guide, inspect, and override any step. You're always in command. NEO is built for real world workflows and ready for production. NEO is here to make every ML engineer truly superhuman. Watch NEO in action:

Introducing NEO: The first Autonomous Machine Learning Engineer. It works like a full-stack ML engineer that never sleeps: handling data exploration, feature engineering, training, tuning, deployment, and monitoring, end to end. Powered by 11 specialized agents, NEO runs autonomously, saving ML engineers thousands of hours and making them 10x faster. Benchmarks: Tested on 75 Kaggle competitions, NEO scored a medal in 34.2% of them, significantly outperforming Microsoft’s RD Agent (22.4%) on OpenAI's MLE Bench. This sets a new state of the art for autonomous ML systems. NEO runs on a novel multi-agent orchestrator, powered by a multi-step reasoning engine, context transfer protocol, and agent memory, built to solve complex workflows end to end. And with human-in-the-loop mode, you can guide, inspect, and override any step. You're always in command. NEO is built for real world workflows and ready for production. NEO is here to make every ML engineer truly superhuman. Watch NEO in action:

Neo AI

861,524 views • 1 year ago

Introducing ExtractBench, the most comprehensive benchmark for information extraction from complex enterprise documents. The latest models are pushing the frontier of coding and knowledge work, but surprisingly they still struggle on complex doc extraction tasks in production. A well-tuned extractor must parse multi-page filings without dropping rows, emit exact spatial citations for auditability, and handle messy scans. Also they must do all of this at a viable per-page cost so that you can scale this to millions of docs in production (you can’t be paying upwards of $1 in tokens per page!) Existing extraction benchmarks fall short: they are not large/diverse enough in document domain (finance, energy, gov, auto), elements (long records, scans, grounding), and schemas. So our applied research team built ExtractBench. We evaluated 14 systems: frontier VLMs, coding agents, and specialized extraction APIs, against 370 enterprise documents: 4,869 pages, 67 document types. Our biggest finding 🧪: Short documents mask critical system flaws. On files past 50 pages, commercial VLMs collapse below 35% recall due to silent list truncation. They hold high precision, but lose output attention and drop most of the table rows. ExtractBench evaluates value accuracy, long-record completeness, spatial grounding, and per-page cost with zero LLM judges. It is 100% deterministic and reproducible. In tandem with ExtractBench, we’re also introducing 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗣𝗹𝘂𝘀, a new Extract tier in LlamaParse that debuts at #1 on the leaderboard: 95.6% value accuracy, at less than a third the cost of the closest peer. Explore the findings, download the dataset, or run the harness: Blog: GitHub: HuggingFace: We will be actively evolving both our extraction benchmark as well as our extraction harness over time. If you check out either ExtractBench or LlamaParse, let us know your feedback!

Introducing ExtractBench, the most comprehensive benchmark for information extraction from complex enterprise documents. The latest models are pushing the frontier of coding and knowledge work, but surprisingly they still struggle on complex doc extraction tasks in production. A well-tuned extractor must parse multi-page filings without dropping rows, emit exact spatial citations for auditability, and handle messy scans. Also they must do all of this at a viable per-page cost so that you can scale this to millions of docs in production (you can’t be paying upwards of $1 in tokens per page!) Existing extraction benchmarks fall short: they are not large/diverse enough in document domain (finance, energy, gov, auto), elements (long records, scans, grounding), and schemas. So our applied research team built ExtractBench. We evaluated 14 systems: frontier VLMs, coding agents, and specialized extraction APIs, against 370 enterprise documents: 4,869 pages, 67 document types. Our biggest finding 🧪: Short documents mask critical system flaws. On files past 50 pages, commercial VLMs collapse below 35% recall due to silent list truncation. They hold high precision, but lose output attention and drop most of the table rows. ExtractBench evaluates value accuracy, long-record completeness, spatial grounding, and per-page cost with zero LLM judges. It is 100% deterministic and reproducible. In tandem with ExtractBench, we’re also introducing 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗣𝗹𝘂𝘀, a new Extract tier in LlamaParse that debuts at #1 on the leaderboard: 95.6% value accuracy, at less than a third the cost of the closest peer. Explore the findings, download the dataset, or run the harness: Blog: GitHub: HuggingFace: We will be actively evolving both our extraction benchmark as well as our extraction harness over time. If you check out either ExtractBench or LlamaParse, let us know your feedback!

Jerry Liu

75,697 views • 1 month ago

LFG! Nous Research's Hermes Agent is live on the Umbrel App Store! It's an AI agent that gets smarter the longer it runs, remembers stuff with persistent memory across sessions, learns your workflow, and autonomously creates its own reusable skills from experience. And it's sandboxed like every Umbrel app so it can't touch anything it's not supposed to. 1-click install. Login with ChatGPT, use an OpenRouter/OpenAI/Anthropic API key, or point it to a local Ollama model running on the same Umbrel.

LFG! Nous Research's Hermes Agent is live on the Umbrel App Store! It's an AI agent that gets smarter the longer it runs, remembers stuff with persistent memory across sessions, learns your workflow, and autonomously creates its own reusable skills from experience. And it's sandboxed like every Umbrel app so it can't touch anything it's not supposed to. 1-click install. Login with ChatGPT, use an OpenRouter/OpenAI/Anthropic API key, or point it to a local Ollama model running on the same Umbrel.

Umbrel ☂️

29,882 views • 5 months ago

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

201,127 views • 1 year ago