Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Announcing: Agentic Document Extraction! PDF files represent information visually - via layout, charts, graphs, etc. - and are more than just text. Unlike traditional OCR and most PDF-to-text approaches, which focus on extracting the text, an agentic approach lets us break a document down into components and reason about... show more

Andrew Ng

1,931,387 subscribers

689,842 views • 1 year ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

11 Comments

Arnav Jaitly1 year ago

This is such a pain point for a lot of companies. Agentic information extraction will be such a productivity and efficiency boost

PDF GPT2 years ago

This is my favorite AI tool for reviewing reports. Just upload a report, ask for a summary, and get one in seconds. It's like ChatGPT, but built for documents. Try it for free.

Andrei Hasna1 year ago

Well done and well needed.

Vasilis K.1 year ago

@JackPNicholls

The AI Agent Architect1 year ago

Great use case. Using an agentic approach to this is kinda like getting the LLM to add metadata to what it finds in the PDF. So many thing you could do with that. If you want to see the opposite side of that coin - the creation of a structured Word document by an agent from only two simple commands, there will be a video demo showing this in my next article tomorrow.

🔮1 year ago

Love what you’re doing! For what it’s worth the URL in your video is wrong and hits a 404

Mr Rio1 year ago

this is what I need

RyanRejoice1 year ago

This is incredible and a game changer for handling and analyzing pdf data.

Clara Data1 year ago

Innovative spin on PDFs! Looking promising for future efficiency!

Huxley MindOS1 year ago

Mastering the PDF mystery: deciphering both scroll & scribe! Sounds innovative. Looking forward to the details.

Javier Modified1 year ago

Agentic Document Extraction? Finally, AI can stop pretending it understands charts. This is how you handle a PDF! 🚀

Related Videos

Agentic Document Extraction just got much faster! From previous 135sec median processing time down to 8sec. Extracts not just text but diagrams, charts, and form fields from PDFs to give LLM-ready output. Please see the video for details and some application ideas.

Agentic Document Extraction just got much faster! From previous 135sec median processing time down to 8sec. Extracts not just text but diagrams, charts, and form fields from PDFs to give LLM-ready output. Please see the video for details and some application ideas.

Andrew Ng

291,138 views • 1 year ago

Announcing a significant upgrade to Agentic Document Extraction! LandingAI's new DPT (Document Pre-trained Transformer) accurately extracts even from complex docs. For example, from large, complex tables, which is important for many finance and healthcare applications. And a new SDK makes using it require only 3 simple lines of code. Please see the video for technical details. I hope this unlocks a lot of value from the "dark data" currently stuck in PDF files, and that you'll build something cool with this!

Announcing a significant upgrade to Agentic Document Extraction! LandingAI's new DPT (Document Pre-trained Transformer) accurately extracts even from complex docs. For example, from large, complex tables, which is important for many finance and healthcare applications. And a new SDK makes using it require only 3 simple lines of code. Please see the video for technical details. I hope this unlocks a lot of value from the "dark data" currently stuck in PDF files, and that you'll build something cool with this!

Andrew Ng

299,866 views • 1 year ago

New course: Document AI: From OCR to Agentic Doc Extraction, built with LandingAI, where I'm executive chairman, and taught by David Park and Andrea Kropp. Much of the world's data is locked in PDFs, JPEGs, and other documents. This short course shows you how to build agentic workflows that process documents accurately: breaking them into parts, examining each piece carefully, and extracting information through multiple iterations. Traditional Optical Character Recognition (OCR) captures text but loses context from table headers, chart captions, or reading order of columns. After exploring OCR's limitations, you’ll use LandingAI's Agentic Document Extraction (ADE) framework to process documents. ADE treats pages as visually -- as images -- to parse information and extract fields. Skills you'll gain: - Build agents to convert unstructured files into structured Markdown/HTML and JSON - Use ADE to parse complex data like forms, handwriting, or equations - Map extracted information to named fields using a specified schema, with bounding boxes for grounding and validation - Deploy RAG applications with event-driven document processing Come learn about the best tools for processing documents like financial invoices, medical records, or academic papers intelligently:

New course: Document AI: From OCR to Agentic Doc Extraction, built with LandingAI, where I'm executive chairman, and taught by David Park and Andrea Kropp. Much of the world's data is locked in PDFs, JPEGs, and other documents. This short course shows you how to build agentic workflows that process documents accurately: breaking them into parts, examining each piece carefully, and extracting information through multiple iterations. Traditional Optical Character Recognition (OCR) captures text but loses context from table headers, chart captions, or reading order of columns. After exploring OCR's limitations, you’ll use LandingAI's Agentic Document Extraction (ADE) framework to process documents. ADE treats pages as visually -- as images -- to parse information and extract fields. Skills you'll gain: - Build agents to convert unstructured files into structured Markdown/HTML and JSON - Use ADE to parse complex data like forms, handwriting, or equations - Map extracted information to named fields using a specified schema, with bounding boxes for grounding and validation - Deploy RAG applications with event-driven document processing Come learn about the best tools for processing documents like financial invoices, medical records, or academic papers intelligently:

Andrew Ng

200,844 views • 9 months ago

Parse text from any PDF in seconds and give it to Claude Code 📑🤖 LiteParse is our open-source, model-free document parser that lets you digitalize text from any document in seconds. This is especially useful for coding agents, which are great at reading plaintext files but terrible at reading traditional document formats (PDF, Office docs). We have a one-line installable skill that lets you plug LiteParse into Claude Code and 40+ other agents. Repo is here:

Parse text from any PDF in seconds and give it to Claude Code 📑🤖 LiteParse is our open-source, model-free document parser that lets you digitalize text from any document in seconds. This is especially useful for coding agents, which are great at reading plaintext files but terrible at reading traditional document formats (PDF, Office docs). We have a one-line installable skill that lets you plug LiteParse into Claude Code and 40+ other agents. Repo is here:

Jerry Liu

30,668 views • 6 months ago

we've built the world's most advanced engine for document extraction over complex documents the video below shows an overview of LlamaExtract Agentic Plus in action. long tables, giant forms, calibrated confidence scores, and grounded extraction. if you have complex document extraction use cases and existing vendors aren't cutting it, let us know: come sign up to LlamaParse today:

we've built the world's most advanced engine for document extraction over complex documents the video below shows an overview of LlamaExtract Agentic Plus in action. long tables, giant forms, calibrated confidence scores, and grounded extraction. if you have complex document extraction use cases and existing vendors aren't cutting it, let us know: come sign up to LlamaParse today:

Jerry Liu

14,285 views • 24 days ago

Agentic Document Extraction now supports field extraction! Many doc extraction use cases extract specific fields from forms and other structured documents. You can now input a picture or PDF of an invoice, request the vendor name, item list, and prices, and get back the extracted fields. Or input a medical form and specify a schema to extract patient name, patient ID, insurance number, etc. One cool feature: If you don't feel like writing a schema (json specification of what fields to extract) yourself, upload one sample document and write a natural language prompt saying what you want, and we automatically generate a schema for you. See the video for details!

Agentic Document Extraction now supports field extraction! Many doc extraction use cases extract specific fields from forms and other structured documents. You can now input a picture or PDF of an invoice, request the vendor name, item list, and prices, and get back the extracted fields. Or input a medical form and specify a schema to extract patient name, patient ID, insurance number, etc. One cool feature: If you don't feel like writing a schema (json specification of what fields to extract) yourself, upload one sample document and write a natural language prompt saying what you want, and we automatically generate a schema for you. See the video for details!

Andrew Ng

193,139 views • 1 year ago

ColPali is changing the game for PDF retrieval by eliminating the need for OCR and chunking methods 🚀 Inspired by ColBERT’s success with text, ColPali splits an image of a document into patches, which are then processed through a vision LLM called PaliGemma. The embeddings for each patch retain contextual information, similarly to text embeddings in methods like ColBERT. During retrieval, user queries are embedded in the same space, and then compared to document patches using the MaxSim operator. ColPali recipe POC Weaviate AI Database: ColPali paper: As always, shoutout to the awesome Daniel Williams for the help! 💙

ColPali is changing the game for PDF retrieval by eliminating the need for OCR and chunking methods 🚀 Inspired by ColBERT’s success with text, ColPali splits an image of a document into patches, which are then processed through a vision LLM called PaliGemma. The embeddings for each patch retain contextual information, similarly to text embeddings in methods like ColBERT. During retrieval, user queries are embedded in the same space, and then compared to document patches using the MaxSim operator. ColPali recipe POC Weaviate AI Database: ColPali paper: As always, shoutout to the awesome Daniel Williams for the help! 💙

Victoria Slocum

67,862 views • 1 year ago

Today we’re introducing Extract v2.5 - a series of frontier agents tuned for document extraction. The agents (cost-effective, agentic, agentic plus) are tuned for value accuracy and grounding. Our extraction agents outperform Opus 5.5 and GPT-6 Sol while being 30%-4x cheaper. We’ve made massive improvements on complex extraction over * long lists (86.1% -> 95.5% on our agentic tier) * records spanning pages (85.5% -> 96.5%) * scanned forms (90.9% -> 95.7%) We’ve also launched the following features: ✅ Advanced citations: we locate bounding boxes for all supporting values for an inferred field, even if there's not an exact match. ✅ Structural Reasoning: we tailor document extraction algorithms depending on the type, layout, and information Our extraction agents are SOTA in price-performance on ExtractBench, across a wide range of cost points. We are the best tool for document extraction across documents of any complexity. Blog: All of these are available on LlamaParse: . Come check it out!

Today we’re introducing Extract v2.5 - a series of frontier agents tuned for document extraction. The agents (cost-effective, agentic, agentic plus) are tuned for value accuracy and grounding. Our extraction agents outperform Opus 5.5 and GPT-6 Sol while being 30%-4x cheaper. We’ve made massive improvements on complex extraction over * long lists (86.1% -> 95.5% on our agentic tier) * records spanning pages (85.5% -> 96.5%) * scanned forms (90.9% -> 95.7%) We’ve also launched the following features: ✅ Advanced citations: we locate bounding boxes for all supporting values for an inferred field, even if there's not an exact match. ✅ Structural Reasoning: we tailor document extraction algorithms depending on the type, layout, and information Our extraction agents are SOTA in price-performance on ExtractBench, across a wide range of cost points. We are the best tool for document extraction across documents of any complexity. Blog: All of these are available on LlamaParse: . Come check it out!

Jerry Liu

52,224 views • 16 days ago

Let's talk parsing charts 📊📈. Last week we released ParseBench, the first document OCR benchmark for AI agents. New in ParseBench: ChartDataPointMatch. Most document look at a chart and OCR the caption. Agents need the actual numbers. That's the gap between "OCR'd the text around the chart" and "actually read the chart." More about ParseBench, the GitHub code, Hugging Face dataset, and scientific paper→

Let's talk parsing charts 📊📈. Last week we released ParseBench, the first document OCR benchmark for AI agents. New in ParseBench: ChartDataPointMatch. Most document look at a chart and OCR the caption. Agents need the actual numbers. That's the gap between "OCR'd the text around the chart" and "actually read the chart." More about ParseBench, the GitHub code, Hugging Face dataset, and scientific paper→

LlamaIndex 🦙

13,987 views • 5 months ago

OCR can process characters but it doesn’t understand pixels. OCR has no way to reason about the headers, totals, or checkboxes found in tables, invoices, or forms. In our course with LandingAI, "Document AI: From OCR to Agentic Doc Extraction," we build agents to address these failure modes by breaking documents into pieces, applying the right tools, and mapping information to expected formats. Learn more and enroll today:

OCR can process characters but it doesn’t understand pixels. OCR has no way to reason about the headers, totals, or checkboxes found in tables, invoices, or forms. In our course with LandingAI, "Document AI: From OCR to Agentic Doc Extraction," we build agents to address these failure modes by breaking documents into pieces, applying the right tools, and mapping information to expected formats. Learn more and enroll today:

DeepLearning.AI

21,554 views • 8 months ago

Introducing Agentic Object Detection! Given a text prompt like “unripe strawberries” or “Kellogg’s branded cereal” and an image, we use an agentic workflow to reason at length and detect the specified objects. No need to label any training data. Watch the video for details.

Introducing Agentic Object Detection! Given a text prompt like “unripe strawberries” or “Kellogg’s branded cereal” and an image, we use an agentic workflow to reason at length and detect the specified objects. No need to label any training data. Watch the video for details.

Andrew Ng

398,323 views • 1 year ago

Most organizations run on unstructured content, but it’s the data locked within that content moves business forward. Box Extract is now generally available, delivering agentic data extraction at scale for smart process automation. Combining the latest AI models with advanced OCR capabilities, and agentic approaches that understand document structure and meaning, Box Extract automatically and accurately extracts high quality data from a variety of content to automate workflows, speed content discovery, and drive smarter business decisions.

Most organizations run on unstructured content, but it’s the data locked within that content moves business forward. Box Extract is now generally available, delivering agentic data extraction at scale for smart process automation. Combining the latest AI models with advanced OCR capabilities, and agentic approaches that understand document structure and meaning, Box Extract automatically and accurately extracts high quality data from a variety of content to automate workflows, speed content discovery, and drive smarter business decisions.

Box

759,033 views • 9 months ago

TikTok user reportedly discover a way to read through redacted text in the Epstein files. The user explained that in certain documents, text was only visually covered in black and could be revealed by copying and pasting it into a blank document. “If you simply copy paste some of them into a blank doc the original text appears!” [🎥: evangeleeeen/TT ]

TikTok user reportedly discover a way to read through redacted text in the Epstein files. The user explained that in certain documents, text was only visually covered in black and could be revealed by copying and pasting it into a blank document. “If you simply copy paste some of them into a blank doc the original text appears!” [🎥: evangeleeeen/TT ]

Complex

491,588 views • 9 months ago

OpenRead is now open for PDF uploading and parsing, check it out! PDF upload parsing is one of the most desired features of our users, and now we can finally bring this futuristic technology to life. We have adopted the most advanced #NLP, #CV, #OCR technology, which can extract the text, formula, table, and figure in PDF to the greatest extent, and re-typeset these elements.

OpenRead is now open for PDF uploading and parsing, check it out! PDF upload parsing is one of the most desired features of our users, and now we can finally bring this futuristic technology to life. We have adopted the most advanced #NLP, #CV, #OCR technology, which can extract the text, formula, table, and figure in PDF to the greatest extent, and re-typeset these elements.

OpenRead

26,656 views • 3 years ago

The web was never meant to be flattened into text. Yet most web RAG systems start by parsing HTML --- a complex and lossy process. 🔥 Introducing PixelRAG: the first RAG system that retrieves and reads 30M+ web pages as pixels. Instead of extracting text, PixelRAG retrieves screenshots and lets a VLM read them directly. PixelRAG not only preserves visual information, but also outperforms text-based RAG on text-only QA benchmarks by +18.1%. Why? (1) HTML-to-text conversion often discards layout, structure, tables, and other useful signals. (2) We continued pretraining a VLM on web page screenshots and turned it into a surprisingly strong visual retriever. (3) Recent VLMs are remarkably good at understanding web pages, often with better accuracy and token efficiency than text-only pipelines. Takeaway: HTML parsing may be one of the biggest self-inflicted bottlenecks in web RAG. Demo below 👇 Code: Paper: Playground:

The web was never meant to be flattened into text. Yet most web RAG systems start by parsing HTML --- a complex and lossy process. 🔥 Introducing PixelRAG: the first RAG system that retrieves and reads 30M+ web pages as pixels. Instead of extracting text, PixelRAG retrieves screenshots and lets a VLM read them directly. PixelRAG not only preserves visual information, but also outperforms text-based RAG on text-only QA benchmarks by +18.1%. Why? (1) HTML-to-text conversion often discards layout, structure, tables, and other useful signals. (2) We continued pretraining a VLM on web page screenshots and turned it into a surprisingly strong visual retriever. (3) Recent VLMs are remarkably good at understanding web pages, often with better accuracy and token efficiency than text-only pipelines. Takeaway: HTML parsing may be one of the biggest self-inflicted bottlenecks in web RAG. Demo below 👇 Code: Paper: Playground:

Yichuan Wang

92,139 views • 4 months ago

We’re excited to officially launch LlamaParse, the first genAI-native document parsing solution. Not only is it better at parsing out images/tables/charts 📊📈 than virtually every other parser, it is now steerable through natural language instructions - output the document in whatever format you desire! It is also the only parsing solution that seamlessly allows you to build accurate RAG over complex documents, free of hallucinations 🔥 We launched it in private preview a few weeks ago and hit 2k users, 1M total PDF pages parsed. And now it’s better than ever. LlamaParse contains the following killer features: ✅ SOTA table/chart extraction ✅ Seamless integration with LlamaIndex 🦙 advanced RAG/agents ✅✨ Natural language Parsing Instructions ✅✨JSON mode and image extraction ✅✨Support for ~10 document types (.pdf, .pptx, .docx, .xml) and more Our pricing is simple: 1k free per day, and additional pages at 0.3c a page, or $3 for 1k pages. If you want advanced document RAG and/or private deployments, come get in touch with us to chat about LlamaCloud. Check out our full blog post here: LlamaParse client repo: Signup at 🦙☁️: Come talk to us:

We’re excited to officially launch LlamaParse, the first genAI-native document parsing solution. Not only is it better at parsing out images/tables/charts 📊📈 than virtually every other parser, it is now steerable through natural language instructions - output the document in whatever format you desire! It is also the only parsing solution that seamlessly allows you to build accurate RAG over complex documents, free of hallucinations 🔥 We launched it in private preview a few weeks ago and hit 2k users, 1M total PDF pages parsed. And now it’s better than ever. LlamaParse contains the following killer features: ✅ SOTA table/chart extraction ✅ Seamless integration with LlamaIndex 🦙 advanced RAG/agents ✅✨ Natural language Parsing Instructions ✅✨JSON mode and image extraction ✅✨Support for ~10 document types (.pdf, .pptx, .docx, .xml) and more Our pricing is simple: 1k free per day, and additional pages at 0.3c a page, or $3 for 1k pages. If you want advanced document RAG and/or private deployments, come get in touch with us to chat about LlamaCloud. Check out our full blog post here: LlamaParse client repo: Signup at 🦙☁️: Come talk to us:

LlamaIndex 🦙

143,198 views • 2 years ago

We tuned an AI agent that can do large-scale document extraction from long docs (50+ pages, some with 10k-100k fields) with 94%+ accuracy 📈 It uses a harness + model set that is tuned specifically for reasoning over extracting out complex information from complex docs. Each extracted field comes with a confidence score as well as a bounding box denoting where it came from. It does 10-20% better in accuracy than generalized coding agent harnesses (e.g. Claude Code Opus 4.8 and Codex GPT-5.6). Check out the video below for a demo. The mode is called LlamaExtract Agentic Plus. Learn more about our extraction benchmark and agentic plus mode here: If you have very complex document extraction needs, come check out LlamaParse!

We tuned an AI agent that can do large-scale document extraction from long docs (50+ pages, some with 10k-100k fields) with 94%+ accuracy 📈 It uses a harness + model set that is tuned specifically for reasoning over extracting out complex information from complex docs. Each extracted field comes with a confidence score as well as a bounding box denoting where it came from. It does 10-20% better in accuracy than generalized coding agent harnesses (e.g. Claude Code Opus 4.8 and Codex GPT-5.6). Check out the video below for a demo. The mode is called LlamaExtract Agentic Plus. Learn more about our extraction benchmark and agentic plus mode here: If you have very complex document extraction needs, come check out LlamaParse!

Jerry Liu

18,359 views • 2 months ago

Andrew Ng: "I coined Agentic AI, and I did not expect it to go parabolic. Loops and Graphs are next." 100% of his tasks are already handled by AI agents. He uses Loops for self-improvement, and Graphs for memory. In a 20-minute talk, Andrew reveals what the future of Agentic AI looks like. Worth more than a $500 agentic course on the internet. Watch it, then read the full guide on Loops and Graphs below.

Andrew Ng: "I coined Agentic AI, and I did not expect it to go parabolic. Loops and Graphs are next." 100% of his tasks are already handled by AI agents. He uses Loops for self-improvement, and Graphs for memory. In a 20-minute talk, Andrew reveals what the future of Agentic AI looks like. Worth more than a $500 agentic course on the internet. Watch it, then read the full guide on Loops and Graphs below.

Mahax

13,469 views • 2 months ago

Astra is awesome, but if you're using it specifically to parse and extract a massive volume of documents, you will break the bank (10c+ per page on parsing and extraction!) We've added a native LlamaParse connector to ChatGPT that gives the model access to high-quality document OCR and extraction at the pareto frontier of accuracy and cost. It contains native access to our parse and extract endpoints. Upload your most complex documents, and you can do the following: * digitalize it into extremely accurate context for any QA task * perform large-scale extraction from a set of documents with accurate grounding and citations Offload the higher-level reasoning to Astra, leave specialized doc parsing and extraction to us. ChatGPT plugin: Come sign up here:

Astra is awesome, but if you're using it specifically to parse and extract a massive volume of documents, you will break the bank (10c+ per page on parsing and extraction!) We've added a native LlamaParse connector to ChatGPT that gives the model access to high-quality document OCR and extraction at the pareto frontier of accuracy and cost. It contains native access to our parse and extract endpoints. Upload your most complex documents, and you can do the following: * digitalize it into extremely accurate context for any QA task * perform large-scale extraction from a set of documents with accurate grounding and citations Offload the higher-level reasoning to Astra, leave specialized doc parsing and extraction to us. ChatGPT plugin: Come sign up here:

Jerry Liu

11,083 views • 1 month ago