Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Announcing a significant upgrade to Agentic Document Extraction! LandingAI's new DPT (Document Pre-trained Transformer) accurately extracts even from complex docs. For example, from large, complex tables, which is important for many finance and healthcare applications. And a new SDK makes using it require only 3 simple lines of code.... show more

Andrew Ng

1,900,220 subscribers

299,778 Aufrufe • vor 1 Jahr •via X (Twitter)

Gesundheit & Wellness Wissenschaft & Technologie Finanzen

Anya Rossi• Live Now

Private livecam show

33 Kommentare

Profilbild von nbs

nbsvor 1 Jahr

Thought it was open source when starting to read; no company would give its internal documents to an external API, especially in a regulated and confidential domain.

Profilbild von Ruhan Ponnada

Ruhan Ponnadavor 1 Jahr

peep the docs:

Profilbild von ⟁ndrew V

⟁ndrew Vvor 1 Jahr

Andrew I’ve tried every tool over the past year or more in fact it’s been way more. Nothing has been able to accomplish this without basically pre training or doing a ton of work on schema and pydantic models. It shouldn’t be hard so I hope this course or process is the unlock I’ve been hoping for.

Profilbild von Alok Mani Tripathi

Alok Mani Tripathivor 1 Jahr

For simple documents like invoices you may try will try to include this model also for complex documents.

Profilbild von Royce

Roycevor 1 Jahr

@chrismattmann was doing this a decade ago with Apache Tika!

Profilbild von CRISPRKing

CRISPRKingvor 1 Jahr

"Dark data" trapped in PDFs is the last frontier of untapped business value. When extraction goes from manual data entry to 3 lines of code, entire industries built on document processing become optional. Finance and healthcare just got their structured data problem solved.

Profilbild von Vinod Kumar

Vinod Kumarvor 1 Jahr

With multi-model LLMs I was able to extract the data from tables, but this looks more accurate I'll try it today👀

Profilbild von Ekhlaque Bari

Ekhlaque Barivor 1 Jahr

“Dark data” really is the right phrase so much knowledge is trapped in PDFs and scans. Unlocking it feels less like an upgrade and more like resurfacing buried intelligence.

Profilbild von Mingtian

Mingtianvor 1 Jahr

Why not just start to use genetic vectorless long pdf chat mcp:

Profilbild von Manuel

Manuelvor 1 Jahr

Really nice tool. I'm very curious about how it performs with edge cases where tables get split into two or more pages, such as when the table is too large for just one page.

Profilbild von Chris

Chrisvor 1 Jahr

Any commitment to accuracy SLAs? Is there a reference architecture available?

Profilbild von Chris

Chrisvor 1 Jahr

What are the token limits?

Profilbild von Little Berg

Little Bergvor 1 Jahr

!submit @curatedotfun #ai

Profilbild von Eric

Ericvor 1 Jahr

DPT: Turning PDFs into gold like a data magician!

Profilbild von Oliver Bowman

Oliver Bowmanvor 1 Jahr

This looks insanely useful. Finally a tool that can actually handle messy PDFs without hours of manual work. 3 lines of code to unlock dark data? Yes please

Profilbild von Sachin Anand

Sachin Anandvor 1 Jahr

How does it differ from LlamaParse ?

Profilbild von OQTACORE

OQTACOREvor 1 Jahr

What are the key improvements in DPT’s architecture that enable such accuracy on complex layouts?

Profilbild von LLM

LLMvor 1 Jahr

Game changer for unlocking PDF dark data!

Profilbild von Rui Diao

Rui Diaovor 1 Jahr

Mastering complex tables in documents is a huge leap for enterprise AI applications. The three-line SDK integration sounds incredibly accessible—looking forward to seeing the practical implementations unlocked by DPT! 🚀

Profilbild von true

truevor 1 Jahr

Anything from Andrew is gold.

Profilbild von JulianSong

JulianSongvor 1 Jahr

@grok, how do I get to download this for free?

Profilbild von LLM

LLMvor 1 Jahr

This upgrade sounds game-changing for handling complex documents.

Profilbild von Sanjiv

Sanjivvor 1 Jahr

@grok How doors this compare to Azure Document Intelligence?

Profilbild von Centropy

Centropyvor 1 Jahr

The concept seems nice, but if it's not open source and the option of a local self-hosted version is available iam not going to join that ride.

Profilbild von VibeEdge

VibeEdgevor 1 Jahr

@AndrewYNg Impressive upgrade DPT's hallucination-free extraction could revolutionize audit trails in finance. Check out the SDK docs for quick integration!

Profilbild von PAI3

PAI3vor 1 Jahr

Looking forward

Profilbild von Devin AI

Devin AIvor 1 Jahr

The “3-line SDK” promise is huge. Making advanced document extraction accessible is just as important as the model itself.

Profilbild von MarketFlux.ai

MarketFlux.aivor 1 Jahr

yes, an AI commodity even today

Profilbild von amogus

amogusvor 1 Jahr

is this free?

Profilbild von حمید

حمیدvor 1 Jahr

Unfortunately it isn't so accurate for Persian Numbers

Profilbild von Firdosh Tangri

Firdosh Tangrivor 10 Monaten

Document extraction from complex tables has been a longstanding pain point in enterprise AI adoption. The key innovation here isn't just OCR accuracy, but understanding document structure and relationships - distinguishing headers from data, handling merged cells, and preserving semantic meaning across rows and columns. The 3-line SDK abstraction is crucial for democratizing access. Most finance and healthcare teams don't have ML expertise to wrangle traditional extraction pipelines. The real test will be how well DPT generalizes to unusual table formats and handwritten annotations that plague legacy documents in regulated industries.

Profilbild von Raymond Z

Raymond Zvor 1 Jahr

I've been troubled by these kinds of issues for long time, finally.

Profilbild von Dr. Mohammed Lubbad | د. محمد لبد

Dr. Mohammed Lubbad | د. محمد لبدvor 1 Jahr

Transforming document extraction opens new avenues in finance and healthcare. How can we harness this power strategically? 🚀 #Innovation

Ähnliche Videos

Announcing: Agentic Document Extraction! PDF files represent information visually - via layout, charts, graphs, etc. - and are more than just text. Unlike traditional OCR and most PDF-to-text approaches, which focus on extracting the text, an agentic approach lets us break a document down into components and reason about them, resulting in more accurate extraction of the underlying meaning for RAG and other applications. Watch the video for details.

Announcing: Agentic Document Extraction! PDF files represent information visually - via layout, charts, graphs, etc. - and are more than just text. Unlike traditional OCR and most PDF-to-text approaches, which focus on extracting the text, an agentic approach lets us break a document down into components and reason about them, resulting in more accurate extraction of the underlying meaning for RAG and other applications. Watch the video for details.

Andrew Ng

689,741 Aufrufe • vor 1 Jahr

We tuned an AI agent that can do large-scale document extraction from long docs (50+ pages, some with 10k-100k fields) with 94%+ accuracy 📈 It uses a harness + model set that is tuned specifically for reasoning over extracting out complex information from complex docs. Each extracted field comes with a confidence score as well as a bounding box denoting where it came from. It does 10-20% better in accuracy than generalized coding agent harnesses (e.g. Claude Code Opus 4.8 and Codex GPT-5.6). Check out the video below for a demo. The mode is called LlamaExtract Agentic Plus. Learn more about our extraction benchmark and agentic plus mode here: If you have very complex document extraction needs, come check out LlamaParse!

We tuned an AI agent that can do large-scale document extraction from long docs (50+ pages, some with 10k-100k fields) with 94%+ accuracy 📈 It uses a harness + model set that is tuned specifically for reasoning over extracting out complex information from complex docs. Each extracted field comes with a confidence score as well as a bounding box denoting where it came from. It does 10-20% better in accuracy than generalized coding agent harnesses (e.g. Claude Code Opus 4.8 and Codex GPT-5.6). Check out the video below for a demo. The mode is called LlamaExtract Agentic Plus. Learn more about our extraction benchmark and agentic plus mode here: If you have very complex document extraction needs, come check out LlamaParse!

Jerry Liu

18,359 Aufrufe • vor 1 Monat

New course: Document AI: From OCR to Agentic Doc Extraction, built with LandingAI, where I'm executive chairman, and taught by David Park and Andrea Kropp. Much of the world's data is locked in PDFs, JPEGs, and other documents. This short course shows you how to build agentic workflows that process documents accurately: breaking them into parts, examining each piece carefully, and extracting information through multiple iterations. Traditional Optical Character Recognition (OCR) captures text but loses context from table headers, chart captions, or reading order of columns. After exploring OCR's limitations, you’ll use LandingAI's Agentic Document Extraction (ADE) framework to process documents. ADE treats pages as visually -- as images -- to parse information and extract fields. Skills you'll gain: - Build agents to convert unstructured files into structured Markdown/HTML and JSON - Use ADE to parse complex data like forms, handwriting, or equations - Map extracted information to named fields using a specified schema, with bounding boxes for grounding and validation - Deploy RAG applications with event-driven document processing Come learn about the best tools for processing documents like financial invoices, medical records, or academic papers intelligently:

New course: Document AI: From OCR to Agentic Doc Extraction, built with LandingAI, where I'm executive chairman, and taught by David Park and Andrea Kropp. Much of the world's data is locked in PDFs, JPEGs, and other documents. This short course shows you how to build agentic workflows that process documents accurately: breaking them into parts, examining each piece carefully, and extracting information through multiple iterations. Traditional Optical Character Recognition (OCR) captures text but loses context from table headers, chart captions, or reading order of columns. After exploring OCR's limitations, you’ll use LandingAI's Agentic Document Extraction (ADE) framework to process documents. ADE treats pages as visually -- as images -- to parse information and extract fields. Skills you'll gain: - Build agents to convert unstructured files into structured Markdown/HTML and JSON - Use ADE to parse complex data like forms, handwriting, or equations - Map extracted information to named fields using a specified schema, with bounding boxes for grounding and validation - Deploy RAG applications with event-driven document processing Come learn about the best tools for processing documents like financial invoices, medical records, or academic papers intelligently:

Andrew Ng

200,844 Aufrufe • vor 8 Monaten

Parse text from any PDF in seconds and give it to Claude Code 📑🤖 LiteParse is our open-source, model-free document parser that lets you digitalize text from any document in seconds. This is especially useful for coding agents, which are great at reading plaintext files but terrible at reading traditional document formats (PDF, Office docs). We have a one-line installable skill that lets you plug LiteParse into Claude Code and 40+ other agents. Repo is here:

Parse text from any PDF in seconds and give it to Claude Code 📑🤖 LiteParse is our open-source, model-free document parser that lets you digitalize text from any document in seconds. This is especially useful for coding agents, which are great at reading plaintext files but terrible at reading traditional document formats (PDF, Office docs). We have a one-line installable skill that lets you plug LiteParse into Claude Code and 40+ other agents. Repo is here:

Jerry Liu

30,668 Aufrufe • vor 6 Monaten

Data preprocessing is critical for building effective RAG systems. Our new short course, Preprocessing Unstructured Data for LLM Applications, taught by Matt Robinson of Unstructured, demonstrates important but sometimes overlooked aspects of RAG systems: - How to extract and normalize content from diverse formats like PDF, Powerpoint, and HTML to expand your LLM's knowledge - Enriching data with metadata to enable more powerful retrieval and reasoning - Applying document layout analysis and vision transforms to process embedded images and tables Then you’ll use all these skills and build a RAG bot that draws from a corpus that includes PDF, PowerPoint, and Markdown documents. Please sign up here:

Data preprocessing is critical for building effective RAG systems. Our new short course, Preprocessing Unstructured Data for LLM Applications, taught by Matt Robinson of Unstructured, demonstrates important but sometimes overlooked aspects of RAG systems: - How to extract and normalize content from diverse formats like PDF, Powerpoint, and HTML to expand your LLM's knowledge - Enriching data with metadata to enable more powerful retrieval and reasoning - Applying document layout analysis and vision transforms to process embedded images and tables Then you’ll use all these skills and build a RAG bot that draws from a corpus that includes PDF, PowerPoint, and Markdown documents. Please sign up here:

Andrew Ng

150,391 Aufrufe • vor 2 Jahren

LangChain: Chat with Your Data, a new free short course created with Harrison Chase, is now available! In this 1 hour course, you’ll learn how to build one of the most requested LLM-based applications: Answering questions using information from a document or collection of documents (often called Retrieval Augmented Generation). You'll also learn how to use vector stores and embeddings to retrieve document chunks relevant to a query. I hope you enjoy the course!

LangChain: Chat with Your Data, a new free short course created with Harrison Chase, is now available! In this 1 hour course, you’ll learn how to build one of the most requested LLM-based applications: Answering questions using information from a document or collection of documents (often called Retrieval Augmented Generation). You'll also learn how to use vector stores and embeddings to retrieve document chunks relevant to a query. I hope you enjoy the course!

Andrew Ng

384,524 Aufrufe • vor 3 Jahren

Agentic Document Extraction now supports field extraction! Many doc extraction use cases extract specific fields from forms and other structured documents. You can now input a picture or PDF of an invoice, request the vendor name, item list, and prices, and get back the extracted fields. Or input a medical form and specify a schema to extract patient name, patient ID, insurance number, etc. One cool feature: If you don't feel like writing a schema (json specification of what fields to extract) yourself, upload one sample document and write a natural language prompt saying what you want, and we automatically generate a schema for you. See the video for details!

Agentic Document Extraction now supports field extraction! Many doc extraction use cases extract specific fields from forms and other structured documents. You can now input a picture or PDF of an invoice, request the vendor name, item list, and prices, and get back the extracted fields. Or input a medical form and specify a schema to extract patient name, patient ID, insurance number, etc. One cool feature: If you don't feel like writing a schema (json specification of what fields to extract) yourself, upload one sample document and write a natural language prompt saying what you want, and we automatically generate a schema for you. See the video for details!

Andrew Ng

193,139 Aufrufe • vor 1 Jahr

Most organizations run on unstructured content, but it’s the data locked within that content moves business forward. Box Extract is now generally available, delivering agentic data extraction at scale for smart process automation. Combining the latest AI models with advanced OCR capabilities, and agentic approaches that understand document structure and meaning, Box Extract automatically and accurately extracts high quality data from a variety of content to automate workflows, speed content discovery, and drive smarter business decisions.

Most organizations run on unstructured content, but it’s the data locked within that content moves business forward. Box Extract is now generally available, delivering agentic data extraction at scale for smart process automation. Combining the latest AI models with advanced OCR capabilities, and agentic approaches that understand document structure and meaning, Box Extract automatically and accurately extracts high quality data from a variety of content to automate workflows, speed content discovery, and drive smarter business decisions.

Box

759,033 Aufrufe • vor 8 Monaten

Reducto CLI is the best way for agents to interact with document data in any workflow, and I’m really excited to see the range of use cases that are possible with this. It uses our frontier models for anything from parsing to editing, and can help automate full workflows like building spreadsheets or reports with really ugly cases. Raunak’s demo is a great example of what this unlocks

Reducto CLI is the best way for agents to interact with document data in any workflow, and I’m really excited to see the range of use cases that are possible with this. It uses our frontier models for anything from parsing to editing, and can help automate full workflows like building spreadsheets or reports with really ugly cases. Raunak’s demo is a great example of what this unlocks

Adit

16,957 Aufrufe • vor 9 Monaten

Announcing my new course: Agentic AI! Building AI agents is one of the most in-demand skills in the job market. This course, available now at teaches you how. You'll learn to implement four key agentic design patterns: - Reflection, in which an agent examines its own output and figures out how to improve it - Tool use, in which an LLM-driven application decides which functions to call to carry out web search, access calendars, send email, write code, etc. - Planning, where you'll use an LLM to decide how to break down a task into sub-tasks for execution, and - Multi-agent collaboration, in which you build multiple specialized agents — much like how a company might hire multiple employees — to perform a complex task You'll also learn to take a complex application and systematically decompose it into a sequence of tasks to implement using these design patterns. But here's what I think is the most important part of this course: Having worked with many teams on AI agents, I've found that the single biggest predictor of whether someone executes well is their ability to drive a disciplined process for evals and error analysis. In this course, you'll learn how to do this, so you can efficiently home in on which components to improve in a complex agentic workflow. Instead of guessing what to work on, you'll let evals data guide you. This will put you significantly ahead of the game compared to the vast majority of teams building agents. Together, we'll build a deep research agent that searches, synthesizes, and reports, using all of these agentic design patterns and best practices. This self-paced course is taught in a vendor neutral way, using raw Python - without hiding details in a framework. You'll see how each step works, and learn the core concepts that you can then implement using any popular agentic AI framework, or using no framework. The only prerequisite is familiarity with Python, though knowing a bit about LLMs helps. Come join me, and let's build some agentic AI systems! Sign up to get started:

Announcing my new course: Agentic AI! Building AI agents is one of the most in-demand skills in the job market. This course, available now at teaches you how. You'll learn to implement four key agentic design patterns: - Reflection, in which an agent examines its own output and figures out how to improve it - Tool use, in which an LLM-driven application decides which functions to call to carry out web search, access calendars, send email, write code, etc. - Planning, where you'll use an LLM to decide how to break down a task into sub-tasks for execution, and - Multi-agent collaboration, in which you build multiple specialized agents — much like how a company might hire multiple employees — to perform a complex task You'll also learn to take a complex application and systematically decompose it into a sequence of tasks to implement using these design patterns. But here's what I think is the most important part of this course: Having worked with many teams on AI agents, I've found that the single biggest predictor of whether someone executes well is their ability to drive a disciplined process for evals and error analysis. In this course, you'll learn how to do this, so you can efficiently home in on which components to improve in a complex agentic workflow. Instead of guessing what to work on, you'll let evals data guide you. This will put you significantly ahead of the game compared to the vast majority of teams building agents. Together, we'll build a deep research agent that searches, synthesizes, and reports, using all of these agentic design patterns and best practices. This self-paced course is taught in a vendor neutral way, using raw Python - without hiding details in a framework. You'll see how each step works, and learn the core concepts that you can then implement using any popular agentic AI framework, or using no framework. The only prerequisite is familiarity with Python, though knowing a bit about LLMs helps. Come join me, and let's build some agentic AI systems! Sign up to get started:

Andrew Ng

890,616 Aufrufe • vor 11 Monaten

Parsing a document accurately is one thing. Proving where every value came from is another. When a compliance team reviews an AI extraction, or an auditor needs to sign off on a figure pulled from a financial filing, "it came from this document" isn't enough. They need to see exactly where. The specific cell in the table, the exact line on the page, the precise word the agent used. Most parsers can get you to a paragraph or a table block. That's where the trail ends. Today we're shipping Granular Bounding Boxes in LlamaParse — word, line, and cell level coordinates for every value in your document. The result is a complete, verifiable trail from every extracted value back to its exact source in the document. Built for audit workflows, compliance review, and any pipeline where verification isn't optional. Read the full announcement →

Parsing a document accurately is one thing. Proving where every value came from is another. When a compliance team reviews an AI extraction, or an auditor needs to sign off on a figure pulled from a financial filing, "it came from this document" isn't enough. They need to see exactly where. The specific cell in the table, the exact line on the page, the precise word the agent used. Most parsers can get you to a paragraph or a table block. That's where the trail ends. Today we're shipping Granular Bounding Boxes in LlamaParse — word, line, and cell level coordinates for every value in your document. The result is a complete, verifiable trail from every extracted value back to its exact source in the document. Built for audit workflows, compliance review, and any pipeline where verification isn't optional. Read the full announcement →

LlamaIndex 🦙

29,090 Aufrufe • vor 3 Monaten

I had my interview at a Media startup from Seoul as a Backend Engineer This was asked from me to build during the interview call and not to use any AI code editor .. I had to rely on docs and other people's code from stack overflow Its a replica of multi gigabyte video upload component from their core infra which I had to build and explain ... > has a grpc api for chunked exports > using redis to track the chunk assembly state like which chunks are received and expiration of chinks > and a simple html client to show the pause and resume of chunks for uploading Built with go, grpc, postgresql and redis

I had my interview at a Media startup from Seoul as a Backend Engineer This was asked from me to build during the interview call and not to use any AI code editor .. I had to rely on docs and other people's code from stack overflow Its a replica of multi gigabyte video upload component from their core infra which I had to build and explain ... > has a grpc api for chunked exports > using redis to track the chunk assembly state like which chunks are received and expiration of chinks > and a simple html client to show the pause and resume of chunks for uploading Built with go, grpc, postgresql and redis

Mrinal

61,153 Aufrufe • vor 3 Monaten

A source from Afghanistan’s Ministry of Defense released a video from Bagram and claimed that Afghanistan has now built and purchased new, modern combat aircraft, which are currently deployed for the defense of Afghanistan. Afghanistan is the house of hope for All Muslims World

A source from Afghanistan’s Ministry of Defense released a video from Bagram and claimed that Afghanistan has now built and purchased new, modern combat aircraft, which are currently deployed for the defense of Afghanistan. Afghanistan is the house of hope for All Muslims World

Mullah Muhammad Yaqoob Mujahid

85,349 Aufrufe • vor 9 Monaten

Today, Box is announcing major new AI agent capabilities to let customers tap into the full value of their unstructured data. First, we’re announcing all new updates to the Box AI Studio to make it even easier to build AI agents that tap into your enterprise content for any job function, business process, or industry specific use case. We are also expanding our set of foundational agents that customers will be able to use to work with their enterprise content, including new features like search and research on unstructured data. Next, we’re announcing Box Extract to enable customers to use AI agents seamlessly for complex data extraction from any type of document or content. This makes it easier than ever to pull out data from contracts, invoices, research data, marketing assets, medical charts, and more. Finally, we’re introducing Box Automate, a new workflow automation solution within Box that lets you deploy AI agents across enterprise content-centric workflows. With Box Automate, you can design your business process in a simple drag and drop builder and then drop in AI agents at any step in the process. This ensures agents execute tasks at the right steps in a workflow every time. Best of all, our AI agents and workflow tools are designed to work across any system our customers work within, whether it’s leveraging pre-built integrations, Box APIs, or the new Box MCP Server. Ultimately, all of these capabilities come together to transform how companies can work with their enterprise content. Software has historically only been good at automating work that deals with structured data, which is why ERP, CRM, and HR systems have been mainstays of enterprise software for so long. The data in these systems fits neatly into a database, and the workflows are very ripe for automation. But it turns out most of the work in the world deals with unstructured data. It’s ideating through research documents, working with a client on contracts, reviewing details for a new product launch, looking at a patient’s healthcare record to make a diagnosis, working through due diligence documents for an M&A deal, and so on. For the first time ever, we can begin to bring all new insights and automation to this work with AI agents. At Box, we’re incredibly excited to be on this journey to help customers transform how they work with their most important data.

Today, Box is announcing major new AI agent capabilities to let customers tap into the full value of their unstructured data. First, we’re announcing all new updates to the Box AI Studio to make it even easier to build AI agents that tap into your enterprise content for any job function, business process, or industry specific use case. We are also expanding our set of foundational agents that customers will be able to use to work with their enterprise content, including new features like search and research on unstructured data. Next, we’re announcing Box Extract to enable customers to use AI agents seamlessly for complex data extraction from any type of document or content. This makes it easier than ever to pull out data from contracts, invoices, research data, marketing assets, medical charts, and more. Finally, we’re introducing Box Automate, a new workflow automation solution within Box that lets you deploy AI agents across enterprise content-centric workflows. With Box Automate, you can design your business process in a simple drag and drop builder and then drop in AI agents at any step in the process. This ensures agents execute tasks at the right steps in a workflow every time. Best of all, our AI agents and workflow tools are designed to work across any system our customers work within, whether it’s leveraging pre-built integrations, Box APIs, or the new Box MCP Server. Ultimately, all of these capabilities come together to transform how companies can work with their enterprise content. Software has historically only been good at automating work that deals with structured data, which is why ERP, CRM, and HR systems have been mainstays of enterprise software for so long. The data in these systems fits neatly into a database, and the workflows are very ripe for automation. But it turns out most of the work in the world deals with unstructured data. It’s ideating through research documents, working with a client on contracts, reviewing details for a new product launch, looking at a patient’s healthcare record to make a diagnosis, working through due diligence documents for an M&A deal, and so on. For the first time ever, we can begin to bring all new insights and automation to this work with AI agents. At Box, we’re incredibly excited to be on this journey to help customers transform how they work with their most important data.

Aaron Levie

91,863 Aufrufe • vor 1 Jahr

I trained a grasping policy on objects of various shapes and sizes using the new mjlab feature and this really cool pivot grasp strategy emerges for large flat objects that are larger than the gripper aperture. So cool and beautiful to see!

I trained a grasping policy on objects of various shapes and sizes using the new mjlab feature and this really cool pivot grasp strategy emerges for large flat objects that are larger than the gripper aperture. So cool and beautiful to see!

Kevin Zakka

20,458 Aufrufe • vor 5 Monaten

MLX + TurboQuant = Local Super Power Take a local (private) document(s) or codebase, pre-fill the 256k KV cache (the context) with the document(s) and system prompt, quantise and run on Apple's MLX and you have almost instantaneous, lossless document queries with total privacy. For a 75-page PDF (some 30k tokens), I was seeing sub-150ms TTFT (Time To First Token), that's "instant" for a human. We run this on an office server (one of our Mac Minis) for staff to work with, instant answers on critical, confidential company/client documents on phone, iPads or laptops. Imagine the "What were the 3Q25 profits?", "When did Joe start working for us?", "Please list the project milestones and team count" answered within seconds of hitting the return key. It works with voice too. This example is 17k lines of code from 177 source files plus an index of the entire 1,902 files, the entire Claude Code codebase (oops Anthropic!). Zero loss (unlike Anthropic's security). What this means is the ability to work better on local (private) code bases, off-the-shelf private document analysis with better response than the best public models - all locally.

MLX + TurboQuant = Local Super Power Take a local (private) document(s) or codebase, pre-fill the 256k KV cache (the context) with the document(s) and system prompt, quantise and run on Apple's MLX and you have almost instantaneous, lossless document queries with total privacy. For a 75-page PDF (some 30k tokens), I was seeing sub-150ms TTFT (Time To First Token), that's "instant" for a human. We run this on an office server (one of our Mac Minis) for staff to work with, instant answers on critical, confidential company/client documents on phone, iPads or laptops. Imagine the "What were the 3Q25 profits?", "When did Joe start working for us?", "Please list the project milestones and team count" answered within seconds of hitting the return key. It works with voice too. This example is 17k lines of code from 177 source files plus an index of the entire 1,902 files, the entire Claude Code codebase (oops Anthropic!). Zero loss (unlike Anthropic's security). What this means is the ability to work better on local (private) code bases, off-the-shelf private document analysis with better response than the best public models - all locally.

John T Davies 🇪🇺

85,778 Aufrufe • vor 5 Monaten

Introducing Document Extraction as an MCP Server ✂️📑 A huge use case for AI agents is being able to extract out items from a diverse set of complex documents in a repeatable manner - whether it’s legal contracts, invoices, financial statements, passports, and more. In this case, 🚫 prompting ChatGPT/Claude is too error-prone/not systematic. 💡We’ve created a plug-and-play MCP server around our core document extraction capabilities within LlamaCloud. 1. Build an extraction agent in LlamaCloud with a standardized schema. 2. Plug it as an MCP tool within ChatGPT/Claude. 3. For your next extraction job, get the outputs in a standardized format through the tool! Huge shoutout to Tuana and Logan Markewich for working on this - this is thanks to the latest updates to our llamacloud-mcp demo repo, which supports both extraction and retrieval as MCP tools. Repo: Sign up:

Introducing Document Extraction as an MCP Server ✂️📑 A huge use case for AI agents is being able to extract out items from a diverse set of complex documents in a repeatable manner - whether it’s legal contracts, invoices, financial statements, passports, and more. In this case, 🚫 prompting ChatGPT/Claude is too error-prone/not systematic. 💡We’ve created a plug-and-play MCP server around our core document extraction capabilities within LlamaCloud. 1. Build an extraction agent in LlamaCloud with a standardized schema. 2. Plug it as an MCP tool within ChatGPT/Claude. 3. For your next extraction job, get the outputs in a standardized format through the tool! Huge shoutout to Tuana and Logan Markewich for working on this - this is thanks to the latest updates to our llamacloud-mcp demo repo, which supports both extraction and retrieval as MCP tools. Repo: Sign up:

Jerry Liu

57,827 Aufrufe • vor 1 Jahr

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. 2. The connector ecosystem to load data from unstructured data sources is very immature. 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. The goal of a RAG Pipeline is to solve these problems. The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. At a high level, there are four different stages in the architecture of a RAG pipeline: 1. Ingestion: Here is where the pipeline loads the information from the data source. 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. 3. Transform: Where the pipeline chunks the data and generates document embeddings. 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. There are different rabbit holes at each one of these stages. Here are three of them: 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. If you have a few documents lying around, set up a free account and give it a try.

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. 2. The connector ecosystem to load data from unstructured data sources is very immature. 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. The goal of a RAG Pipeline is to solve these problems. The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. At a high level, there are four different stages in the architecture of a RAG pipeline: 1. Ingestion: Here is where the pipeline loads the information from the data source. 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. 3. Transform: Where the pipeline chunks the data and generates document embeddings. 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. There are different rabbit holes at each one of these stages. Here are three of them: 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. If you have a few documents lying around, set up a free account and give it a try.

Santiago

40,627 Aufrufe • vor 1 Jahr

Announcing a new Coursera course: Retrieval Augmented Generation (RAG) You'll learn to build high performance, production-ready RAG systems in this hands-on, in-depth course created by and taught by , experienced AI and ML engineer, researcher, and educator. RAG is a critical component today of many LLM-based applications in customer support, internal company Q&A systems, even many of the leading chatbots that use web search to answer your questions. This course teaches you in-depth how to make RAG work well. LLMs can produce generic or outdated responses, especially when asked specialized questions not covered in its training data. RAG is the most widely used technique for addressing this. It brings in data from new data sources, such as internal documents or recent news, to give the LLM the relevant context to private, recent, or specialized information. This lets it generate more grounded and accurate responses. In this course, you’ll learn to design and implement every part of a RAG system, from retrievers to vector databases to generation to evals. You’ll learn about the fundamental principles behind RAG and how to optimize it at both the component and whole-system levels. As AI evolves, RAG is evolving too. New models can handle longer context windows, reason more effectively, and can be parts of complex agentic workflows. One exciting growth area is Agentic RAG, in which an AI agent at runtime (rather than it being hardcoded at development time) autonomously decides what data to retrieve, and when/how to go deeper. Even with this evolution, access to high-quality data at runtime is essential, which is why RAG is a key part of so many applications. You'll learn via hands-on experiences to: - Build a RAG system with retrieval and prompt augmentation - Compare retrieval methods like BM25, semantic search, and Reciprocal Rank Fusion - Chunk, index, and retrieve documents using a Weaviate vector database and a news dataset - Develop a chatbot, using open-source LLMs hosted by Together AI, for a fictional store that answers product and FAQ questions - Use evals to drive improving reliability, and incorporate multi-modal data RAG is an important foundational technique. Become good at it through this course! Please sign up here:

Announcing a new Coursera course: Retrieval Augmented Generation (RAG) You'll learn to build high performance, production-ready RAG systems in this hands-on, in-depth course created by and taught by , experienced AI and ML engineer, researcher, and educator. RAG is a critical component today of many LLM-based applications in customer support, internal company Q&A systems, even many of the leading chatbots that use web search to answer your questions. This course teaches you in-depth how to make RAG work well. LLMs can produce generic or outdated responses, especially when asked specialized questions not covered in its training data. RAG is the most widely used technique for addressing this. It brings in data from new data sources, such as internal documents or recent news, to give the LLM the relevant context to private, recent, or specialized information. This lets it generate more grounded and accurate responses. In this course, you’ll learn to design and implement every part of a RAG system, from retrievers to vector databases to generation to evals. You’ll learn about the fundamental principles behind RAG and how to optimize it at both the component and whole-system levels. As AI evolves, RAG is evolving too. New models can handle longer context windows, reason more effectively, and can be parts of complex agentic workflows. One exciting growth area is Agentic RAG, in which an AI agent at runtime (rather than it being hardcoded at development time) autonomously decides what data to retrieve, and when/how to go deeper. Even with this evolution, access to high-quality data at runtime is essential, which is why RAG is a key part of so many applications. You'll learn via hands-on experiences to: - Build a RAG system with retrieval and prompt augmentation - Compare retrieval methods like BM25, semantic search, and Reciprocal Rank Fusion - Chunk, index, and retrieve documents using a Weaviate vector database and a news dataset - Develop a chatbot, using open-source LLMs hosted by Together AI, for a fictional store that answers product and FAQ questions - Use evals to drive improving reliability, and incorporate multi-modal data RAG is an important foundational technique. Become good at it through this course! Please sign up here:

Andrew Ng

124,656 Aufrufe • vor 1 Jahr