Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Document Querying with Qwen2-VL-7B and JSON Output Complete video: I demonstrate how to perform document queries using Qwen2-VL-7B. By simplifying field names, we streamline the prompts, making them more efficient and reusable across different documents. This approach is similar to running SQL queries on a database, but tailored for... show more

Andrej Baranovskij

6,682 subscribers

15,981 Aufrufe • vor 2 Jahren •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

6 Kommentare

Profilbild von Rubén Aros

Rubén Arosvor 2 Jahren

It's the best vllm?

Profilbild von Andrej Baranovskij

Andrej Baranovskijvor 2 Jahren

At the moment yes definitely- qwen2 is the best open source llm with apache license. China rules 🇨🇳👍

Profilbild von optimizium.base.eth (ξ/e)

optimizium.base.eth (ξ/e)vor 2 Jahren

Still unable to compile flash attention 2 on my Windows machine. :( So annoying.

Profilbild von Andrej Baranovskij

Andrej Baranovskijvor 2 Jahren

I believe Windows is not well supported platform. Btw, it looks like new Qwen models will be announced next week -

Profilbild von Z❤️H

Z❤️Hvor 2 Jahren

it hallucinante when it should retrieve a correlated informations from multiple lines

Profilbild von Andrej Baranovskij

Andrej Baranovskijvor 2 Jahren

Not per my tests, qwen2 7b works excellent. For more complex docs use 72b

Ähnliche Videos

Molmo by Ai2 - Open source SoTA Multimodal (Vision) Language model, beating Claude 3.5 Sonnet, GPT4V and comparable to GPT4o 🔥 They release four model checkpoints: 1. MolmoE-1B, a mixture of experts model with 1B (active) 7B (total) 2. Molmo-7B-O, most open 7B model 3. Molmo-7B-D, demo model 4. Molmo-72B, best model System Architecture > Input: Multi-scale, multi-crop images generated from the original image. > Vision Encoder: OpenAI's ViT-L/14 336px CLIP model, a powerful ViT, encodes images into vision tokens. > Connector: MLP projects tokens to LLM input space, followed by pooling for dimensionality reduction. > LLM: Decoder-only Transformer, various options (OLMo, OLMoE, Qwen2, Mistral, Gemma2, Phi) with diverse scales and openness. Model Variants > Vision Encoder: Consistent ViT-L/14 CLIP model across variants. > LLM: OLMo-7B-1024, OLMoE-1B-7B-0924, Qwen2 (7B, 72B), Mistral 7B, Gemma2 9B, Phi 3 Medium, offering different capacities and openness levels. Training Strategy > Stage 1: Multimodal pre-training for caption generation with new captioning data. > Stage 2: Supervised fine-tuning on a dataset mixture, updating all parameters. > No RLHF involved, Learning rates adjusted based on component types and pre-training status. > All the weights are available on Hugging Face Hub 🤗 > Compatible with Transformers (Remote Code) Kudos Ai2 for such a brilliant and open work! 🐐 Video credits: Allen AI YT Channel

Molmo by Ai2 - Open source SoTA Multimodal (Vision) Language model, beating Claude 3.5 Sonnet, GPT4V and comparable to GPT4o 🔥 They release four model checkpoints: 1. MolmoE-1B, a mixture of experts model with 1B (active) 7B (total) 2. Molmo-7B-O, most open 7B model 3. Molmo-7B-D, demo model 4. Molmo-72B, best model System Architecture > Input: Multi-scale, multi-crop images generated from the original image. > Vision Encoder: OpenAI's ViT-L/14 336px CLIP model, a powerful ViT, encodes images into vision tokens. > Connector: MLP projects tokens to LLM input space, followed by pooling for dimensionality reduction. > LLM: Decoder-only Transformer, various options (OLMo, OLMoE, Qwen2, Mistral, Gemma2, Phi) with diverse scales and openness. Model Variants > Vision Encoder: Consistent ViT-L/14 CLIP model across variants. > LLM: OLMo-7B-1024, OLMoE-1B-7B-0924, Qwen2 (7B, 72B), Mistral 7B, Gemma2 9B, Phi 3 Medium, offering different capacities and openness levels. Training Strategy > Stage 1: Multimodal pre-training for caption generation with new captioning data. > Stage 2: Supervised fine-tuning on a dataset mixture, updating all parameters. > No RLHF involved, Learning rates adjusted based on component types and pre-training status. > All the weights are available on Hugging Face Hub 🤗 > Compatible with Transformers (Remote Code) Kudos Ai2 for such a brilliant and open work! 🐐 Video credits: Allen AI YT Channel

Vaibhav (VB) Srivastav

80,474 Aufrufe • vor 2 Jahren

Text-to-SQL is dead 💀 The next generation of querying is agentic - and it’s already here. This paper ( introduces a type of agentic querying called 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻 𝗖𝗮𝗹𝗹𝗶𝗻𝗴 that uses an LLM to structure queries using predefined function calls in JSON format, with optional arguments for search, filters, aggregation, and grouping. Along with that, it also tests out a bunch of different models with a new dataset, DBGorilla, designed to evaluate agentic querying techniques on real-world use cases. Weaviate AI Database also just released a 𝗤𝘂𝗲𝗿𝘆 𝗔𝗴𝗲𝗻𝘁, designed based on some of the work in this paper, to handle advanced agentic querying out of the box, find out more here:

Text-to-SQL is dead 💀 The next generation of querying is agentic - and it’s already here. This paper ( introduces a type of agentic querying called 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻 𝗖𝗮𝗹𝗹𝗶𝗻𝗴 that uses an LLM to structure queries using predefined function calls in JSON format, with optional arguments for search, filters, aggregation, and grouping. Along with that, it also tests out a bunch of different models with a new dataset, DBGorilla, designed to evaluate agentic querying techniques on real-world use cases. Weaviate AI Database also just released a 𝗤𝘂𝗲𝗿𝘆 𝗔𝗴𝗲𝗻𝘁, designed based on some of the work in this paper, to handle advanced agentic querying out of the box, find out more here:

Victoria Slocum

189,664 Aufrufe • vor 1 Jahr

🚀 Exciting News! We're thrilled to announce the launch of Qwen Chat ( ) – your new go-to Web UI for interacting with Qwen models! 🌟 💬 Chat effortlessly with our flagship model Qwen2.5-Plus , explore vision-language capabilities with Qwen2-VL-Max , and dive into reasoning models like QwQ and QVQ, code with coding expert Qwen2.5-Coder-32B-Instruct, etc. ✨ Key Features: * Choose & compare multiple models in one interface * Upload documents & get answers based on them * Preview mode with HTML support * Upload images for visual understanding 🌟 Coming Soon: Web search, image generation, voice mode, and more! 🔗 Ready to experience the future? Start to chat with Qwen today 💬 Feel free to send us your feedback and suggestions!

🚀 Exciting News! We're thrilled to announce the launch of Qwen Chat ( ) – your new go-to Web UI for interacting with Qwen models! 🌟 💬 Chat effortlessly with our flagship model Qwen2.5-Plus , explore vision-language capabilities with Qwen2-VL-Max , and dive into reasoning models like QwQ and QVQ, code with coding expert Qwen2.5-Coder-32B-Instruct, etc. ✨ Key Features: * Choose & compare multiple models in one interface * Upload documents & get answers based on them * Preview mode with HTML support * Upload images for visual understanding 🌟 Coming Soon: Web search, image generation, voice mode, and more! 🔗 Ready to experience the future? Start to chat with Qwen today 💬 Feel free to send us your feedback and suggestions!

Qwen

215,564 Aufrufe • vor 1 Jahr

Check out our demos using LFM2.5-VL-3B, our latest lightweight, vision-language model that reads screens, documents, and the physical world. First up: LFM2.5-VL-3B running fully on-device in the browser with WebGPU to understand a document page. The model parses the entire layout in one pass and returns regions and labels that the interface renders as an overlay. The demo highlights OCR and layout understanding for visually structured content such as forms, reports, receipts, and other documents. 🧵

Check out our demos using LFM2.5-VL-3B, our latest lightweight, vision-language model that reads screens, documents, and the physical world. First up: LFM2.5-VL-3B running fully on-device in the browser with WebGPU to understand a document page. The model parses the entire layout in one pass and returns regions and labels that the interface renders as an overlay. The demo highlights OCR and layout understanding for visually structured content such as forms, reports, receipts, and other documents. 🧵

Liquid AI

12,613 Aufrufe • vor 1 Monat

Baidu Inc. just dropped PaddleOCR-VL-1.5, and it’s not just another OCR model. This is a fully open-source OCR model with just 0.9B parameters, built specifically for production-grade document understanding. And the results speak loudly 👇 On OmniDocBench V1.5, the most authoritative global document parsing benchmark, PaddleOCR-VL-1.5 ranks #1 overall, hitting 94.5% overall accuracy, outperforming models like DeepSeek-OCR2. As the first OCR model natively supporting irregular document layout positioning, it perfectly solves OCR pain points in production for complex structured business documents such as financial reports and insurance forms. While a lot of the industry is racing toward bigger models, PaddleOCR is betting on something smarter: small parameters + high performance + open source + real usability. That combination is exactly what scalable document intelligence needs right now. Demo / API → Open source → Model → Curious to see how teams start building on top of this.

Baidu Inc. just dropped PaddleOCR-VL-1.5, and it’s not just another OCR model. This is a fully open-source OCR model with just 0.9B parameters, built specifically for production-grade document understanding. And the results speak loudly 👇 On OmniDocBench V1.5, the most authoritative global document parsing benchmark, PaddleOCR-VL-1.5 ranks #1 overall, hitting 94.5% overall accuracy, outperforming models like DeepSeek-OCR2. As the first OCR model natively supporting irregular document layout positioning, it perfectly solves OCR pain points in production for complex structured business documents such as financial reports and insurance forms. While a lot of the industry is racing toward bigger models, PaddleOCR is betting on something smarter: small parameters + high performance + open source + real usability. That combination is exactly what scalable document intelligence needs right now. Demo / API → Open source → Model → Curious to see how teams start building on top of this.

Bishal Nandi

35,318 Aufrufe • vor 8 Monaten

New LLMs that control UIs! ByteDance Research releases UI-TARS, fine-tuned GUI agent that integrates reasoning, and action capabilities into a single vision-language model. Think of computer use but open. 👀 TL;DR; 3️⃣ Available in 3 sizes: 2B, 7B, and 72B parameters 🧠 Trained Qwen2-VL models with SFT & DPO 🥇 72B version achieves 82.8% on VisualWebBench (beating GPT-4 and Claude) 🏆 Achieves state-of-the-art results on 10+ GUI agent benchmarks 💡 Reasons before taking an action 🧑🏻‍💻 Can Click, Long Press, type, scroll, open app, navigate back/home, wait 🤗 Released under Apache 2.0 on Hugging Face

New LLMs that control UIs! ByteDance Research releases UI-TARS, fine-tuned GUI agent that integrates reasoning, and action capabilities into a single vision-language model. Think of computer use but open. 👀 TL;DR; 3️⃣ Available in 3 sizes: 2B, 7B, and 72B parameters 🧠 Trained Qwen2-VL models with SFT & DPO 🥇 72B version achieves 82.8% on VisualWebBench (beating GPT-4 and Claude) 🏆 Achieves state-of-the-art results on 10+ GUI agent benchmarks 💡 Reasons before taking an action 🧑🏻‍💻 Can Click, Long Press, type, scroll, open app, navigate back/home, wait 🤗 Released under Apache 2.0 on Hugging Face

Philipp Schmid

48,170 Aufrufe • vor 1 Jahr

🎉 恭喜发财🧧🐍 As we welcome the Chinese New Year, we're thrilled to announce the launch of Qwen2.5-VL , our latest flagship vision-language model! 🚀 💗 Qwen Chat: 📖 Blog: 🤗 Hugging Face: 🤖 ModelScope: 🌟 Key Highlights: * Visual Understanding : From flowers to complex charts, Qwen2.5-VL sees it all! * Agentic Capabilities : It’s a visual agent that can reason and interact with tools like computers & phones. * Long Video Comprehension : Captures events in videos over 1 hour long! ⏳🎥 * Precise Localization : Generates bounding boxes & JSON outputs for accurate object detection. * Structured Data Outputs : Perfect for finance & commerce, handling invoices, forms & more! 💼📊 Try Qwen2.5-VL now at Qwen Chat or explore models on Hugging Face & ModelScope . 🌐

🎉 恭喜发财🧧🐍 As we welcome the Chinese New Year, we're thrilled to announce the launch of Qwen2.5-VL , our latest flagship vision-language model! 🚀 💗 Qwen Chat: 📖 Blog: 🤗 Hugging Face: 🤖 ModelScope: 🌟 Key Highlights: * Visual Understanding : From flowers to complex charts, Qwen2.5-VL sees it all! * Agentic Capabilities : It’s a visual agent that can reason and interact with tools like computers & phones. * Long Video Comprehension : Captures events in videos over 1 hour long! ⏳🎥 * Precise Localization : Generates bounding boxes & JSON outputs for accurate object detection. * Structured Data Outputs : Perfect for finance & commerce, handling invoices, forms & more! 💼📊 Try Qwen2.5-VL now at Qwen Chat or explore models on Hugging Face & ModelScope . 🌐

Qwen

762,963 Aufrufe • vor 1 Jahr

ColPali is changing the game for PDF retrieval by eliminating the need for OCR and chunking methods 🚀 Inspired by ColBERT’s success with text, ColPali splits an image of a document into patches, which are then processed through a vision LLM called PaliGemma. The embeddings for each patch retain contextual information, similarly to text embeddings in methods like ColBERT. During retrieval, user queries are embedded in the same space, and then compared to document patches using the MaxSim operator. ColPali recipe POC Weaviate AI Database: ColPali paper: As always, shoutout to the awesome Daniel Williams for the help! 💙

ColPali is changing the game for PDF retrieval by eliminating the need for OCR and chunking methods 🚀 Inspired by ColBERT’s success with text, ColPali splits an image of a document into patches, which are then processed through a vision LLM called PaliGemma. The embeddings for each patch retain contextual information, similarly to text embeddings in methods like ColBERT. During retrieval, user queries are embedded in the same space, and then compared to document patches using the MaxSim operator. ColPali recipe POC Weaviate AI Database: ColPali paper: As always, shoutout to the awesome Daniel Williams for the help! 💙

Victoria Slocum

67,862 Aufrufe • vor 1 Jahr

I promised a longer video on how to build a full-stack ICP app, powered by SQLite. Here it is! This is a nose to tail walkthrough, we look at database migrations, automatic input validation, DTO objects, performant frontend queries using Tanstack Query and lots more. Links to demo app, libraries and YT video in the responses. 🌱

I promised a longer video on how to build a full-stack ICP app, powered by SQLite. Here it is! This is a nose to tail walkthrough, we look at database migrations, automatic input validation, DTO objects, performant frontend queries using Tanstack Query and lots more. Links to demo app, libraries and YT video in the responses. 🌱

Kristofer

13,063 Aufrufe • vor 11 Monaten

Introducing VL-JEPA: Vision-Language Joint Embedding Predictive Architecture for streaming, live action recognition, retrieval, VQA, and classification tasks with better performance and higher efficiency than large VLMs. • VL-JEPA is the first non-generative model that can perform general-domain vision-language tasks in real-time, built on a joint embedding predictive architecture. • We demonstrate in controlled experiments that VL-JEPA, trained with latent space embedding prediction, outperforms VLMs that rely on data space token prediction. • We show that VL-JEPA delivers significant efficiency gains over VLMs for online video streaming applications, thanks to its non-autoregressive design and native support for selective decoding. • We highlight that our VL-JEPA model, with an unified model architecture, can effectively handle a wide range of classification, retrieval, and VQA tasks at the same time. by Delong Chen (陈德龙) Mustafa Shukor Théo Moutakanni Willy Jade Lei Yu Tejaswi Kasarla Allen Bolourchi Yann LeCun Pascale Fung

Introducing VL-JEPA: Vision-Language Joint Embedding Predictive Architecture for streaming, live action recognition, retrieval, VQA, and classification tasks with better performance and higher efficiency than large VLMs. • VL-JEPA is the first non-generative model that can perform general-domain vision-language tasks in real-time, built on a joint embedding predictive architecture. • We demonstrate in controlled experiments that VL-JEPA, trained with latent space embedding prediction, outperforms VLMs that rely on data space token prediction. • We show that VL-JEPA delivers significant efficiency gains over VLMs for online video streaming applications, thanks to its non-autoregressive design and native support for selective decoding. • We highlight that our VL-JEPA model, with an unified model architecture, can effectively handle a wide range of classification, retrieval, and VQA tasks at the same time. by Delong Chen (陈德龙) Mustafa Shukor Théo Moutakanni Willy Jade Lei Yu Tejaswi Kasarla Allen Bolourchi Yann LeCun Pascale Fung

Pascale Fung

90,144 Aufrufe • vor 9 Monaten

New course: Document AI: From OCR to Agentic Doc Extraction, built with LandingAI, where I'm executive chairman, and taught by David Park and Andrea Kropp. Much of the world's data is locked in PDFs, JPEGs, and other documents. This short course shows you how to build agentic workflows that process documents accurately: breaking them into parts, examining each piece carefully, and extracting information through multiple iterations. Traditional Optical Character Recognition (OCR) captures text but loses context from table headers, chart captions, or reading order of columns. After exploring OCR's limitations, you’ll use LandingAI's Agentic Document Extraction (ADE) framework to process documents. ADE treats pages as visually -- as images -- to parse information and extract fields. Skills you'll gain: - Build agents to convert unstructured files into structured Markdown/HTML and JSON - Use ADE to parse complex data like forms, handwriting, or equations - Map extracted information to named fields using a specified schema, with bounding boxes for grounding and validation - Deploy RAG applications with event-driven document processing Come learn about the best tools for processing documents like financial invoices, medical records, or academic papers intelligently:

New course: Document AI: From OCR to Agentic Doc Extraction, built with LandingAI, where I'm executive chairman, and taught by David Park and Andrea Kropp. Much of the world's data is locked in PDFs, JPEGs, and other documents. This short course shows you how to build agentic workflows that process documents accurately: breaking them into parts, examining each piece carefully, and extracting information through multiple iterations. Traditional Optical Character Recognition (OCR) captures text but loses context from table headers, chart captions, or reading order of columns. After exploring OCR's limitations, you’ll use LandingAI's Agentic Document Extraction (ADE) framework to process documents. ADE treats pages as visually -- as images -- to parse information and extract fields. Skills you'll gain: - Build agents to convert unstructured files into structured Markdown/HTML and JSON - Use ADE to parse complex data like forms, handwriting, or equations - Map extracted information to named fields using a specified schema, with bounding boxes for grounding and validation - Deploy RAG applications with event-driven document processing Come learn about the best tools for processing documents like financial invoices, medical records, or academic papers intelligently:

Andrew Ng

200,844 Aufrufe • vor 8 Monaten

New Short Course: Getting Structured LLM Output! Learn how to get structured outputs from your LLM applications in this course, built in partnership with .txt, and taught by Will Kurt, a Founding Engineer, and , Developer Relations Engineer. It's challenging for software to automatically parse through an LLM's freeform text outputs. Structured outputs—like JSON—solve this by converting natural language into consistent, clear, data that a machine can read and process. This course teaches you how to generate structured outputs while building several use cases, including a social media analysis agent. You’ll learn about structured outputs and efficient ways to generate outputs in your defined schema or format. You’ll begin by using structured output APIs, then use re-prompting libraries like “instructor” to generate structured output. Finally, you’ll learn how constrained decoding works; this is a very clever technique in which constraints are applied on each subsequent token generated, blocking any tokens that don’t fit your defined schema. In detail, you’ll: - Learn why structured outputs are important, how they allow for scalable software development, and the different approaches to generate them, including vendor-provided APIs, re-prompting libraries, and structured generation. - Build a simple social media agent using OpenAI’s structured output API, learn how to define a model's desired structured output using Pydantic, and perform basic programming with your outputs, such as importing structured data into a data frame using pandas. - Learn how to use the open-source library "instructor," which checks the structured output of the model and re-prompts the model until it validates the desired output, and explore the limitations of this approach. - Understand how structured generation by the “outlines” library works by modifying LLM logits, on a per-generated-token basis based on the desired format, to give a particular output structure. - Learn how regular expressions, which outlines works with, are represented as finite-state machines, and how they can be used to develop a range of structured outputs beyond JSON. By the end of this course, you’ll have broadened your knowledge of the approaches you can use to get structured outputs from your LLM applications. Please sign up here:

New Short Course: Getting Structured LLM Output! Learn how to get structured outputs from your LLM applications in this course, built in partnership with .txt, and taught by Will Kurt, a Founding Engineer, and , Developer Relations Engineer. It's challenging for software to automatically parse through an LLM's freeform text outputs. Structured outputs—like JSON—solve this by converting natural language into consistent, clear, data that a machine can read and process. This course teaches you how to generate structured outputs while building several use cases, including a social media analysis agent. You’ll learn about structured outputs and efficient ways to generate outputs in your defined schema or format. You’ll begin by using structured output APIs, then use re-prompting libraries like “instructor” to generate structured output. Finally, you’ll learn how constrained decoding works; this is a very clever technique in which constraints are applied on each subsequent token generated, blocking any tokens that don’t fit your defined schema. In detail, you’ll: - Learn why structured outputs are important, how they allow for scalable software development, and the different approaches to generate them, including vendor-provided APIs, re-prompting libraries, and structured generation. - Build a simple social media agent using OpenAI’s structured output API, learn how to define a model's desired structured output using Pydantic, and perform basic programming with your outputs, such as importing structured data into a data frame using pandas. - Learn how to use the open-source library "instructor," which checks the structured output of the model and re-prompts the model until it validates the desired output, and explore the limitations of this approach. - Understand how structured generation by the “outlines” library works by modifying LLM logits, on a per-generated-token basis based on the desired format, to give a particular output structure. - Learn how regular expressions, which outlines works with, are represented as finite-state machines, and how they can be used to develop a range of structured outputs beyond JSON. By the end of this course, you’ll have broadened your knowledge of the approaches you can use to get structured outputs from your LLM applications. Please sign up here:

Andrew Ng

89,792 Aufrufe • vor 1 Jahr

Quick demo of Zero’s new background queries. Zero’s sync is query-based. Rather than specifying what data you want using rules or some other separate system, you just use queries. Right inside the client app, you do a query using a full sql-style language. You get filters, subqueries , limits, etc. Zero syncs the data backing these queries to the client. It’s important to realize Zero isn’t really a cache. It’s a replica. It’s eagerly replicating a precise snapshot of a slice of your database - the slice covered by the queries you have open. So there is never stale data in Zero. When you close a query we delete the rows uniquely returned by that query because we can no longer keep them up to date. Of course that kind of sucks for the common case of doing a query, navigating, then pressing “back”. Ideally we want that back nav to be fast. To address this Zero 0.17 adds background queries. You can add a ttl to a query and it will keep running and syncing in the background. This is much different than normal caching because this data stays up to date. If you make the same query again, the results will be *instantly* available *and already up to date with server*. If you make a different query the data from the background query is used client-side to answer the new query instantly if possible. This all happens completely automatically. Just by adding the ttl flag.

Quick demo of Zero’s new background queries. Zero’s sync is query-based. Rather than specifying what data you want using rules or some other separate system, you just use queries. Right inside the client app, you do a query using a full sql-style language. You get filters, subqueries , limits, etc. Zero syncs the data backing these queries to the client. It’s important to realize Zero isn’t really a cache. It’s a replica. It’s eagerly replicating a precise snapshot of a slice of your database - the slice covered by the queries you have open. So there is never stale data in Zero. When you close a query we delete the rows uniquely returned by that query because we can no longer keep them up to date. Of course that kind of sucks for the common case of doing a query, navigating, then pressing “back”. Ideally we want that back nav to be fast. To address this Zero 0.17 adds background queries. You can add a ttl to a query and it will keep running and syncing in the background. This is much different than normal caching because this data stays up to date. If you make the same query again, the results will be instantly available and already up to date with server. If you make a different query the data from the background query is used client-side to answer the new query instantly if possible. This all happens completely automatically. Just by adding the ttl flag.

Aaron Boodman

18,808 Aufrufe • vor 1 Jahr

This is, by far, one of the best uses of modern AI. If you don't use embeddings when querying your database, you are definitely leaving a lot on the table. In this video, I'll show you how to run semantic searches using OpenAI and PostgreSQL. It's all thanks to Pgai, an open-source PostgreSQL extension: Here's what will happen: 1. We'll create a simple table with news articles 2. We'll generate embeddings for those articles 3. We'll run queries on top of those embeddings For this video, I generated the embeddings using a simple query, but pgai Vectorizer would do the same automatically as new information makes it into the database. This is awesome! If you have a PostgreSQL database with data you are searching over, you should start experimenting with semantic searches immediately. For most use cases, a combination of full-text search + semantic search is the best approach. If you don't have a PostgreSQL database around, you can try free for 30 days using Timescale: Thanks to the TimescaleDB (by Tiger Data) team for partnering with me on this post!

This is, by far, one of the best uses of modern AI. If you don't use embeddings when querying your database, you are definitely leaving a lot on the table. In this video, I'll show you how to run semantic searches using OpenAI and PostgreSQL. It's all thanks to Pgai, an open-source PostgreSQL extension: Here's what will happen: 1. We'll create a simple table with news articles 2. We'll generate embeddings for those articles 3. We'll run queries on top of those embeddings For this video, I generated the embeddings using a simple query, but pgai Vectorizer would do the same automatically as new information makes it into the database. This is awesome! If you have a PostgreSQL database with data you are searching over, you should start experimenting with semantic searches immediately. For most use cases, a combination of full-text search + semantic search is the best approach. If you don't have a PostgreSQL database around, you can try free for 30 days using Timescale: Thanks to the TimescaleDB (by Tiger Data) team for partnering with me on this post!

Santiago

110,133 Aufrufe • vor 1 Jahr

Today, we’re excited to announce Deep Extract. We’re pushing the boundary of complex document extraction by utilizing an agent harness approach, iterating and verifying outputs until they are at human-level accuracy. The results speak for themselves: we’ve handled documents exceeding 2,500 pages, extracting a total of 30M+ fields in the last couple weeks with results in our production beta reaching 99-100% accuracy. It’s particularly effective on complex documents with long lists, such as invoice line items, brokerage statement transactions, equipment manifests, and more. See more on how we built it, how to use it, and more👇

Today, we’re excited to announce Deep Extract. We’re pushing the boundary of complex document extraction by utilizing an agent harness approach, iterating and verifying outputs until they are at human-level accuracy. The results speak for themselves: we’ve handled documents exceeding 2,500 pages, extracting a total of 30M+ fields in the last couple weeks with results in our production beta reaching 99-100% accuracy. It’s particularly effective on complex documents with long lists, such as invoice line items, brokerage statement transactions, equipment manifests, and more. See more on how we built it, how to use it, and more👇

Reducto

23,949,618 Aufrufe • vor 5 Monaten

Introducing DocJev - a lightning-fast OSS library for document classification and splitting with jev ⚡️ Give a document alongside some natural language category rules. Jev will predict the document category (classify) or the boundaries between sub-documents (split). It is 6x faster than gpt-5.6-luna with equivalent accuracy. You can choose between different OCR backends: liteparse: the fastest/most accurate free+OSS text parser out there. Perfect for digitalized documents. The speedups vs. luna above are inclusive of liteparse times. LlamaParse: the most advanced VLM-based solution for complex docs. Using this will of course add latency on document preprocessing. But it’s a good option if you want to cache the digitalized representations of documents for a variety of downstream tasks. Check it out: If you want to dive deeper into liteparse or LlamaParse check the links below: liteparse: LlamaParse:

Introducing DocJev - a lightning-fast OSS library for document classification and splitting with jev ⚡️ Give a document alongside some natural language category rules. Jev will predict the document category (classify) or the boundaries between sub-documents (split). It is 6x faster than gpt-5.6-luna with equivalent accuracy. You can choose between different OCR backends: liteparse: the fastest/most accurate free+OSS text parser out there. Perfect for digitalized documents. The speedups vs. luna above are inclusive of liteparse times. LlamaParse: the most advanced VLM-based solution for complex docs. Using this will of course add latency on document preprocessing. But it’s a good option if you want to cache the digitalized representations of documents for a variety of downstream tasks. Check it out: If you want to dive deeper into liteparse or LlamaParse check the links below: liteparse: LlamaParse:

Jerry Liu

213,302 Aufrufe • vor 6 Tagen

Databases are far from dead. Hot take within the vibe-coding community, but you can't build a reliable agentic memory system using files alone. The filesystem is a great interface for agents, but for complex, distributed, production applications, databases win hands down. I recorded a video to show you the benchmarks. Large Language Models know how to navigate and work with the filesystem, but as soon as you add complexity, files will fall short. You need databases whenever any of the following happens: 1. You have concurrent writes from multiple agents or users 2. You need semantic retrieval at scale 3. You need ACID guarantees for shared state 4. You need audit trails and row-level access control 5. You need indexed queries over growing memory In the attached video, I'm running a notebook comparing a filesystem-backed agent with a database-backed agent. The three most important findings: • Filesystem = Database with small corpus, keyword-friendly queries • Databases > Filesystem with large corpus, fuzzy queries • Databases > Filesystem with concurrent writes without locking Numbers don't lie. You can run the benchmarks yourself.

Databases are far from dead. Hot take within the vibe-coding community, but you can't build a reliable agentic memory system using files alone. The filesystem is a great interface for agents, but for complex, distributed, production applications, databases win hands down. I recorded a video to show you the benchmarks. Large Language Models know how to navigate and work with the filesystem, but as soon as you add complexity, files will fall short. You need databases whenever any of the following happens: 1. You have concurrent writes from multiple agents or users 2. You need semantic retrieval at scale 3. You need ACID guarantees for shared state 4. You need audit trails and row-level access control 5. You need indexed queries over growing memory In the attached video, I'm running a notebook comparing a filesystem-backed agent with a database-backed agent. The three most important findings: • Filesystem = Database with small corpus, keyword-friendly queries • Databases > Filesystem with large corpus, fuzzy queries • Databases > Filesystem with concurrent writes without locking Numbers don't lie. You can run the benchmarks yourself.

Santiago

29,445 Aufrufe • vor 4 Monaten

GPT-4o is bad at processing PDF documents. Whoever tells you otherwise is not living in the real world. In 2024, people fill out forms using pen and paper. Try to answer questions from those forms using modern models, and you'll be disappointed. I recorded a video to show you how to fix this. The answer is simple: stop letting the model see your PDF document. Instead, preprocess it and stick to showing the model text. Watch the video. You'll go from "THIS IS CRAP" to "EXCELLENT" in no time. I'm using Unstract to turn the documents into text while keeping the original format (this is crucial!) You can use them to process up to 100 pages for free. They collaborated with me on this post. You can find the code I wrote right here:

GPT-4o is bad at processing PDF documents. Whoever tells you otherwise is not living in the real world. In 2024, people fill out forms using pen and paper. Try to answer questions from those forms using modern models, and you'll be disappointed. I recorded a video to show you how to fix this. The answer is simple: stop letting the model see your PDF document. Instead, preprocess it and stick to showing the model text. Watch the video. You'll go from "THIS IS CRAP" to "EXCELLENT" in no time. I'm using Unstract to turn the documents into text while keeping the original format (this is crucial!) You can use them to process up to 100 pages for free. They collaborated with me on this post. You can find the code I wrote right here:

Santiago

196,868 Aufrufe • vor 2 Jahren

We’re excited to officially launch LlamaParse, the first genAI-native document parsing solution. Not only is it better at parsing out images/tables/charts 📊📈 than virtually every other parser, it is now steerable through natural language instructions - output the document in whatever format you desire! It is also the only parsing solution that seamlessly allows you to build accurate RAG over complex documents, free of hallucinations 🔥 We launched it in private preview a few weeks ago and hit 2k users, 1M total PDF pages parsed. And now it’s better than ever. LlamaParse contains the following killer features: ✅ SOTA table/chart extraction ✅ Seamless integration with LlamaIndex 🦙 advanced RAG/agents ✅✨ Natural language Parsing Instructions ✅✨JSON mode and image extraction ✅✨Support for ~10 document types (.pdf, .pptx, .docx, .xml) and more Our pricing is simple: 1k free per day, and additional pages at 0.3c a page, or $3 for 1k pages. If you want advanced document RAG and/or private deployments, come get in touch with us to chat about LlamaCloud. Check out our full blog post here: LlamaParse client repo: Signup at 🦙☁️: Come talk to us:

We’re excited to officially launch LlamaParse, the first genAI-native document parsing solution. Not only is it better at parsing out images/tables/charts 📊📈 than virtually every other parser, it is now steerable through natural language instructions - output the document in whatever format you desire! It is also the only parsing solution that seamlessly allows you to build accurate RAG over complex documents, free of hallucinations 🔥 We launched it in private preview a few weeks ago and hit 2k users, 1M total PDF pages parsed. And now it’s better than ever. LlamaParse contains the following killer features: ✅ SOTA table/chart extraction ✅ Seamless integration with LlamaIndex 🦙 advanced RAG/agents ✅✨ Natural language Parsing Instructions ✅✨JSON mode and image extraction ✅✨Support for ~10 document types (.pdf, .pptx, .docx, .xml) and more Our pricing is simple: 1k free per day, and additional pages at 0.3c a page, or $3 for 1k pages. If you want advanced document RAG and/or private deployments, come get in touch with us to chat about LlamaCloud. Check out our full blog post here: LlamaParse client repo: Signup at 🦙☁️: Come talk to us:

LlamaIndex 🦙

143,189 Aufrufe • vor 2 Jahren

New short course with Mistral AI ! Mistral's open-source Mixtral 8x7B model uses a "mixture of experts" (MoE) architecture. Unlike a standard transformer, an MoE model has multiple expert feed-forward networks (8 in this case), with a gating network selecting two experts at inference time. This enables MoE to match the performance of a large model but faster inference. Mixtral 8x7B has 46.7B parameters but activates only 12.9B at inference to predict the next token. In Getting Started with Mistral, you’ll learn from Mistral’s Sophia Yang, Ph.D. to: - Explore Mistral's open-source models (Mistral 7B, Mixtral 8x7B) and commercial models via API calls and Mistral AI's Le Chat website - Implement JSON mode to generate structured outputs to integrate directly into larger software systems. - Use function calling for Tool Use, such as calling custom Python code that queries tabular data - Ground your LLM's response with external knowledge sources using RAG - Build a Mistral-powered chat interface that can reference external documents This course will help deepen your prompt engineering skills. Please sign up here:

New short course with Mistral AI ! Mistral's open-source Mixtral 8x7B model uses a "mixture of experts" (MoE) architecture. Unlike a standard transformer, an MoE model has multiple expert feed-forward networks (8 in this case), with a gating network selecting two experts at inference time. This enables MoE to match the performance of a large model but faster inference. Mixtral 8x7B has 46.7B parameters but activates only 12.9B at inference to predict the next token. In Getting Started with Mistral, you’ll learn from Mistral’s Sophia Yang, Ph.D. to: - Explore Mistral's open-source models (Mistral 7B, Mixtral 8x7B) and commercial models via API calls and Mistral AI's Le Chat website - Implement JSON mode to generate structured outputs to integrate directly into larger software systems. - Use function calling for Tool Use, such as calling custom Python code that queries tabular data - Ground your LLM's response with external knowledge sources using RAG - Build a Mistral-powered chat interface that can reference external documents This course will help deepen your prompt engineering skills. Please sign up here:

Andrew Ng

387,172 Aufrufe • vor 2 Jahren