Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Traditional chunking: cheap but dumb. ColBERT: smart but expensive. 𝗟𝗮𝘁𝗲 𝗰𝗵𝘂𝗻𝗸𝗶𝗻𝗴: the solution we've been waiting for. Here’s a quick evolution of chunking strategies: → 𝗧𝗿𝗮𝗱𝗶𝘁𝗶𝗼𝗻𝗮𝗹 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴 (the basics we all started with) • Token Chunking - split by token count • Sentence Chunking - split by sentence boundaries... • Document-Based Chunking - split by sections/paragraphs → 𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴 (when things got sophisticated) • Semantic Chunking - split by meaning • LLM-Based Chunking - let the model decide But each chunking method separates text at defined points, meaning context is lost within the document from one chunk to the next. → 𝗘𝗻𝘁𝗲𝗿 𝗟𝗮𝘁𝗲 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴 (the game changer) Traditional approach: Chunk first → Embed each chunk separately Late chunking approach: Embed the entire document → Then chunk with context preserved 𝗪𝗵𝘆 𝗰𝗵𝗼𝗼𝘀𝗲 𝗹𝗮𝘁𝗲 𝗰𝗵𝘂𝗻𝗸𝗶𝗻𝗴? When you chunk first, each piece loses its contextual relationship to the rest of the document. It's like reading a book by randomly picking paragraphs - you miss the flow. With late chunking, every chunk maintains awareness of its neighbors because the embedding happens at the document level first. Mean pooling is done on segments AFTER the full context is embedded. Jina AI tested and saw significant improvements in retrieval quality - chunks that were previously disconnected now maintain their semantic relationships. As documents get longer and context windows expand, late chunking might just become the new standard for high-quality retrieval systems. 𝗪𝗵𝗮𝘁 𝗱𝗼 𝘆𝗼𝘂 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗺𝗮𝗸𝗲 𝘁𝗵𝗶𝘀 𝘄𝗼𝗿𝗸? No modifications to your retrieval pipeline are needed. 1. Long context embedding models (8192+ tokens) 2. Chunking logic that tracks token spans 3. Less than 30 lines of code to implement All you need is to switch the order at which you chunk and embed. Embed FIRST, then chunk, not the other way around. Dive deeper into late chunking:show more

Femke Plantinga

13,725 subscribers

125,412 views • 1 year ago •via X (Twitter)

Science & Technology News & Politics Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

How do professional RAG applications chunk their text? Let’s cover some Advanced Chunking Techniques. In our latest video, we cover simple chunking methods like splitting documents into sentences or sections. But these methods often miss out on ensuring each chunk has independent meaning. Semantic chunking solved exactly this! By measuring the semantic similarity between sentences using vector embeddings, we can combine similar sentences into meaningful chunks. With LLM-based chunking, large language models help break down text effectively, although it can be slow and costly. And what about the newest Late Chunking? Which keeps context intact across chunks—more on that soon. 👀 In this video, we cover these advanced techniques in detail. Watch it to learn more. A big shoutout to Daniel Williams for helping create this video! 💚

How do professional RAG applications chunk their text? Let’s cover some Advanced Chunking Techniques. In our latest video, we cover simple chunking methods like splitting documents into sentences or sections. But these methods often miss out on ensuring each chunk has independent meaning. Semantic chunking solved exactly this! By measuring the semantic similarity between sentences using vector embeddings, we can combine similar sentences into meaningful chunks. With LLM-based chunking, large language models help break down text effectively, although it can be slow and costly. And what about the newest Late Chunking? Which keeps context intact across chunks—more on that soon. 👀 In this video, we cover these advanced techniques in detail. Watch it to learn more. A big shoutout to Daniel Williams for helping create this video! 💚

Femke Plantinga

29,675 views • 1 year ago

Researchers built a new RAG approach that: - does not need a vector DB. - does not embed data. - involves no chunking. - performs no similarity search. And it hit 98.7% accuracy on a financial benchmark (SOTA). Here's the core problem with RAG that this new approach solves: Traditional RAG chunks documents, embeds them into vectors, and retrieves based on semantic similarity. But similarity ≠ relevance. When you ask "What were the debt trends in 2023?", a vector search returns chunks that look similar. But the actual answer might be buried in some Appendix, referenced on some page, in a section that shares zero semantic overlap with your query. Traditional RAG would likely never find it. PageIndex (open-source) solves this. Instead of chunking and embedding, PageIndex builds a hierarchical tree structure from your documents, like an intelligent table of contents. Then it uses reasoning to traverse that tree. For instance, the model doesn't ask: "What text looks similar to this query?" Instead, it asks: "Based on this document's structure, where would a human expert look for this answer?" That's a fundamentally different approach with: - No arbitrary chunking that breaks context. - No vector DB infrastructure to maintain. - Traceable retrieval to see exactly why it chose a specific section. - The ability to see in-document references ("see Table 5.3") the way a human would. But here's the deeper issue that it solves. Vector search treats every query as independent. But documents have structure and logic, like sections that reference other sections and context that builds across pages. PageIndex respects that structure instead of flattening it into embeddings. Do note that this approach may not make sense in every use case since traditional vector search is still fast, simple, and works well for many applications. But for professional documents that require domain expertise and multi-step reasoning, this tree-based, reasoning-first approach shines. For instance, PageIndex achieved 98.7% accuracy on FinanceBench, significantly outperforming traditional vector-based RAG systems on complex financial document analysis. Everything is fully open-source, so you can see the full implementation in GitHub and try it yourself. I have shared the GitHub repo in the replies!

Researchers built a new RAG approach that: - does not need a vector DB. - does not embed data. - involves no chunking. - performs no similarity search. And it hit 98.7% accuracy on a financial benchmark (SOTA). Here's the core problem with RAG that this new approach solves: Traditional RAG chunks documents, embeds them into vectors, and retrieves based on semantic similarity. But similarity ≠ relevance. When you ask "What were the debt trends in 2023?", a vector search returns chunks that look similar. But the actual answer might be buried in some Appendix, referenced on some page, in a section that shares zero semantic overlap with your query. Traditional RAG would likely never find it. PageIndex (open-source) solves this. Instead of chunking and embedding, PageIndex builds a hierarchical tree structure from your documents, like an intelligent table of contents. Then it uses reasoning to traverse that tree. For instance, the model doesn't ask: "What text looks similar to this query?" Instead, it asks: "Based on this document's structure, where would a human expert look for this answer?" That's a fundamentally different approach with: - No arbitrary chunking that breaks context. - No vector DB infrastructure to maintain. - Traceable retrieval to see exactly why it chose a specific section. - The ability to see in-document references ("see Table 5.3") the way a human would. But here's the deeper issue that it solves. Vector search treats every query as independent. But documents have structure and logic, like sections that reference other sections and context that builds across pages. PageIndex respects that structure instead of flattening it into embeddings. Do note that this approach may not make sense in every use case since traditional vector search is still fast, simple, and works well for many applications. But for professional documents that require domain expertise and multi-step reasoning, this tree-based, reasoning-first approach shines. For instance, PageIndex achieved 98.7% accuracy on FinanceBench, significantly outperforming traditional vector-based RAG systems on complex financial document analysis. Everything is fully open-source, so you can see the full implementation in GitHub and try it yourself. I have shared the GitHub repo in the replies!

Avi Chawla

973,546 views • 7 months ago

New short course on sophisticated RAG (Retrieval Augmented Generation) techniques is out! Taught by Jerry Liu and Anupam Datta of LlamaIndex 🦙 and TruEra , this teaches advanced techniques that help your LLM generate good answers. Topics include: - Sentence-window retrieval, which retrieves not just the most relevant sentence, but a window of sentences around it for higher quality context. - Auto-merging retrieval, which organizes your document into a hierarchical tree structure, where each parent node's text is split among its child nodes. Based on the relevance of the child nodes to a user query, this lets you better decide whether the entire parent node should be provided as context to the LLM. - Evaluation methodology for separately evaluating the quality of the key steps of RAG (context relevance, answer relevance, groundedness) so that you can perform error analysis, identify which part of your pipeline needs work, and tune components systematically. Please check out the course!

New short course on sophisticated RAG (Retrieval Augmented Generation) techniques is out! Taught by Jerry Liu and Anupam Datta of LlamaIndex 🦙 and TruEra , this teaches advanced techniques that help your LLM generate good answers. Topics include: - Sentence-window retrieval, which retrieves not just the most relevant sentence, but a window of sentences around it for higher quality context. - Auto-merging retrieval, which organizes your document into a hierarchical tree structure, where each parent node's text is split among its child nodes. Based on the relevance of the child nodes to a user query, this lets you better decide whether the entire parent node should be provided as context to the LLM. - Evaluation methodology for separately evaluating the quality of the key steps of RAG (context relevance, answer relevance, groundedness) so that you can perform error analysis, identify which part of your pipeline needs work, and tune components systematically. Please check out the course!

Andrew Ng

655,661 views • 2 years ago

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. 2. The connector ecosystem to load data from unstructured data sources is very immature. 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. The goal of a RAG Pipeline is to solve these problems. The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. At a high level, there are four different stages in the architecture of a RAG pipeline: 1. Ingestion: Here is where the pipeline loads the information from the data source. 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. 3. Transform: Where the pipeline chunks the data and generates document embeddings. 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. There are different rabbit holes at each one of these stages. Here are three of them: 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. If you have a few documents lying around, set up a free account and give it a try.

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. 2. The connector ecosystem to load data from unstructured data sources is very immature. 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. The goal of a RAG Pipeline is to solve these problems. The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. At a high level, there are four different stages in the architecture of a RAG pipeline: 1. Ingestion: Here is where the pipeline loads the information from the data source. 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. 3. Transform: Where the pipeline chunks the data and generates document embeddings. 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. There are different rabbit holes at each one of these stages. Here are three of them: 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. If you have a few documents lying around, set up a free account and give it a try.

Santiago

40,627 views • 1 year ago

What started as building a personal taste.md skill for myself, turned into building a pipeline to create any taste as a skill. The most important piece is references. This is where you should spend time. If the references suck, so does the skill. I find that references cropped tightly on details in high resolution work the best. Each image gets analyzed by both Opus 4.7 and GPT 5.5. The analysis is based on why the reference is successful as a piece of design - not what it does functionally. Using two models helps rule out biases and gaps from each. The models focus on layout, spacing, typography, rhythm, composition, hierarchy, etc. At the end, each image has: reference-01/ - opus-4-7-analysis.md - gpt-5-5-analysis.md Then we fuse them together using GPT 5.5 - but the md files are anonymized so 5.5 doesn't prefer itself. reference-01/ - fused-analysis.md reference-02/ - fused-analysis.md etc. After fusion, we have one synthesized analysis per reference. Now the goal is to combine all of those into a single rule set. This is where chunking matters. If you ask one model to combine 100 image analyses at once, the result becomes too broad. It summarizes instead of preserving the granular design rules we want. Instead we chunk the fused analyses into smaller groups. Each group gets merged into a chunk-level synthesis, usually from around 6 to 8 image notes at a time. Then one final model pass fuses those chunks into a single md rule set. Finally, using the rule set, we write a skill of concrete instructions. It enforces constraints, uses imperative wording, and avoids vague taste words.

What started as building a personal taste.md skill for myself, turned into building a pipeline to create any taste as a skill. The most important piece is references. This is where you should spend time. If the references suck, so does the skill. I find that references cropped tightly on details in high resolution work the best. Each image gets analyzed by both Opus 4.7 and GPT 5.5. The analysis is based on why the reference is successful as a piece of design - not what it does functionally. Using two models helps rule out biases and gaps from each. The models focus on layout, spacing, typography, rhythm, composition, hierarchy, etc. At the end, each image has: reference-01/ - opus-4-7-analysis.md - gpt-5-5-analysis.md Then we fuse them together using GPT 5.5 - but the md files are anonymized so 5.5 doesn't prefer itself. reference-01/ - fused-analysis.md reference-02/ - fused-analysis.md etc. After fusion, we have one synthesized analysis per reference. Now the goal is to combine all of those into a single rule set. This is where chunking matters. If you ask one model to combine 100 image analyses at once, the result becomes too broad. It summarizes instead of preserving the granular design rules we want. Instead we chunk the fused analyses into smaller groups. Each group gets merged into a chunk-level synthesis, usually from around 6 to 8 image notes at a time. Then one final model pass fuses those chunks into a single md rule set. Finally, using the rule set, we write a skill of concrete instructions. It enforces constraints, uses imperative wording, and avoids vague taste words.

Jaytel

59,853 views • 3 months ago

⚡️We are excited to announce that our new no-code Enterprise Platform is NOW available in private beta! As RAG apps advance from prototype to production we’ve been overwhelmed by requests for an enterprise grade solution to provide these applications with the data they need. Designed to make it easy to get your data #RAGready, our Platform can preprocess more than 25 file types and soon will be fully #multimodal, also able to ingest audio, video and image files. We ship with a baseline suite of source connectors, including Amazon Web Services S3, Microsoft Azure Blob Storage, OneDrive, SFTP, Databricks Delta Table, Google Drive, Salesforce, Elastic, OpenSearch, and Google Cloud storage with many more fast following. Platform transforms your documents into a standardized JSON schema, broken down into semantically coherent elements allowing you to reconstruct your document in the manner most useful to you. Want only the narrative text but not the headers and footers? This is entirely configurable through the UI. Additionally, we generate more than 30 types of metadata for each element to make it easy to curate the data being written downstream and to support metadata filtering during retrieval. Smart chunking and the ability to choose from a range of embedding models are in from launch, delivering a turnkey solution for chunk and embedding experimentation. As for destination connectors, we've got that covered too, with Amazon Web Services S3, Pinecone, Chroma , Weaviate AI Database, Google Cloud storage, MongoDB, Microsoft Azure cognitive search, PostgreSQL, Elastic, OpenSearch, and Databricks Delta Table. And of course, all of this can be scheduled to keep your data continuously hydrated. The private-beta is live today! Sign-up to get access and come build the future of LLM data foundations with us: 🚀 #ETLforLLMs #AI #DataPreprocessing #DataScience #DataTransformation #LLMs #ETL #ML #PreppingData #MachineLearning #RAG #Engineer #Unstructured #Unstructuredio #RetrievalAugmentedGeneration #multimodal #AIJobs

⚡️We are excited to announce that our new no-code Enterprise Platform is NOW available in private beta! As RAG apps advance from prototype to production we’ve been overwhelmed by requests for an enterprise grade solution to provide these applications with the data they need. Designed to make it easy to get your data #RAGready, our Platform can preprocess more than 25 file types and soon will be fully #multimodal, also able to ingest audio, video and image files. We ship with a baseline suite of source connectors, including Amazon Web Services S3, Microsoft Azure Blob Storage, OneDrive, SFTP, Databricks Delta Table, Google Drive, Salesforce, Elastic, OpenSearch, and Google Cloud storage with many more fast following. Platform transforms your documents into a standardized JSON schema, broken down into semantically coherent elements allowing you to reconstruct your document in the manner most useful to you. Want only the narrative text but not the headers and footers? This is entirely configurable through the UI. Additionally, we generate more than 30 types of metadata for each element to make it easy to curate the data being written downstream and to support metadata filtering during retrieval. Smart chunking and the ability to choose from a range of embedding models are in from launch, delivering a turnkey solution for chunk and embedding experimentation. As for destination connectors, we've got that covered too, with Amazon Web Services S3, Pinecone, Chroma , Weaviate AI Database, Google Cloud storage, MongoDB, Microsoft Azure cognitive search, PostgreSQL, Elastic, OpenSearch, and Databricks Delta Table. And of course, all of this can be scheduled to keep your data continuously hydrated. The private-beta is live today! Sign-up to get access and come build the future of LLM data foundations with us: 🚀 #ETLforLLMs #AI #DataPreprocessing #DataScience #DataTransformation #LLMs #ETL #ML #PreppingData #MachineLearning #RAG #Engineer #Unstructured #Unstructuredio #RetrievalAugmentedGeneration #multimodal #AIJobs

Unstructured

21,911 views • 2 years ago

RLM is the most import foundation of my Pi Harness (other than Pi of course). It's seeded with late interaction retrieval results (thanks to @lightonai for pylate). The Agent initiates it with query then.. 𝐒𝐞𝐭𝐮𝐩 A python REPL is created and seeded with: 1. Late interaction search to pre-filter. Instead of doing top 3/5/10, it's top hundreds of documents. This is set into a `context` variable. 2. Python functions are loaded in to do more searches if `context` variable isn't enough. And to make llm calls with cheaper models in parallel batches. 𝐈𝐭𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐋𝐨𝐨𝐩 From there, an LLM iterates in the REPL based on the query. It's just like exploring in a jupyter notebook. The LLM writes prose (like a markdown cell) and code to be run in the REPL each turn. This allows the LLM to sort, filter, and synthesize information. It can fan out and ask smaller models to summarize, combine, contrast, or do anything else to documents to help it understand the data. After several turns the LLM reponds with the final answer. Either because it found the answer, or hit the budget limit. Context as a Python variable, LLM as the programmer, REPL as the runtime. 𝐖𝐡𝐲 𝐃𝐨𝐞𝐬 𝐓𝐡𝐢𝐬 𝐖𝐨𝐫𝐤 1. Richer Shell. Agents (and subagents) work by intermixing code and prose/thinking. But they use static scripts or bash that run and exit and start over each tool call. That's not ideal for exploration and synthesis of data. For that, state is useful to continue building and exploring the data as you learn more. There's a reason jupyter notebooks have been popular with data scientists. 2. Keeps main agent context clean. The better context you have the better the agent will perform (duh!). This means three thing: better human input, less missing search results, and less incorrect search results. Letting the agent iterate allows it to synthesize just what is needed and nothing else. All bad paths or peeks at something that turns out to be irrelevant stays out of main agent context. 3. Stack the good ideas! People often compare late interaction search vs RLM. Or static vs dynamic languages. Or agentic search vs semantic search. But...You can just use them all together for what they're each good at. Use them all for the area they're really great for. Read the full post which has more detail about how and why.

RLM is the most import foundation of my Pi Harness (other than Pi of course). It's seeded with late interaction retrieval results (thanks to @lightonai for pylate). The Agent initiates it with query then.. 𝐒𝐞𝐭𝐮𝐩 A python REPL is created and seeded with: 1. Late interaction search to pre-filter. Instead of doing top 3/5/10, it's top hundreds of documents. This is set into a `context` variable. 2. Python functions are loaded in to do more searches if `context` variable isn't enough. And to make llm calls with cheaper models in parallel batches. 𝐈𝐭𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐋𝐨𝐨𝐩 From there, an LLM iterates in the REPL based on the query. It's just like exploring in a jupyter notebook. The LLM writes prose (like a markdown cell) and code to be run in the REPL each turn. This allows the LLM to sort, filter, and synthesize information. It can fan out and ask smaller models to summarize, combine, contrast, or do anything else to documents to help it understand the data. After several turns the LLM reponds with the final answer. Either because it found the answer, or hit the budget limit. Context as a Python variable, LLM as the programmer, REPL as the runtime. 𝐖𝐡𝐲 𝐃𝐨𝐞𝐬 𝐓𝐡𝐢𝐬 𝐖𝐨𝐫𝐤 1. Richer Shell. Agents (and subagents) work by intermixing code and prose/thinking. But they use static scripts or bash that run and exit and start over each tool call. That's not ideal for exploration and synthesis of data. For that, state is useful to continue building and exploring the data as you learn more. There's a reason jupyter notebooks have been popular with data scientists. 2. Keeps main agent context clean. The better context you have the better the agent will perform (duh!). This means three thing: better human input, less missing search results, and less incorrect search results. Letting the agent iterate allows it to synthesize just what is needed and nothing else. All bad paths or peeks at something that turns out to be irrelevant stays out of main agent context. 3. Stack the good ideas! People often compare late interaction search vs RLM. Or static vs dynamic languages. Or agentic search vs semantic search. But...You can just use them all together for what they're each good at. Use them all for the area they're really great for. Read the full post which has more detail about how and why.

Isaac Flath

42,620 views • 4 months ago

I cant believe this guy just made a permanent solution to context bloat and open sourced it all! when we tested this tool (Context+) for solving an issue on the OpenCode repository, the agent using this tool used ~6.5k fewer tokens, found the code and fixed it in half the time! the results were surprising: 6 to 10k tokens saved per prompt, completed task in ~2 minutes while the agent running without the tool took ~4 mins for the same and got stuck in loops bro built an entire beast by using all the modern tools that we could think of: undo trees, semantic search by meaning (by haskellforall), advanced refactoring, blast radius, advanced file context trees, restore points... i can keep going on semantic code search and context trees are the future of agentic coding and this tool proves it the feature i loved the most is semantic search and how it gets things done 2x faster with least possible tokens it makes an agent that actually knows what it’s doing and not just guessing, it makes meaning from your code similar to RAG. if you aren't optimizing your context, you are just burning money the developer says this tool is still under development, it can have unexpected behavior and the docs need updates but the video shows the reality of how fast it can be github: get here:

I cant believe this guy just made a permanent solution to context bloat and open sourced it all! when we tested this tool (Context+) for solving an issue on the OpenCode repository, the agent using this tool used ~6.5k fewer tokens, found the code and fixed it in half the time! the results were surprising: 6 to 10k tokens saved per prompt, completed task in ~2 minutes while the agent running without the tool took ~4 mins for the same and got stuck in loops bro built an entire beast by using all the modern tools that we could think of: undo trees, semantic search by meaning (by haskellforall), advanced refactoring, blast radius, advanced file context trees, restore points... i can keep going on semantic code search and context trees are the future of agentic coding and this tool proves it the feature i loved the most is semantic search and how it gets things done 2x faster with least possible tokens it makes an agent that actually knows what it’s doing and not just guessing, it makes meaning from your code similar to RAG. if you aren't optimizing your context, you are just burning money the developer says this tool is still under development, it can have unexpected behavior and the docs need updates but the video shows the reality of how fast it can be github: get here:

forloop

226,271 views • 6 months ago

How can you solve complex tasks using a Large Language Model? Here is a 2-minute introduction to everything you need to know to 10x the quality of your results. Let's talk about three techniques, in order of complexity, starting with the easiest one: • In-Context Learning • Indexing + In-Context Learning • Fine-tuning In-Context Learning The team that trained GPT-3 found something they couldn't explain: You can condition a model using examples of how you want it to behave. I included an example prompt in the attached video. You can "teach" the model how you want it to interpret questions, select the correct answers, and format the results by giving a few examples. You can also give specific knowledge to the model that will be helpful when formulating answers. We call this approach "grounding the model." There's another example in the video. Indexing + In-Context Learning Unfortunately, there is a limit to how much data you can include in a prompt. We call this the "context size." One version of GPT-4 supports a context of approximately 6,000 words, while the other supports 25,000 words. Although this sounds like a lot, many applications need more than that. Imagine you wrote a book and want to build an application to answer any questions about your story. What happens if your book is longer than the context? That's where Indexing comes in. Using a model, you can turn every book passage into an embedding. These are vectors, numbers that "encode" the passage's text. You can then store these embeddings in a particular database that supports fast retrieval of these vectors. You can then turn any question into an embedding and search the database for the list of passages that are similar to that query. Instead of using the entire book to ask the model, you can now use the relevant passages as in-context information, effectively working around the context size limitation. Fine-tuning Fine-tuning can give you an extra boost to get reliable outputs from your LLM. It is, however, the most complex approach on the list. There are different approaches to fine-tuning a model with your data. A popular technique is to process your data with your LLM and use the outputs to train a new classifier that solves your specific task. Notice that here you aren't modifying the LLM. Instead, you are chaining it with your trained classifier. Another approach is to modify the parameters of the LLM using your data. Think of this as "rewiring" the model in a way that solves your particular task. The results and costs will vary depending on how many layers you want to fine-tune from the original model. Many companies think that fine-tuning is the solution to their problems. In my experience, many will benefit from exploring the other two approaches. I love explaining Machine Learning and Artificial Intelligence ideas. If you enjoy in-depth content like this, follow me Santiago so you don't miss what comes next.

How can you solve complex tasks using a Large Language Model? Here is a 2-minute introduction to everything you need to know to 10x the quality of your results. Let's talk about three techniques, in order of complexity, starting with the easiest one: • In-Context Learning • Indexing + In-Context Learning • Fine-tuning In-Context Learning The team that trained GPT-3 found something they couldn't explain: You can condition a model using examples of how you want it to behave. I included an example prompt in the attached video. You can "teach" the model how you want it to interpret questions, select the correct answers, and format the results by giving a few examples. You can also give specific knowledge to the model that will be helpful when formulating answers. We call this approach "grounding the model." There's another example in the video. Indexing + In-Context Learning Unfortunately, there is a limit to how much data you can include in a prompt. We call this the "context size." One version of GPT-4 supports a context of approximately 6,000 words, while the other supports 25,000 words. Although this sounds like a lot, many applications need more than that. Imagine you wrote a book and want to build an application to answer any questions about your story. What happens if your book is longer than the context? That's where Indexing comes in. Using a model, you can turn every book passage into an embedding. These are vectors, numbers that "encode" the passage's text. You can then store these embeddings in a particular database that supports fast retrieval of these vectors. You can then turn any question into an embedding and search the database for the list of passages that are similar to that query. Instead of using the entire book to ask the model, you can now use the relevant passages as in-context information, effectively working around the context size limitation. Fine-tuning Fine-tuning can give you an extra boost to get reliable outputs from your LLM. It is, however, the most complex approach on the list. There are different approaches to fine-tuning a model with your data. A popular technique is to process your data with your LLM and use the outputs to train a new classifier that solves your specific task. Notice that here you aren't modifying the LLM. Instead, you are chaining it with your trained classifier. Another approach is to modify the parameters of the LLM using your data. Think of this as "rewiring" the model in a way that solves your particular task. The results and costs will vary depending on how many layers you want to fine-tune from the original model. Many companies think that fine-tuning is the solution to their problems. In my experience, many will benefit from exploring the other two approaches. I love explaining Machine Learning and Artificial Intelligence ideas. If you enjoy in-depth content like this, follow me Santiago so you don't miss what comes next.

Santiago

384,510 views • 3 years ago

There’s been two papers released in the past couple months, one by Google and one by NVIDIA, that argue that ordering the documents retrieved by RAG systems can enhance performance. However, they both give two different strategies on HOW these documents should be ordered 🤔 Both papers agree on two main points: 1️⃣ There’s a fundamental issue in RAG - as more documents are retrieved, more irrelevant context (e.g., hard negatives) are introduced, which leads to confusion for the LLM and eventually degrades the quality of the generated output. This is called an inverted-U performance curve. 2️⃣ Ordering the retrieved documents is a key lever for optimizing RAG performance. Google Cloud researchers proposed ordering results based on relevance scores: The authors in this paper argue for relevance-based reordering, or ordering the retrieved chunks based on their similarity scores, so the most relevant documents are at the beginning and the end of the inputs to counter the “lost in the middle” effect. NVIDIA researchers proposed ordering results based on the original sequence of document chunks: The authors of this paper argue for Order-Preserving Reordering, or Order-Preserve RAG (OP-RAG), to maintain the logically coherent content flow of the document. So they preserved the original order of retrieved document chunks in the source text, instead of ranking them by relevance scores. So which one is right? It probably depends on the specific use case and dataset - relevance-based reordering could perform better in tasks where you need fast access to the most critical information (e.g., fact retrieval, QA systems), while order-preserving RAG might be better where you need to understand the sequential structure of information (e.g., narrative or legal documents). There are still so many uncertainties in AI - we don’t actually know what we’re doing, and it takes awhile to figure out the best strategies for most things! Excited to see more research about this.

There’s been two papers released in the past couple months, one by Google and one by NVIDIA, that argue that ordering the documents retrieved by RAG systems can enhance performance. However, they both give two different strategies on HOW these documents should be ordered 🤔 Both papers agree on two main points: 1️⃣ There’s a fundamental issue in RAG - as more documents are retrieved, more irrelevant context (e.g., hard negatives) are introduced, which leads to confusion for the LLM and eventually degrades the quality of the generated output. This is called an inverted-U performance curve. 2️⃣ Ordering the retrieved documents is a key lever for optimizing RAG performance. Google Cloud researchers proposed ordering results based on relevance scores: The authors in this paper argue for relevance-based reordering, or ordering the retrieved chunks based on their similarity scores, so the most relevant documents are at the beginning and the end of the inputs to counter the “lost in the middle” effect. NVIDIA researchers proposed ordering results based on the original sequence of document chunks: The authors of this paper argue for Order-Preserving Reordering, or Order-Preserve RAG (OP-RAG), to maintain the logically coherent content flow of the document. So they preserved the original order of retrieved document chunks in the source text, instead of ranking them by relevance scores. So which one is right? It probably depends on the specific use case and dataset - relevance-based reordering could perform better in tasks where you need fast access to the most critical information (e.g., fact retrieval, QA systems), while order-preserving RAG might be better where you need to understand the sequential structure of information (e.g., narrative or legal documents). There are still so many uncertainties in AI - we don’t actually know what we’re doing, and it takes awhile to figure out the best strategies for most things! Excited to see more research about this.

Victoria Slocum

15,333 views • 1 year ago

IT WORKS! Demo time 🥳 (next 10x productivity bump is here?) Suppose you must refactor a large codebase, e.g.: > "use I32 instead of U32 as the native number type" That task, by itself, is simple enough to be automated by Sonnet alone, right? Problem is: what if the codebase is too large to fit on the AI's context, and RAG solutions / editors like Cursor can't precisely isolate which parts require edit (because it needs some semantic reasoning)? What now? Do we fallback to human coding, as did the old Mayans? Of course not! We just split the codebase in chunks, and send each chunk to ~500 DeepSeek instances in parallel (!). Each instance then decides if given block should be edited or not. This is viable because DeepSeek is smart, fast and really cheap. Finally, we send the blocks that need edit to Sonnet, in a single prompt, and let it work normally. The entire process takes less than a minute, is cheap enough, and very effective. We conclude with a 'git diff' for manual review. In short, AI has came to grep search & replace. Since this script feels like dropping a bomb in your codebase, I'm calling it AoE, and it is available on VictorTaelin/AI-Scripts. Enjoy! Here's the demo:

IT WORKS! Demo time 🥳 (next 10x productivity bump is here?) Suppose you must refactor a large codebase, e.g.: > "use I32 instead of U32 as the native number type" That task, by itself, is simple enough to be automated by Sonnet alone, right? Problem is: what if the codebase is too large to fit on the AI's context, and RAG solutions / editors like Cursor can't precisely isolate which parts require edit (because it needs some semantic reasoning)? What now? Do we fallback to human coding, as did the old Mayans? Of course not! We just split the codebase in chunks, and send each chunk to ~500 DeepSeek instances in parallel (!). Each instance then decides if given block should be edited or not. This is viable because DeepSeek is smart, fast and really cheap. Finally, we send the blocks that need edit to Sonnet, in a single prompt, and let it work normally. The entire process takes less than a minute, is cheap enough, and very effective. We conclude with a 'git diff' for manual review. In short, AI has came to grep search & replace. Since this script feels like dropping a bomb in your codebase, I'm calling it AoE, and it is available on VictorTaelin/AI-Scripts. Enjoy! Here's the demo:

Taelin

631,272 views • 1 year ago

Microsoft did it again! Speech AI models have a major limitation. They slice long recordings into tiny chunks, lose track of who's speaking, and forget all context halfway through. This is exactly what Microsoft's VibeVoice solves. It's an open-source family of frontier voice AI models for both speech recognition and speech generation. Here's what it can do: > VibeVoice-ASR processes up to 60 minutes of audio in a single pass. No chunking. It outputs structured transcriptions with who spoke, when they spoke, and what they said. > You can feed it custom hotwords like names, technical jargon, or domain-specific terms. The model uses them to significantly improve accuracy on specialized content. > VibeVoice-TTS generates up to 90 minutes of multi-speaker speech with up to 4 distinct speakers. Natural turn-taking, emotional expression, all in one pass. > VibeVoice-Realtime is a 0.5B streaming TTS model with ~300ms first-audio latency. Small enough to deploy practically anywhere. All of this is powered by continuous speech tokenizers running at just 7.5 Hz. This ultra-low frame rate preserves audio quality while making long sequences computationally feasible. I have shared the link to the GitHub repo in the replies!

Microsoft did it again! Speech AI models have a major limitation. They slice long recordings into tiny chunks, lose track of who's speaking, and forget all context halfway through. This is exactly what Microsoft's VibeVoice solves. It's an open-source family of frontier voice AI models for both speech recognition and speech generation. Here's what it can do: > VibeVoice-ASR processes up to 60 minutes of audio in a single pass. No chunking. It outputs structured transcriptions with who spoke, when they spoke, and what they said. > You can feed it custom hotwords like names, technical jargon, or domain-specific terms. The model uses them to significantly improve accuracy on specialized content. > VibeVoice-TTS generates up to 90 minutes of multi-speaker speech with up to 4 distinct speakers. Natural turn-taking, emotional expression, all in one pass. > VibeVoice-Realtime is a 0.5B streaming TTS model with ~300ms first-audio latency. Small enough to deploy practically anywhere. All of this is powered by continuous speech tokenizers running at just 7.5 Hz. This ultra-low frame rate preserves audio quality while making long sequences computationally feasible. I have shared the link to the GitHub repo in the replies!

Akshay 🚀

45,256 views • 5 months ago

The latest RAG trend for the current agent harnesses (Codex, Cowork) is to do two passes of document processing to solve a knowledge work task over a data room of documents: 1️⃣ A fast and light pass, oftentimes using a free/OSS doc parsing tool. This can be cheaply run across 10-100-1k’s of files, and enables the agent to then do retrieval (e.g. grep, semantic) to find relevant subsets of context. 2️⃣ A “just-in-time” VLM-based pass. Once the agent finds the relevant pages of context, it will screenshot the documents can call its own VLM (or write code) to dissect the pages. The issue with only using VLM-based OCR tools over massive ad-hoc customer file dumps is that it’s slow and expensive. Doing JIT VLM OCR allows the agent to filter through the data cheaply, but still preserve accuracy for the context that’s needed for the task. The agent harnesses do two-pass document processing by default using off the shelf-tools: pdf2text as the first pass, and using itself (Opus 5) as the second pass. See the below video where Cowork runs over a bunch of PDFs to answer a question about a benchmark graph in the Kimi k3 paper. The main issues here with the “out of the box” doc processing these agents offer are: * Opus 5 is not the best VLM for OCR. It is also way too expensive at scale and lacks grounding * The OSS tools like pypdf, pdf2text, may not be versatile enough as the first pass. * The agent will write a lot of throwaway code to rewrite things an OCR tool would’ve provided out of the box, like chart processing, bounding boxes, confidence scores, leading to increased cost and speed. We have all the tools within LlamaIndex 🦙 to help any agent do two-pass document processing with higher accuracy and lower cost. 1️⃣ We have liteparse for the first pass - a free/OSS parser written in Rust that’s faster/more accurate than other OSS parsers, and supports 50+ document types 2️⃣ We have LlamaParse for the second pass - an agentic document engine that uses VLMs+harnesses to achieve SOTA in accuracy and cost across various doc parsing and extraction tasks. It can be called from any agent harness as an MCP or skill. It takes in page numbers as input, so that the agent can choose to run LlamaParse over a subset of the doc instead of the full doc as a “zoom-in” pass. Come check it out! LiteParse: LlamaParse: All the relevant docs, including MCP, are here:

The latest RAG trend for the current agent harnesses (Codex, Cowork) is to do two passes of document processing to solve a knowledge work task over a data room of documents: 1️⃣ A fast and light pass, oftentimes using a free/OSS doc parsing tool. This can be cheaply run across 10-100-1k’s of files, and enables the agent to then do retrieval (e.g. grep, semantic) to find relevant subsets of context. 2️⃣ A “just-in-time” VLM-based pass. Once the agent finds the relevant pages of context, it will screenshot the documents can call its own VLM (or write code) to dissect the pages. The issue with only using VLM-based OCR tools over massive ad-hoc customer file dumps is that it’s slow and expensive. Doing JIT VLM OCR allows the agent to filter through the data cheaply, but still preserve accuracy for the context that’s needed for the task. The agent harnesses do two-pass document processing by default using off the shelf-tools: pdf2text as the first pass, and using itself (Opus 5) as the second pass. See the below video where Cowork runs over a bunch of PDFs to answer a question about a benchmark graph in the Kimi k3 paper. The main issues here with the “out of the box” doc processing these agents offer are: * Opus 5 is not the best VLM for OCR. It is also way too expensive at scale and lacks grounding * The OSS tools like pypdf, pdf2text, may not be versatile enough as the first pass. * The agent will write a lot of throwaway code to rewrite things an OCR tool would’ve provided out of the box, like chart processing, bounding boxes, confidence scores, leading to increased cost and speed. We have all the tools within LlamaIndex 🦙 to help any agent do two-pass document processing with higher accuracy and lower cost. 1️⃣ We have liteparse for the first pass - a free/OSS parser written in Rust that’s faster/more accurate than other OSS parsers, and supports 50+ document types 2️⃣ We have LlamaParse for the second pass - an agentic document engine that uses VLMs+harnesses to achieve SOTA in accuracy and cost across various doc parsing and extraction tasks. It can be called from any agent harness as an MCP or skill. It takes in page numbers as input, so that the agent can choose to run LlamaParse over a subset of the doc instead of the full doc as a “zoom-in” pass. Come check it out! LiteParse: LlamaParse: All the relevant docs, including MCP, are here:

Jerry Liu

22,763 views • 21 days ago

Assumptions about the new "Can More" ChatGPT tool were right - ChatGPT is introducing own take on Claude Artifacts - code & document writing tools with persisted text documents, history revisions (restore previous version), edits and comments (probably used to apply suggested edits) New document symbol in the top navigation shows how many documents you have and allows you to open a resizable canvas to edit them in split view - your ChatGPT conversation on the left side and canvas on the right side, but the code/documents can also be accessed in fullscreen view The canvas is built using ProseMirror (open source WYSIWYM editor) and has an inline action to "Ask ChatGPT" (explain or make edits) for your document and code plus document formatting tools (like bold, italic, font style, etc.) But in addition to that, there are also special action shortcuts for documents and code, with an interesting decision to use sliders for the selection of the desired outcome For Documents - Suggest edits ("How can I improve this. Leave as few comments as possible, but add a few more comments if the text is long. DO NOT leave more than 5 comments. You can reply that you added comments and suggestions to help improve the writing quality, but do not mention the prompt.") - Add emojis ("Replace as many words as possible with emojis.") - Add final polish ("Add some final polish to the text. If relevant, add a large title or any section titles. Check grammar and mechanics, make sure everything is consistent and reads well. You can reply that you added some final polish and checked for grammar, but do not mention the prompt.") - Reading level (Graduate School - "Rewrite this text at the reading level of a doctoral writer in this subject. You may reply that you adjusted the text to reflect a graduate school reading level, but do not mention the prompt", College - "Rewrite this text at the reading level of a college student majoring in this subject", High School - "Rewrite this text at the reading level of a high school student who has taken a couple of classes in this subject.", Keep current reading level, Middle School - "Rewrite this text at the reading level of a middle schooler.", Kindergarten - "Rewrite this text at the reading level of a kindergartener.") - Adjust the length (Longest - "Make this text 75% longer.", Longer - "Make this text 50% longer.", Keep current length, Shorter - "Make this text 50% shorter.", Shortest - "Make this text 75% shorter.") For Code - Code review ("Search for bugs and opportunities to improve the code—for example, ways that performance or code structure could be improved. Leave as few comments as possible, but add more comments if the text is long. DO NOT leave more than 5 comments. You may reply that you reviewed the code and left suggestions to improve the coding quality, but do not mention the prompt.") - Add comments ("Add inline code comments to explain the code, especially parts that are more complex. Make sure to rewrite all the code. You may reply that you added inline comments, but do not mention the prompt.") - Add logs ("Insert logs/print statements in the code that will help debug its behavior. Do not make any other changes to the code.") - Fix bugs ("Find any bugs and rewrite all the code to fix the bugs. Do not leave comments. If there are no bugs, reply that you reviewed the code and found no bugs.") - Port to a language ("Port to a language. Create a new document that rewrites the code in ..." - PHP, C++, Python, Keep current code. No changes will be made, JavaScript, TypeScript, Java) - Suggest edits ("How can I improve this. Leave as few comments as possible, but add a few more comments if the text is long. DO NOT leave more than 5 comments. You can reply that you added comments and suggestions to help improve the writing quality, but do not mention the prompt.")

Assumptions about the new "Can More" ChatGPT tool were right - ChatGPT is introducing own take on Claude Artifacts - code & document writing tools with persisted text documents, history revisions (restore previous version), edits and comments (probably used to apply suggested edits) New document symbol in the top navigation shows how many documents you have and allows you to open a resizable canvas to edit them in split view - your ChatGPT conversation on the left side and canvas on the right side, but the code/documents can also be accessed in fullscreen view The canvas is built using ProseMirror (open source WYSIWYM editor) and has an inline action to "Ask ChatGPT" (explain or make edits) for your document and code plus document formatting tools (like bold, italic, font style, etc.) But in addition to that, there are also special action shortcuts for documents and code, with an interesting decision to use sliders for the selection of the desired outcome For Documents - Suggest edits ("How can I improve this. Leave as few comments as possible, but add a few more comments if the text is long. DO NOT leave more than 5 comments. You can reply that you added comments and suggestions to help improve the writing quality, but do not mention the prompt.") - Add emojis ("Replace as many words as possible with emojis.") - Add final polish ("Add some final polish to the text. If relevant, add a large title or any section titles. Check grammar and mechanics, make sure everything is consistent and reads well. You can reply that you added some final polish and checked for grammar, but do not mention the prompt.") - Reading level (Graduate School - "Rewrite this text at the reading level of a doctoral writer in this subject. You may reply that you adjusted the text to reflect a graduate school reading level, but do not mention the prompt", College - "Rewrite this text at the reading level of a college student majoring in this subject", High School - "Rewrite this text at the reading level of a high school student who has taken a couple of classes in this subject.", Keep current reading level, Middle School - "Rewrite this text at the reading level of a middle schooler.", Kindergarten - "Rewrite this text at the reading level of a kindergartener.") - Adjust the length (Longest - "Make this text 75% longer.", Longer - "Make this text 50% longer.", Keep current length, Shorter - "Make this text 50% shorter.", Shortest - "Make this text 75% shorter.") For Code - Code review ("Search for bugs and opportunities to improve the code—for example, ways that performance or code structure could be improved. Leave as few comments as possible, but add more comments if the text is long. DO NOT leave more than 5 comments. You may reply that you reviewed the code and left suggestions to improve the coding quality, but do not mention the prompt.") - Add comments ("Add inline code comments to explain the code, especially parts that are more complex. Make sure to rewrite all the code. You may reply that you added inline comments, but do not mention the prompt.") - Add logs ("Insert logs/print statements in the code that will help debug its behavior. Do not make any other changes to the code.") - Fix bugs ("Find any bugs and rewrite all the code to fix the bugs. Do not leave comments. If there are no bugs, reply that you reviewed the code and found no bugs.") - Port to a language ("Port to a language. Create a new document that rewrites the code in ..." - PHP, C++, Python, Keep current code. No changes will be made, JavaScript, TypeScript, Java) - Suggest edits ("How can I improve this. Leave as few comments as possible, but add a few more comments if the text is long. DO NOT leave more than 5 comments. You can reply that you added comments and suggestions to help improve the writing quality, but do not mention the prompt.")

Tibor Blaho

136,084 views • 1 year ago

Kings of the Road (1976, Wenders) "'Kings of the Road' (1976) is a story that you're not often told in films about men. The story of the absence of women, which is at the same time the story of the longing for their presence!" --- Wim Wenders Full Excerpt: "'Kings of the Road' (1976) is the story of two men, but it doesn't take a Hollywood approach to the subject. American films about men - especially recent ones - are exercises in suppression: the men's true relationships with women, or with each other, are displaced by story, action and the need to entertain. They leave out the real nub: why the men prefer to be together, why they get on with each other, why they don't get on with women, or, if they do, then only as a pastime. My film is about precisely that: two men getting on together, each preferring the other's company to that of a woman. You get to see the shortcomings of both of them, their emotional insecurity; you see them trying to be mutually supportive and to hide their faults. But with the passage of time they're no longer bothered by these faults, and when they know each other well enough they begin discussing them. As a consequence of that, they split up. They split up because, on their journey across Germany, they've suddenly grown too close. It's a story that you're not often told in films about men. The story of the absence of women, which is at the same time the story of the longing for their presence!" ('The Logic of Images, Essays and Conversations, Wim Wenders, 1988)

Kings of the Road (1976, Wenders) "'Kings of the Road' (1976) is a story that you're not often told in films about men. The story of the absence of women, which is at the same time the story of the longing for their presence!" --- Wim Wenders Full Excerpt: "'Kings of the Road' (1976) is the story of two men, but it doesn't take a Hollywood approach to the subject. American films about men - especially recent ones - are exercises in suppression: the men's true relationships with women, or with each other, are displaced by story, action and the need to entertain. They leave out the real nub: why the men prefer to be together, why they get on with each other, why they don't get on with women, or, if they do, then only as a pastime. My film is about precisely that: two men getting on together, each preferring the other's company to that of a woman. You get to see the shortcomings of both of them, their emotional insecurity; you see them trying to be mutually supportive and to hide their faults. But with the passage of time they're no longer bothered by these faults, and when they know each other well enough they begin discussing them. As a consequence of that, they split up. They split up because, on their journey across Germany, they've suddenly grown too close. It's a story that you're not often told in films about men. The story of the absence of women, which is at the same time the story of the longing for their presence!" ('The Logic of Images, Essays and Conversations, Wim Wenders, 1988)

DepressedBergman

23,433 views • 7 months ago

‘pip install elysia’ and ‘elysia start’ That’s literally all it takes to get the most advanced open source agentic RAG app running on your data. We just released 𝗘𝗹𝘆𝘀𝗶𝗮, our open source, agentic RAG framework and an app so cool needed a cool video to go with it. Watch the full video: In the video, we go through these components of Elysia: 1️⃣ 𝗗𝗲𝗰𝗶𝘀𝗶𝗼𝗻 𝗧𝗿𝗲𝗲 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲: Instead of giving agents access to all tools at once, Elysia uses a pre-defined web of nodes with corresponding actions. Each decision agent has global context awareness. 2️⃣ 𝗗𝘆𝗻𝗮𝗺𝗶𝗰 𝗗𝗮𝘁𝗮 𝗗𝗶𝘀𝗽𝗹𝗮𝘆𝘀: Seven different data display formats including tables, e-commerce product cards, GitHub tickets, and charts. The system automatically choses the best display format. 3️⃣ 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗰 𝗗𝗮𝘁𝗮 𝗘𝘅𝗽𝗲𝗿𝘁𝗶𝘀𝗲: Unlike naive RAG systems that perform blind vector searches, Elysia analyzes your collections to understand data structure and meaning before performing queries. 𝗢𝘁𝗵𝗲𝗿 𝗖𝗼𝗼𝗹 𝗙𝗲𝗮𝘁𝘂𝗿𝗲𝘀: • 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸 𝗦𝘆𝘀𝘁𝗲𝗺: Uses positive examples as few-shot demonstrations for smaller, faster models • 𝗖𝗵𝘂𝗻𝗸-𝗢𝗻-𝗗𝗲𝗺𝗮𝗻𝗱: Dynamically chunks documents at query time instead of pre-chunking • 𝗠𝘂𝗹𝘁𝗶-𝗠𝗼𝗱𝗲𝗹 𝗦𝘁𝗿𝗮𝘁𝗲𝗴𝘆: Routes different tasks to appropriate model sizes based on complexity …And also how to get started with your own data! The entire project is open source and designed with customization in mind. You can use it as-is for effective data searching, or install the Python package to create custom tools for whatever agentic AI purposes you need. Big kudos to Edward for the vision, filming, and editing this masterpiece

‘pip install elysia’ and ‘elysia start’ That’s literally all it takes to get the most advanced open source agentic RAG app running on your data. We just released 𝗘𝗹𝘆𝘀𝗶𝗮, our open source, agentic RAG framework and an app so cool needed a cool video to go with it. Watch the full video: In the video, we go through these components of Elysia: 1️⃣ 𝗗𝗲𝗰𝗶𝘀𝗶𝗼𝗻 𝗧𝗿𝗲𝗲 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲: Instead of giving agents access to all tools at once, Elysia uses a pre-defined web of nodes with corresponding actions. Each decision agent has global context awareness. 2️⃣ 𝗗𝘆𝗻𝗮𝗺𝗶𝗰 𝗗𝗮𝘁𝗮 𝗗𝗶𝘀𝗽𝗹𝗮𝘆𝘀: Seven different data display formats including tables, e-commerce product cards, GitHub tickets, and charts. The system automatically choses the best display format. 3️⃣ 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗰 𝗗𝗮𝘁𝗮 𝗘𝘅𝗽𝗲𝗿𝘁𝗶𝘀𝗲: Unlike naive RAG systems that perform blind vector searches, Elysia analyzes your collections to understand data structure and meaning before performing queries. 𝗢𝘁𝗵𝗲𝗿 𝗖𝗼𝗼𝗹 𝗙𝗲𝗮𝘁𝘂𝗿𝗲𝘀: • 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸 𝗦𝘆𝘀𝘁𝗲𝗺: Uses positive examples as few-shot demonstrations for smaller, faster models • 𝗖𝗵𝘂𝗻𝗸-𝗢𝗻-𝗗𝗲𝗺𝗮𝗻𝗱: Dynamically chunks documents at query time instead of pre-chunking • 𝗠𝘂𝗹𝘁𝗶-𝗠𝗼𝗱𝗲𝗹 𝗦𝘁𝗿𝗮𝘁𝗲𝗴𝘆: Routes different tasks to appropriate model sizes based on complexity …And also how to get started with your own data! The entire project is open source and designed with customization in mind. You can use it as-is for effective data searching, or install the Python package to create custom tools for whatever agentic AI purposes you need. Big kudos to Edward for the vision, filming, and editing this masterpiece

Victoria Slocum

45,497 views • 1 year ago

Announcing a new Coursera course: Retrieval Augmented Generation (RAG) You'll learn to build high performance, production-ready RAG systems in this hands-on, in-depth course created by and taught by , experienced AI and ML engineer, researcher, and educator. RAG is a critical component today of many LLM-based applications in customer support, internal company Q&A systems, even many of the leading chatbots that use web search to answer your questions. This course teaches you in-depth how to make RAG work well. LLMs can produce generic or outdated responses, especially when asked specialized questions not covered in its training data. RAG is the most widely used technique for addressing this. It brings in data from new data sources, such as internal documents or recent news, to give the LLM the relevant context to private, recent, or specialized information. This lets it generate more grounded and accurate responses. In this course, you’ll learn to design and implement every part of a RAG system, from retrievers to vector databases to generation to evals. You’ll learn about the fundamental principles behind RAG and how to optimize it at both the component and whole-system levels. As AI evolves, RAG is evolving too. New models can handle longer context windows, reason more effectively, and can be parts of complex agentic workflows. One exciting growth area is Agentic RAG, in which an AI agent at runtime (rather than it being hardcoded at development time) autonomously decides what data to retrieve, and when/how to go deeper. Even with this evolution, access to high-quality data at runtime is essential, which is why RAG is a key part of so many applications. You'll learn via hands-on experiences to: - Build a RAG system with retrieval and prompt augmentation - Compare retrieval methods like BM25, semantic search, and Reciprocal Rank Fusion - Chunk, index, and retrieve documents using a Weaviate vector database and a news dataset - Develop a chatbot, using open-source LLMs hosted by Together AI, for a fictional store that answers product and FAQ questions - Use evals to drive improving reliability, and incorporate multi-modal data RAG is an important foundational technique. Become good at it through this course! Please sign up here:

Announcing a new Coursera course: Retrieval Augmented Generation (RAG) You'll learn to build high performance, production-ready RAG systems in this hands-on, in-depth course created by and taught by , experienced AI and ML engineer, researcher, and educator. RAG is a critical component today of many LLM-based applications in customer support, internal company Q&A systems, even many of the leading chatbots that use web search to answer your questions. This course teaches you in-depth how to make RAG work well. LLMs can produce generic or outdated responses, especially when asked specialized questions not covered in its training data. RAG is the most widely used technique for addressing this. It brings in data from new data sources, such as internal documents or recent news, to give the LLM the relevant context to private, recent, or specialized information. This lets it generate more grounded and accurate responses. In this course, you’ll learn to design and implement every part of a RAG system, from retrievers to vector databases to generation to evals. You’ll learn about the fundamental principles behind RAG and how to optimize it at both the component and whole-system levels. As AI evolves, RAG is evolving too. New models can handle longer context windows, reason more effectively, and can be parts of complex agentic workflows. One exciting growth area is Agentic RAG, in which an AI agent at runtime (rather than it being hardcoded at development time) autonomously decides what data to retrieve, and when/how to go deeper. Even with this evolution, access to high-quality data at runtime is essential, which is why RAG is a key part of so many applications. You'll learn via hands-on experiences to: - Build a RAG system with retrieval and prompt augmentation - Compare retrieval methods like BM25, semantic search, and Reciprocal Rank Fusion - Chunk, index, and retrieve documents using a Weaviate vector database and a news dataset - Develop a chatbot, using open-source LLMs hosted by Together AI, for a fictional store that answers product and FAQ questions - Use evals to drive improving reliability, and incorporate multi-modal data RAG is an important foundational technique. Become good at it through this course! Please sign up here:

Andrew Ng

124,656 views • 1 year ago

Karpathy said something you'll regret ignoring: "You are still responsible for your software, just as before. You are not allowed to introduce vulnerabilities because of vibe coding. " He said it while drawing the line between vibe coding and agentic engineering. Agents write more of the code now, but none of that takes the responsibility off you. The assumption underneath that is that a careful enough reader catches the problem. But some failures don't show up in anything there is to read. For instance, a common fear with a RAG agent is that it could hallucinate when a question asks something outside its corpus. But such cases are actually well handled by any competent model now. If nothing in the retrieved context looks relevant, there's no material to build an answer on. Instead, the majority of failures originate when the retrieved context has partial coverage. The retrieval pipeline returns context that's topically correct but doesn't cover the full question, and the model completes the remainder from parametric knowledge. There are no token-level labels in the output to tell what was generated using retrieved context and what came from weights. Both are streamed the same way. Detecting this for production-grade apps needs a metric written for it, one that's also aligned with principles of agentic engineering. And the solution is actually implemented in the eval skill that comes with Google’s Agents CLI. I described the concern to Claude Code in plain English. It read the agent's code, came back with a plan I approved. It then reported that no built-in metric isolates the behaviour and wrote a custom rubric called corpus_abstention. It assigned a single categorical verdict per case rather than aggregating everything into one score, since the built-in raters regenerate their rubrics each run and leave no stable number to trend. → GROUNDED_ANSWER → CORRECT_ABSTENTION → UNGROUNDED_ANSWER (answered entirely from outside knowledge) → MIXED_LEAKAGE (grounded, but slips in one unsupported claim) → WRONG_ABSTENTION (refused something the docs actually covered) After this, it automatically generated 33 scenarios partitioned by where the failure could occur, like: - in-corpus - off-domain - out-of-corpus but plausibly answerable - boundary cases where the topic is covered, but a specific detail isn't. The baseline score was 19 of 33. - Off-domain passed 3 of 3, as expected. - But 6 of 15 in-corpus cases retrieved the right document, cited it correctly, answered accurately, and added a claim the source never made. The root cause was one line in the agent's instruction: "If you already know the answer to a simple question and no document lookup is needed, you may respond directly without citations." The eval skill helped flag this, and then Claude removed it and forced retrieval on every question. This took the suite to 30 of 33, and ungrounded answers went from 6 to 0. The full recording of my run is below, and I worked with the Google Cloud team on this. Agents CLI GitHub repo → (don't forget to star 🌟) I wrote up the full build covering all six steps from install to enterprise registration. It includes the eval scorecard, the instruction loophole the eval caught before deployment, and what the deployment process actually looks like end-to-end. Read it below.

Karpathy said something you'll regret ignoring: "You are still responsible for your software, just as before. You are not allowed to introduce vulnerabilities because of vibe coding. " He said it while drawing the line between vibe coding and agentic engineering. Agents write more of the code now, but none of that takes the responsibility off you. The assumption underneath that is that a careful enough reader catches the problem. But some failures don't show up in anything there is to read. For instance, a common fear with a RAG agent is that it could hallucinate when a question asks something outside its corpus. But such cases are actually well handled by any competent model now. If nothing in the retrieved context looks relevant, there's no material to build an answer on. Instead, the majority of failures originate when the retrieved context has partial coverage. The retrieval pipeline returns context that's topically correct but doesn't cover the full question, and the model completes the remainder from parametric knowledge. There are no token-level labels in the output to tell what was generated using retrieved context and what came from weights. Both are streamed the same way. Detecting this for production-grade apps needs a metric written for it, one that's also aligned with principles of agentic engineering. And the solution is actually implemented in the eval skill that comes with Google’s Agents CLI. I described the concern to Claude Code in plain English. It read the agent's code, came back with a plan I approved. It then reported that no built-in metric isolates the behaviour and wrote a custom rubric called corpus_abstention. It assigned a single categorical verdict per case rather than aggregating everything into one score, since the built-in raters regenerate their rubrics each run and leave no stable number to trend. → GROUNDED_ANSWER → CORRECT_ABSTENTION → UNGROUNDED_ANSWER (answered entirely from outside knowledge) → MIXED_LEAKAGE (grounded, but slips in one unsupported claim) → WRONG_ABSTENTION (refused something the docs actually covered) After this, it automatically generated 33 scenarios partitioned by where the failure could occur, like: - in-corpus - off-domain - out-of-corpus but plausibly answerable - boundary cases where the topic is covered, but a specific detail isn't. The baseline score was 19 of 33. - Off-domain passed 3 of 3, as expected. - But 6 of 15 in-corpus cases retrieved the right document, cited it correctly, answered accurately, and added a claim the source never made. The root cause was one line in the agent's instruction: "If you already know the answer to a simple question and no document lookup is needed, you may respond directly without citations." The eval skill helped flag this, and then Claude removed it and forced retrieval on every question. This took the suite to 30 of 33, and ungrounded answers went from 6 to 0. The full recording of my run is below, and I worked with the Google Cloud team on this. Agents CLI GitHub repo → (don't forget to star 🌟) I wrote up the full build covering all six steps from install to enterprise registration. It includes the eval scorecard, the instruction loophole the eval caught before deployment, and what the deployment process actually looks like end-to-end. Read it below.

Akshay 🚀

82,546 views • 1 month ago

DEEP STATE TACTICS TO HIDE THE INCRIMINATING EVIDENCE Listen to Tulsi explain "burn bags", "over-classification" use of "hard copies" etc Burn bag is something that you'll find in many offices throughout the intelligence community, throughout these different workspaces in national security. Basically, if I have a document that's highly classified, I read it, I review it, I'm done with it, my copy can then go in the burn bag to make sure that it doesn't end up in the wrong hands, that there's not an unauthorized disclosure of classified information. It's a very different story, however, if you use a burn bag to try to get rid of evidence of which maybe there is only one copy of, and then hide it in a secret room in the FBI. That is a tactic, obviously, that has been used by those deep staters, these bad actors within the intelligence community, to try to get rid of evidence. Another tactic that's often used is over-classifying information. This is what happened with the Steele dossier. This was already widely publicized as a discredited document, but what John Brennan did and James Clapper did as the CIA director and Obama's director of national intelligence, they didn't want anybody to know that they used the Steele dossier as a source for this intelligence assessment that was filled with falsehoods. They published in January 2017, as well as other really, really shoddy intelligence sources that would not be acceptable on any topic, because they were not deemed credible. They wanted to hide all of this from members of Congress, perhaps, who were reading this, and the American people, and President Trump then, President-elect Trump at that time. They over-classified it in a way that there were less than 10 people who could access it. Then they locked up the only hard copy document. This document did not exist on Digis, the document that actually exposed. This was the document that we released. They locked that document up, and the five copies that existed in a safe, and the first time it had been seen, certainly in public, but even by many within the government, was when we released it.

DEEP STATE TACTICS TO HIDE THE INCRIMINATING EVIDENCE Listen to Tulsi explain "burn bags", "over-classification" use of "hard copies" etc Burn bag is something that you'll find in many offices throughout the intelligence community, throughout these different workspaces in national security. Basically, if I have a document that's highly classified, I read it, I review it, I'm done with it, my copy can then go in the burn bag to make sure that it doesn't end up in the wrong hands, that there's not an unauthorized disclosure of classified information. It's a very different story, however, if you use a burn bag to try to get rid of evidence of which maybe there is only one copy of, and then hide it in a secret room in the FBI. That is a tactic, obviously, that has been used by those deep staters, these bad actors within the intelligence community, to try to get rid of evidence. Another tactic that's often used is over-classifying information. This is what happened with the Steele dossier. This was already widely publicized as a discredited document, but what John Brennan did and James Clapper did as the CIA director and Obama's director of national intelligence, they didn't want anybody to know that they used the Steele dossier as a source for this intelligence assessment that was filled with falsehoods. They published in January 2017, as well as other really, really shoddy intelligence sources that would not be acceptable on any topic, because they were not deemed credible. They wanted to hide all of this from members of Congress, perhaps, who were reading this, and the American people, and President Trump then, President-elect Trump at that time. They over-classified it in a way that there were less than 10 people who could access it. Then they locked up the only hard copy document. This document did not exist on Digis, the document that actually exposed. This was the document that we released. They locked that document up, and the five copies that existed in a safe, and the first time it had been seen, certainly in public, but even by many within the government, was when we released it.

Svetlana Lokhova

18,864 views • 10 months ago

The People Of America Have Been Deceived. They’ve Been Cheated & As A Result Our Country & World Is Now In Turmoil. It all Started With A Fake Barack Obama Birth Certificate EXPERTS CONFIRM Obama’s Certificate Was A Fraud “Today you're going to hear lots of information that some of you are going to understand and going to be able to tell the true story. In fact, please know that this is a very technical, but the evidence is clear if you'll pay attention. Please note you're going to hear about two separate experts. These experts are two separate continents with no knowledge of each other and they draw similar conclusions. Again, that said, I know some of you are going to get this story and are going to tell the story the way it was.” —- “We and anyone else who dared to question the document have been the line falsely labeled grossly criticized in the bulk of the media on certain internet sources for years. Today we're going to set the record straight. I believe you will be shocked by what you hear and see today.” —- “Like the sheriff just told you, when you conduct criminal investigations, you have to let the evidence lead you. You never lead the evidence. And in doing this, my motive was to clear the document. Because to be quite honest with you, I didn't believe it. I didn't believe this was possible. I didn't think this would ever happen in this nation. I didn't believe it.” — “Back in 2012, I told you about Reed Hayes, a document examiner. Let me tell you about Reed Hayes, a man with 40 years, since 1974, 40 plus years of experience in examining forensic document, handwriting, a man who's well respected in his expertise, a court recognized expert, a document examiner. He is the man you go to when somebody gives you a bad check with a bad signature. This is the guy you run to. Law firms use him all the time. He's been maligned. And let me tell you something about Mr. Hayes. When I contacted Mr. Hayes, Mr. Hayes told me right off the bat, I'm an Obama supporter. I voted for him twice. He goes, and I will never do anything to hurt the President of the United States. What I had said to him was, Reed, I am not asking you to hurt the President of the United States. I'm asking you to take a look at this document and clear it and tell me there's nothing wrong with it. Would you at least do that? And he took a look at it. And when he called me back, he told me, Mike.” “I can't clear this, there's something wrong with it." And I asked him, I said, Reed, would you continue? I said, I know your position, but would you continue? And his answer to me was, this is what I do. I'll look at it, I'll do it. That's a man of integrity, respecting what his ability is to get to the truth. Because for Sheriff Apoyo and myself, this was never about Barack Obama. This is about a document. You take that document and you remove the name, Barack Hussein Obama, and put your name on there. If it was your document and it was brought to us, we would do the same thing with this document.” They COULD NOT clear the document. Much more info in this video if you watch the whole thing

The People Of America Have Been Deceived. They’ve Been Cheated & As A Result Our Country & World Is Now In Turmoil. It all Started With A Fake Barack Obama Birth Certificate EXPERTS CONFIRM Obama’s Certificate Was A Fraud “Today you're going to hear lots of information that some of you are going to understand and going to be able to tell the true story. In fact, please know that this is a very technical, but the evidence is clear if you'll pay attention. Please note you're going to hear about two separate experts. These experts are two separate continents with no knowledge of each other and they draw similar conclusions. Again, that said, I know some of you are going to get this story and are going to tell the story the way it was.” —- “We and anyone else who dared to question the document have been the line falsely labeled grossly criticized in the bulk of the media on certain internet sources for years. Today we're going to set the record straight. I believe you will be shocked by what you hear and see today.” —- “Like the sheriff just told you, when you conduct criminal investigations, you have to let the evidence lead you. You never lead the evidence. And in doing this, my motive was to clear the document. Because to be quite honest with you, I didn't believe it. I didn't believe this was possible. I didn't think this would ever happen in this nation. I didn't believe it.” — “Back in 2012, I told you about Reed Hayes, a document examiner. Let me tell you about Reed Hayes, a man with 40 years, since 1974, 40 plus years of experience in examining forensic document, handwriting, a man who's well respected in his expertise, a court recognized expert, a document examiner. He is the man you go to when somebody gives you a bad check with a bad signature. This is the guy you run to. Law firms use him all the time. He's been maligned. And let me tell you something about Mr. Hayes. When I contacted Mr. Hayes, Mr. Hayes told me right off the bat, I'm an Obama supporter. I voted for him twice. He goes, and I will never do anything to hurt the President of the United States. What I had said to him was, Reed, I am not asking you to hurt the President of the United States. I'm asking you to take a look at this document and clear it and tell me there's nothing wrong with it. Would you at least do that? And he took a look at it. And when he called me back, he told me, Mike.” “I can't clear this, there's something wrong with it." And I asked him, I said, Reed, would you continue? I said, I know your position, but would you continue? And his answer to me was, this is what I do. I'll look at it, I'll do it. That's a man of integrity, respecting what his ability is to get to the truth. Because for Sheriff Apoyo and myself, this was never about Barack Obama. This is about a document. You take that document and you remove the name, Barack Hussein Obama, and put your name on there. If it was your document and it was brought to us, we would do the same thing with this document.” They COULD NOT clear the document. Much more info in this video if you watch the whole thing

Wall Street Apes

1,140,943 views • 3 years ago