Loading video...

Video Failed to Load

Go Home

ok so let me explain why subagents kill long context Like you can spend $500m building 100 million context models, and they would be 1) slow, 2) expensive to use, 3) have huge context rot. O(n) is the lower bound. Cog's approach is something you learn in day 1...

swyx's profile picture

swyx ๐Ÿ‡ธ๐Ÿ‡ฌ

155,072 subscribers

276,060 views โ€ข 10 months ago โ€ขvia X (Twitter)

Science & TechnologyEducationComedy
LIVE

Anya Rossiโ€ข Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

I cant believe this guy just made a permanent solution to context bloat and open sourced it all! when we tested this tool (Context+) for solving an issue on the OpenCode repository, the agent using this tool used ~6.5k fewer tokens, found the code and fixed it in half the time! the results were surprising: 6 to 10k tokens saved per prompt, completed task in ~2 minutes while the agent running without the tool took ~4 mins for the same and got stuck in loops bro built an entire beast by using all the modern tools that we could think of: undo trees, semantic search by meaning (by haskellforall), advanced refactoring, blast radius, advanced file context trees, restore points... i can keep going on semantic code search and context trees are the future of agentic coding and this tool proves it the feature i loved the most is semantic search and how it gets things done 2x faster with least possible tokens it makes an agent that actually knows what itโ€™s doing and not just guessing, it makes meaning from your code similar to RAG. if you aren't optimizing your context, you are just burning money the developer says this tool is still under development, it can have unexpected behavior and the docs need updates but the video shows the reality of how fast it can be github: get here:
0:46

I cant believe this guy just made a permanent solution to context bloat and open sourced it all! when we tested this tool (Context+) for solving an issue on the OpenCode repository, the agent using this tool used ~6.5k fewer tokens, found the code and fixed it in half the time! the results were surprising: 6 to 10k tokens saved per prompt, completed task in ~2 minutes while the agent running without the tool took ~4 mins for the same and got stuck in loops bro built an entire beast by using all the modern tools that we could think of: undo trees, semantic search by meaning (by haskellforall), advanced refactoring, blast radius, advanced file context trees, restore points... i can keep going on semantic code search and context trees are the future of agentic coding and this tool proves it the feature i loved the most is semantic search and how it gets things done 2x faster with least possible tokens it makes an agent that actually knows what itโ€™s doing and not just guessing, it makes meaning from your code similar to RAG. if you aren't optimizing your context, you are just burning money the developer says this tool is still under development, it can have unexpected behavior and the docs need updates but the video shows the reality of how fast it can be github: get here:

forloop

226,271 views โ€ข 5 months ago

RLM is the most import foundation of my Pi Harness (other than Pi of course). It's seeded with late interaction retrieval results (thanks to @lightonai for pylate). The Agent initiates it with query then.. ๐’๐ž๐ญ๐ฎ๐ฉ A python REPL is created and seeded with: 1. Late interaction search to pre-filter. Instead of doing top 3/5/10, it's top hundreds of documents. This is set into a `context` variable. 2. Python functions are loaded in to do more searches if `context` variable isn't enough. And to make llm calls with cheaper models in parallel batches. ๐ˆ๐ญ๐ž๐ซ๐š๐ญ๐ข๐จ๐ง ๐‹๐จ๐จ๐ฉ From there, an LLM iterates in the REPL based on the query. It's just like exploring in a jupyter notebook. The LLM writes prose (like a markdown cell) and code to be run in the REPL each turn. This allows the LLM to sort, filter, and synthesize information. It can fan out and ask smaller models to summarize, combine, contrast, or do anything else to documents to help it understand the data. After several turns the LLM reponds with the final answer. Either because it found the answer, or hit the budget limit. Context as a Python variable, LLM as the programmer, REPL as the runtime. ๐–๐ก๐ฒ ๐ƒ๐จ๐ž๐ฌ ๐“๐ก๐ข๐ฌ ๐–๐จ๐ซ๐ค 1. Richer Shell. Agents (and subagents) work by intermixing code and prose/thinking. But they use static scripts or bash that run and exit and start over each tool call. That's not ideal for exploration and synthesis of data. For that, state is useful to continue building and exploring the data as you learn more. There's a reason jupyter notebooks have been popular with data scientists. 2. Keeps main agent context clean. The better context you have the better the agent will perform (duh!). This means three thing: better human input, less missing search results, and less incorrect search results. Letting the agent iterate allows it to synthesize just what is needed and nothing else. All bad paths or peeks at something that turns out to be irrelevant stays out of main agent context. 3. Stack the good ideas! People often compare late interaction search vs RLM. Or static vs dynamic languages. Or agentic search vs semantic search. But...You can just use them all together for what they're each good at. Use them all for the area they're really great for. Read the full post which has more detail about how and why.
3:43

RLM is the most import foundation of my Pi Harness (other than Pi of course). It's seeded with late interaction retrieval results (thanks to @lightonai for pylate). The Agent initiates it with query then.. ๐’๐ž๐ญ๐ฎ๐ฉ A python REPL is created and seeded with: 1. Late interaction search to pre-filter. Instead of doing top 3/5/10, it's top hundreds of documents. This is set into a `context` variable. 2. Python functions are loaded in to do more searches if `context` variable isn't enough. And to make llm calls with cheaper models in parallel batches. ๐ˆ๐ญ๐ž๐ซ๐š๐ญ๐ข๐จ๐ง ๐‹๐จ๐จ๐ฉ From there, an LLM iterates in the REPL based on the query. It's just like exploring in a jupyter notebook. The LLM writes prose (like a markdown cell) and code to be run in the REPL each turn. This allows the LLM to sort, filter, and synthesize information. It can fan out and ask smaller models to summarize, combine, contrast, or do anything else to documents to help it understand the data. After several turns the LLM reponds with the final answer. Either because it found the answer, or hit the budget limit. Context as a Python variable, LLM as the programmer, REPL as the runtime. ๐–๐ก๐ฒ ๐ƒ๐จ๐ž๐ฌ ๐“๐ก๐ข๐ฌ ๐–๐จ๐ซ๐ค 1. Richer Shell. Agents (and subagents) work by intermixing code and prose/thinking. But they use static scripts or bash that run and exit and start over each tool call. That's not ideal for exploration and synthesis of data. For that, state is useful to continue building and exploring the data as you learn more. There's a reason jupyter notebooks have been popular with data scientists. 2. Keeps main agent context clean. The better context you have the better the agent will perform (duh!). This means three thing: better human input, less missing search results, and less incorrect search results. Letting the agent iterate allows it to synthesize just what is needed and nothing else. All bad paths or peeks at something that turns out to be irrelevant stays out of main agent context. 3. Stack the good ideas! People often compare late interaction search vs RLM. Or static vs dynamic languages. Or agentic search vs semantic search. But...You can just use them all together for what they're each good at. Use them all for the area they're really great for. Read the full post which has more detail about how and why.

Isaac Flath

40,212 views โ€ข 4 months ago

How can you solve complex tasks using a Large Language Model? Here is a 2-minute introduction to everything you need to know to 10x the quality of your results. Let's talk about three techniques, in order of complexity, starting with the easiest one: โ€ข In-Context Learning โ€ข Indexing + In-Context Learning โ€ข Fine-tuning In-Context Learning The team that trained GPT-3 found something they couldn't explain: You can condition a model using examples of how you want it to behave. I included an example prompt in the attached video. You can "teach" the model how you want it to interpret questions, select the correct answers, and format the results by giving a few examples. You can also give specific knowledge to the model that will be helpful when formulating answers. We call this approach "grounding the model." There's another example in the video. Indexing + In-Context Learning Unfortunately, there is a limit to how much data you can include in a prompt. We call this the "context size." One version of GPT-4 supports a context of approximately 6,000 words, while the other supports 25,000 words. Although this sounds like a lot, many applications need more than that. Imagine you wrote a book and want to build an application to answer any questions about your story. What happens if your book is longer than the context? That's where Indexing comes in. Using a model, you can turn every book passage into an embedding. These are vectors, numbers that "encode" the passage's text. You can then store these embeddings in a particular database that supports fast retrieval of these vectors. You can then turn any question into an embedding and search the database for the list of passages that are similar to that query. Instead of using the entire book to ask the model, you can now use the relevant passages as in-context information, effectively working around the context size limitation. Fine-tuning Fine-tuning can give you an extra boost to get reliable outputs from your LLM. It is, however, the most complex approach on the list. There are different approaches to fine-tuning a model with your data. A popular technique is to process your data with your LLM and use the outputs to train a new classifier that solves your specific task. Notice that here you aren't modifying the LLM. Instead, you are chaining it with your trained classifier. Another approach is to modify the parameters of the LLM using your data. Think of this as "rewiring" the model in a way that solves your particular task. The results and costs will vary depending on how many layers you want to fine-tune from the original model. Many companies think that fine-tuning is the solution to their problems. In my experience, many will benefit from exploring the other two approaches. I love explaining Machine Learning and Artificial Intelligence ideas. If you enjoy in-depth content like this, follow me Santiago so you don't miss what comes next.
4:50

How can you solve complex tasks using a Large Language Model? Here is a 2-minute introduction to everything you need to know to 10x the quality of your results. Let's talk about three techniques, in order of complexity, starting with the easiest one: โ€ข In-Context Learning โ€ข Indexing + In-Context Learning โ€ข Fine-tuning In-Context Learning The team that trained GPT-3 found something they couldn't explain: You can condition a model using examples of how you want it to behave. I included an example prompt in the attached video. You can "teach" the model how you want it to interpret questions, select the correct answers, and format the results by giving a few examples. You can also give specific knowledge to the model that will be helpful when formulating answers. We call this approach "grounding the model." There's another example in the video. Indexing + In-Context Learning Unfortunately, there is a limit to how much data you can include in a prompt. We call this the "context size." One version of GPT-4 supports a context of approximately 6,000 words, while the other supports 25,000 words. Although this sounds like a lot, many applications need more than that. Imagine you wrote a book and want to build an application to answer any questions about your story. What happens if your book is longer than the context? That's where Indexing comes in. Using a model, you can turn every book passage into an embedding. These are vectors, numbers that "encode" the passage's text. You can then store these embeddings in a particular database that supports fast retrieval of these vectors. You can then turn any question into an embedding and search the database for the list of passages that are similar to that query. Instead of using the entire book to ask the model, you can now use the relevant passages as in-context information, effectively working around the context size limitation. Fine-tuning Fine-tuning can give you an extra boost to get reliable outputs from your LLM. It is, however, the most complex approach on the list. There are different approaches to fine-tuning a model with your data. A popular technique is to process your data with your LLM and use the outputs to train a new classifier that solves your specific task. Notice that here you aren't modifying the LLM. Instead, you are chaining it with your trained classifier. Another approach is to modify the parameters of the LLM using your data. Think of this as "rewiring" the model in a way that solves your particular task. The results and costs will vary depending on how many layers you want to fine-tune from the original model. Many companies think that fine-tuning is the solution to their problems. In my experience, many will benefit from exploring the other two approaches. I love explaining Machine Learning and Artificial Intelligence ideas. If you enjoy in-depth content like this, follow me Santiago so you don't miss what comes next.

Santiago

384,510 views โ€ข 3 years ago

New short course: Build Long-Context AI Apps with Jamba. Learn about state space models (SSMs), which have emerged as an alternative to transformers! Specifically, Jamba is a hybrid transformer-Mamba architecture that combines strengths of the transformer with ideas from SSMs. This course is built with AI21 Labs and taught by Chen Wang and Chen Almagor. The transformer architecture is computationally expensive when handling very long input contexts. But there's an alternative called Mamba, a selective state space model that can process very long contexts with a much lower computational cost. However, researchers found that the pure Mamba architecture underperforms in understanding the context, and gives lower-quality responses. To overcome this, AI21 developed the Jamba model, which combines Mamba's computational efficiency with the transformer's attention mechanism to help with the output quality. In this course, youโ€™ll learn about how state space models, and Jamba, work. Youโ€™ll also learn how to prompt Jamba, use it to process long documents, and build long-context RAG apps. - Learn how Jamba combines transformer and state space model architectures to achieve high performance and quality - Use the AI21 SDK, with an example of prompting over a large 200k-token annual financial report of Nvidia - Use Jamba for tool-calling, with hands-on examples from calling simple arithmetic calculations to a function that returns quarterly company financial reports. - Learn how training for long context is done, and the metrics used for its evaluation - Create a RAG app using the AI21 Conversational RAG tool and build your own RAG pipeline that uses Jamba and LangChain. By the end of this course, you'll learn how to build applications that can handle context as long as an entire book. Please sign up here:
2:04

New short course: Build Long-Context AI Apps with Jamba. Learn about state space models (SSMs), which have emerged as an alternative to transformers! Specifically, Jamba is a hybrid transformer-Mamba architecture that combines strengths of the transformer with ideas from SSMs. This course is built with AI21 Labs and taught by Chen Wang and Chen Almagor. The transformer architecture is computationally expensive when handling very long input contexts. But there's an alternative called Mamba, a selective state space model that can process very long contexts with a much lower computational cost. However, researchers found that the pure Mamba architecture underperforms in understanding the context, and gives lower-quality responses. To overcome this, AI21 developed the Jamba model, which combines Mamba's computational efficiency with the transformer's attention mechanism to help with the output quality. In this course, youโ€™ll learn about how state space models, and Jamba, work. Youโ€™ll also learn how to prompt Jamba, use it to process long documents, and build long-context RAG apps. - Learn how Jamba combines transformer and state space model architectures to achieve high performance and quality - Use the AI21 SDK, with an example of prompting over a large 200k-token annual financial report of Nvidia - Use Jamba for tool-calling, with hands-on examples from calling simple arithmetic calculations to a function that returns quarterly company financial reports. - Learn how training for long context is done, and the metrics used for its evaluation - Create a RAG app using the AI21 Conversational RAG tool and build your own RAG pipeline that uses Jamba and LangChain. By the end of this course, you'll learn how to build applications that can handle context as long as an entire book. Please sign up here:

Andrew Ng

77,792 views โ€ข 1 year ago

I can search a database with 32 million records in less than 50 milliseconds. โ€‹ This speed is insane! โ€‹ There's no reason your search is slow. There's no reason you aren't using this library when building applications with a lot of data. โ€‹ Take a look at typesense |. It's an open-source search engine. It's crazy fast and easy to use. They are sponsoring this post. โ€‹ Here is their GitHub repository: โ€‹ โ€‹ You can use the library in Python, JavaScript, PHP, Ruby, Swift, and pretty much any other programming language out there. โ€‹ Here are some of the things you can do with Typesense: โ€‹ โ€ข Vector search over embeddings โ€ข Conversational search for RAG applications โ€ข Image similarity search โ€ข Voice search โ€‹ And it does all of this extremely fast. โ€‹ A few interesting notes about the library: โ€‹ โ€ข Optimized for developer happiness โ€ข Works out of the box for most use cases โ€ข Very intuitive API โ€ข Easy to deploy and scale โ€‹ Probably the best way to compare them is against ElasticSearch. The latter is much more complex to configure and run with a steep learning curve. โ€‹ Check their GitHub repository. They have examples, tutorials, and everything you need.
1:18

I can search a database with 32 million records in less than 50 milliseconds. โ€‹ This speed is insane! โ€‹ There's no reason your search is slow. There's no reason you aren't using this library when building applications with a lot of data. โ€‹ Take a look at typesense |. It's an open-source search engine. It's crazy fast and easy to use. They are sponsoring this post. โ€‹ Here is their GitHub repository: โ€‹ โ€‹ You can use the library in Python, JavaScript, PHP, Ruby, Swift, and pretty much any other programming language out there. โ€‹ Here are some of the things you can do with Typesense: โ€‹ โ€ข Vector search over embeddings โ€ข Conversational search for RAG applications โ€ข Image similarity search โ€ข Voice search โ€‹ And it does all of this extremely fast. โ€‹ A few interesting notes about the library: โ€‹ โ€ข Optimized for developer happiness โ€ข Works out of the box for most use cases โ€ข Very intuitive API โ€ข Easy to deploy and scale โ€‹ Probably the best way to compare them is against ElasticSearch. The latter is much more complex to configure and run with a steep learning curve. โ€‹ Check their GitHub repository. They have examples, tutorials, and everything you need.

Santiago

160,154 views โ€ข 2 years ago

Traditional (SQL) databases rely primarily on keyword-based searches to retrieve information. These searches match the exact words or phrases in your query to the text stored in the database. While effective for many applications, this method has limitations when it comes to understanding context or finding relevant information that doesnโ€™t include the exact keywords. Hybrid search combines the strengths of traditional keyword-based BM25 search with the advanced capabilities of semantic search. To effectively implement a hybrid search, a vector database is essential. Vector databases go beyond just words; they understand the meaning behind the data. They transform data such as text, images, or audio into numerical representations called vectors. These vector embeddings enable the database to find similar items, even if they don't share exact keywords. When you integrate hybrid search with Retrieval-Augmented Generation (RAG) systems, you can achieve higher accuracy in retrieved context and better output in generated responses. Learn more about RAG systems in this video with Victoria Slocum:
0:44

Traditional (SQL) databases rely primarily on keyword-based searches to retrieve information. These searches match the exact words or phrases in your query to the text stored in the database. While effective for many applications, this method has limitations when it comes to understanding context or finding relevant information that doesnโ€™t include the exact keywords. Hybrid search combines the strengths of traditional keyword-based BM25 search with the advanced capabilities of semantic search. To effectively implement a hybrid search, a vector database is essential. Vector databases go beyond just words; they understand the meaning behind the data. They transform data such as text, images, or audio into numerical representations called vectors. These vector embeddings enable the database to find similar items, even if they don't share exact keywords. When you integrate hybrid search with Retrieval-Augmented Generation (RAG) systems, you can achieve higher accuracy in retrieved context and better output in generated responses. Learn more about RAG systems in this video with Victoria Slocum:

Femke Plantinga

140,618 views โ€ข 2 years ago

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, youโ€™ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!
2:57

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, youโ€™ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

201,127 views โ€ข 1 year ago

New RLM trajectory that blew my mind! I will use this one as the main example in the YT tutorial. I passed in a CSV containing transcripts of 320 episodes of the Lex Fridman podcast and asked it to find what his first 10 ML guests had to say about AGI. The context had 60,855,062 characters. > Main agent explored data format, understood its CSV > extracted all 320 guests, identified the first 10 ML guys (Benegio, Brockman, Goodfellow etc) > Launched parallel subagents passing just their corresponding transcripts (about 35K chars each) > Subagents performed find operations to search for AGI, read the context and returned outputs > Main agent gathered all the data, generated a summary of all AGI conversations It took 4 minutes to crunch, and the fun part is it cost me 0.2$ with Minimax-M2.5. It read 1M tokens (825K was cache hits so it was quite cheap), produced just 69K tokens (19K were reasoning). ---- My notes: - This would be basically impossible to do at this quality with a base LM. (context rot, since 99% of the data is useless) - It will cost 20x more with ReAct model (too many tasks) - It will cost 10x more with a React + Subagent model (read/write contexts instead of using symbolic variables) - I'm a happy panda. (thanks for reading)
0:45

New RLM trajectory that blew my mind! I will use this one as the main example in the YT tutorial. I passed in a CSV containing transcripts of 320 episodes of the Lex Fridman podcast and asked it to find what his first 10 ML guests had to say about AGI. The context had 60,855,062 characters. > Main agent explored data format, understood its CSV > extracted all 320 guests, identified the first 10 ML guys (Benegio, Brockman, Goodfellow etc) > Launched parallel subagents passing just their corresponding transcripts (about 35K chars each) > Subagents performed find operations to search for AGI, read the context and returned outputs > Main agent gathered all the data, generated a summary of all AGI conversations It took 4 minutes to crunch, and the fun part is it cost me 0.2$ with Minimax-M2.5. It read 1M tokens (825K was cache hits so it was quite cheap), produced just 69K tokens (19K were reasoning). ---- My notes: - This would be basically impossible to do at this quality with a base LM. (context rot, since 99% of the data is useless) - It will cost 20x more with ReAct model (too many tasks) - It will cost 10x more with a React + Subagent model (read/write contexts instead of using symbolic variables) - I'm a happy panda. (thanks for reading)

AVB

43,794 views โ€ข 6 months ago

Our new short course, Knowledge Graphs for RAG, is now available! Knowledge graphs are a data structure that is great at capturing complex relationships between data of multiple types. By enabling more sophisticated retrieval of text than similarity search alone, knowledge graphs can improve the context you pass to the LLM and the performance of your RAG applications. In this course, taught by Andreas Kollegger of Neo4j, youโ€™ll - Explore how knowledge graphs work by building a graph of public financial documents from scratch - Learn to write queries that retrieve text and data from the graph and use it to enhance the context you pass to an LLM chatbot - Combine a knowledge graph with a question-answer chain to build better RAG-powered chat systems Sign up here!
3:46

Our new short course, Knowledge Graphs for RAG, is now available! Knowledge graphs are a data structure that is great at capturing complex relationships between data of multiple types. By enabling more sophisticated retrieval of text than similarity search alone, knowledge graphs can improve the context you pass to the LLM and the performance of your RAG applications. In this course, taught by Andreas Kollegger of Neo4j, youโ€™ll - Explore how knowledge graphs work by building a graph of public financial documents from scratch - Learn to write queries that retrieve text and data from the graph and use it to enhance the context you pass to an LLM chatbot - Combine a knowledge graph with a question-answer chain to build better RAG-powered chat systems Sign up here!

Andrew Ng

244,435 views โ€ข 2 years ago

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: โ€‹ 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. โ€‹ 2. The connector ecosystem to load data from unstructured data sources is very immature. โ€‹ 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. โ€‹ The goal of a RAG Pipeline is to solve these problems. โ€‹ The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. โ€‹ At a high level, there are four different stages in the architecture of a RAG pipeline: โ€‹ 1. Ingestion: Here is where the pipeline loads the information from the data source. โ€‹ 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. โ€‹ 3. Transform: Where the pipeline chunks the data and generates document embeddings. โ€‹ 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. โ€‹ There are different rabbit holes at each one of these stages. Here are three of them: โ€‹ 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. โ€‹ 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. โ€‹ 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. โ€‹ In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. โ€‹ I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. โ€‹ โ€‹ If you have a few documents lying around, set up a free account and give it a try.
3:33

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: โ€‹ 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. โ€‹ 2. The connector ecosystem to load data from unstructured data sources is very immature. โ€‹ 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. โ€‹ The goal of a RAG Pipeline is to solve these problems. โ€‹ The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. โ€‹ At a high level, there are four different stages in the architecture of a RAG pipeline: โ€‹ 1. Ingestion: Here is where the pipeline loads the information from the data source. โ€‹ 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. โ€‹ 3. Transform: Where the pipeline chunks the data and generates document embeddings. โ€‹ 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. โ€‹ There are different rabbit holes at each one of these stages. Here are three of them: โ€‹ 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. โ€‹ 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. โ€‹ 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. โ€‹ In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. โ€‹ I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. โ€‹ โ€‹ If you have a few documents lying around, set up a free account and give it a try.

Santiago

40,625 views โ€ข 1 year ago

A builder wants to ship an agent that can search live data, use it instantly, and keep going without manual steps. That is why OptimAI Search now ships with x402, SDK, and MCP support. x402 makes access and payment seamless. SDK and MCP make integration straightforward for builders. And under the hood, OptimAI Search gives agents real-time access to huge volumes of public and social web data, structured into context they can actually use. Add Agent Marketplace on top, and domain knowledge becomes reusable: build specialized agents, publish them, and let others build on what you create. This is what search looks like in the agent-native internet. ๐Ÿ’กTry it:
1:07

A builder wants to ship an agent that can search live data, use it instantly, and keep going without manual steps. That is why OptimAI Search now ships with x402, SDK, and MCP support. x402 makes access and payment seamless. SDK and MCP make integration straightforward for builders. And under the hood, OptimAI Search gives agents real-time access to huge volumes of public and social web data, structured into context they can actually use. Add Agent Marketplace on top, and domain knowledge becomes reusable: build specialized agents, publish them, and let others build on what you create. This is what search looks like in the agent-native internet. ๐Ÿ’กTry it:

OptimAI Network

36,562 views โ€ข 3 months ago

Searching 32 million records in 30 milliseconds is insane! I don't know how large your database is, but there's no reason your search results are slower than this. typesense | is a crazy fast open-source search engine. They are like ElasticSearch but way easier to set up, configure, and use. They are sponsoring this post. You can do all of this with Typesense: โ€ข Fuzzy search: Typo-tolerant fuzzy search-as-you-type โ€ข Vector search: Return records that are similar to each other โ€ข Semantic search: Return records that are conceptually related โ€ข Geo Search: Return records that are close to a given lat/long They have client libraries in Python, JavaScript, PHP, Ruby, Go, Rust, Swift, .Net, Java, and pretty much any other programming language out there. You can start searching in less than a minute: 1. Run their Docker image (or one of their native binaries) 2. Import your data 3. Start searching! GitHub repository: I recorded a quick video to show you a demo. I promise you haven't seen something this fast! This is the best upgrade you'll make to your application this year. Guaranteed.
2:38

Searching 32 million records in 30 milliseconds is insane! I don't know how large your database is, but there's no reason your search results are slower than this. typesense | is a crazy fast open-source search engine. They are like ElasticSearch but way easier to set up, configure, and use. They are sponsoring this post. You can do all of this with Typesense: โ€ข Fuzzy search: Typo-tolerant fuzzy search-as-you-type โ€ข Vector search: Return records that are similar to each other โ€ข Semantic search: Return records that are conceptually related โ€ข Geo Search: Return records that are close to a given lat/long They have client libraries in Python, JavaScript, PHP, Ruby, Go, Rust, Swift, .Net, Java, and pretty much any other programming language out there. You can start searching in less than a minute: 1. Run their Docker image (or one of their native binaries) 2. Import your data 3. Start searching! GitHub repository: I recorded a quick video to show you a demo. I promise you haven't seen something this fast! This is the best upgrade you'll make to your application this year. Guaranteed.

Santiago

172,755 views โ€ข 1 year ago

clickup just mass-obsoleted every AI agent platform here's the thing nobody's talking about: AI doesn't fail because the models are bad. it fails because it has no context. you paste in a prompt. it hallucinates. you blame the tool. but the tool was never the problem. the CONTEXT was. clickup just solved this. their new "super agents" live INSIDE your workspace. they see everything: - your tasks - your docs - your meetings - your conversations - your decisions 100% context. always on. always scraping. this isn't bolted-on AI calling APIs. this is AI that already knows your business. i've been running an AI backend for my agency. the architecture works - but you have to BUILD the context layer. clickup just made that automatic. i already use it. have tons of data there. this is about to be unbelievable. if you're building automations and you're not paying attention to thisโ€” you're about to get lapped. playing with it hard over the next 2 days. full deep dive coming. drop "SUPER" and i'll send you the breakdown when it's ready.
5:50

clickup just mass-obsoleted every AI agent platform here's the thing nobody's talking about: AI doesn't fail because the models are bad. it fails because it has no context. you paste in a prompt. it hallucinates. you blame the tool. but the tool was never the problem. the CONTEXT was. clickup just solved this. their new "super agents" live INSIDE your workspace. they see everything: - your tasks - your docs - your meetings - your conversations - your decisions 100% context. always on. always scraping. this isn't bolted-on AI calling APIs. this is AI that already knows your business. i've been running an AI backend for my agency. the architecture works - but you have to BUILD the context layer. clickup just made that automatic. i already use it. have tons of data there. this is about to be unbelievable. if you're building automations and you're not paying attention to thisโ€” you're about to get lapped. playing with it hard over the next 2 days. full deep dive coming. drop "SUPER" and i'll send you the breakdown when it's ready.

Nozz

20,298 views โ€ข 8 months ago

Tune Studio is an end-to-end platform for developing applications using Large Language Models. So far, I haven't seen any other platform like this one. You can do everything here: 1. You can curate your data. 2. Use the playground to play with different models and try your ideas. 3. Fine-tune an open-source model on your data. 4. Deploy the model when you are done. This is awesome for anyone building generative AI applications. You can use Tune Studio to work with any of the open-source models out there. They were one of the few companies to host Llama 2 and Llama 3 before anyone else. Here is a link to check it out: One of their main selling points is that Tune Studio scales! You don't have to worry about serving your model to lots of users. They also have built-in user management, authentication, on-prem support, user context management, and pretty much everything you need to build generative AI applications. Thanks to the Tune team for collaborating with me on this post. We are living through the best years of development tools for AI developers. The field is unstoppable.
0:23

Tune Studio is an end-to-end platform for developing applications using Large Language Models. So far, I haven't seen any other platform like this one. You can do everything here: 1. You can curate your data. 2. Use the playground to play with different models and try your ideas. 3. Fine-tune an open-source model on your data. 4. Deploy the model when you are done. This is awesome for anyone building generative AI applications. You can use Tune Studio to work with any of the open-source models out there. They were one of the few companies to host Llama 2 and Llama 3 before anyone else. Here is a link to check it out: One of their main selling points is that Tune Studio scales! You don't have to worry about serving your model to lots of users. They also have built-in user management, authentication, on-prem support, user context management, and pretty much everything you need to build generative AI applications. Thanks to the Tune team for collaborating with me on this post. We are living through the best years of development tools for AI developers. The field is unstoppable.

Santiago

39,101 views โ€ข 2 years ago

Building with AI gets easier every day. Here is an open-source library that makes integrating AI into an application extremely easy: Star the repository! This library alone can make React the best front-end framework out there! There are a bunch of cool things I like about CopilotKit. Here are 3 of them: 1. It allows you to take any -powered agent and bring it into your application. (This is a brand-new feature!) 2. You can build an AI-powered chatbot in your application. The chatbot will have access to your context and can act on the application. 3. You can build a RAG workflow to process and answer questions from a real-time knowledge base. I recorded a video to show you how simple it is to make some of this happen. A few lines of code, and you are in business. Here is a link to the sample application: CopilotKit is open-source. You can self-host it. You can use it with any LLM. Thanks to the team for showing me their tool and collaborating with me on this post!
5:59

Building with AI gets easier every day. Here is an open-source library that makes integrating AI into an application extremely easy: Star the repository! This library alone can make React the best front-end framework out there! There are a bunch of cool things I like about CopilotKit. Here are 3 of them: 1. It allows you to take any -powered agent and bring it into your application. (This is a brand-new feature!) 2. You can build an AI-powered chatbot in your application. The chatbot will have access to your context and can act on the application. 3. You can build a RAG workflow to process and answer questions from a real-time knowledge base. I recorded a video to show you how simple it is to make some of this happen. A few lines of code, and you are in business. Here is a link to the sample application: CopilotKit is open-source. You can self-host it. You can use it with any LLM. Thanks to the team for showing me their tool and collaborating with me on this post!

Santiago

108,835 views โ€ข 2 years ago

I write dozens of cold email script variations in <15 minutes using ChatGPT-o3 + Claude Sonnet 4. Here's the full AI prompting process breakdown & walkthrough: There are two main phases of this script writing process. 1. Developing context 2. Setting guidelines Since AI has improved so much, the prompting process has become significantly simpler, as long as enough context has been given. This is essentially the full process behind this method: Context to begin with: - Onboarding form - Onboarding call recording - Deep research company summary Writing process: - Give Claude guardrails - Give messaging angles to emphasize - Give strict detail on variations needed AI is great, but it still requires human expertise since there is a lot of nuance that comes with cold email messaging. That's where the experts can shine in the prompting process. If you want access to the exact prompting frameworks I use to generate winning copy: Like + comment "Prompts" and I'll DM you the frameworks. (Must be following to receive)
14:59

I write dozens of cold email script variations in <15 minutes using ChatGPT-o3 + Claude Sonnet 4. Here's the full AI prompting process breakdown & walkthrough: There are two main phases of this script writing process. 1. Developing context 2. Setting guidelines Since AI has improved so much, the prompting process has become significantly simpler, as long as enough context has been given. This is essentially the full process behind this method: Context to begin with: - Onboarding form - Onboarding call recording - Deep research company summary Writing process: - Give Claude guardrails - Give messaging angles to emphasize - Give strict detail on variations needed AI is great, but it still requires human expertise since there is a lot of nuance that comes with cold email messaging. That's where the experts can shine in the prompting process. If you want access to the exact prompting frameworks I use to generate winning copy: Like + comment "Prompts" and I'll DM you the frameworks. (Must be following to receive)

Christian

14,864 views โ€ข 1 year ago

Some vendors are pitching AI as a โ€œset it and forget itโ€ solution in auto retail. Early use cases are showing that dealer input and context remain critical for performance. Monik Pamecha, CEO of Toma, and Brian Hoang, CEO of Mia Labs, break down how AI is being shaped by real-world dealership adoption. "One of the things that they mention is like: you just set it and forget it, you will never have to touch it, and it will just print money for you. That does not work, you have to put effort into the tool." " You understand your business more than anyone else. The tool needs that information and context to perform well. If you give it, you'll see crazy results." "The potential for these roles to evolve is coming, but it's not quite there yet. So if somebody comes in and they kind of feed off of your hunger to just eliminate all the staff, and save up on this SG&A, I would be very careful with how you evaluate the product." - Thank you to our partner Dealer Pay LLC! Check them out by visiting
2:38

Some vendors are pitching AI as a โ€œset it and forget itโ€ solution in auto retail. Early use cases are showing that dealer input and context remain critical for performance. Monik Pamecha, CEO of Toma, and Brian Hoang, CEO of Mia Labs, break down how AI is being shaped by real-world dealership adoption. "One of the things that they mention is like: you just set it and forget it, you will never have to touch it, and it will just print money for you. That does not work, you have to put effort into the tool." " You understand your business more than anyone else. The tool needs that information and context to perform well. If you give it, you'll see crazy results." "The potential for these roles to evolve is coming, but it's not quite there yet. So if somebody comes in and they kind of feed off of your hunger to just eliminate all the staff, and save up on this SG&A, I would be very careful with how you evaluate the product." - Thank you to our partner Dealer Pay LLC! Check them out by visiting

Car Dealership Guy

28,569 views โ€ข 11 months ago

holy shit, i built a better Cluely. introducing: Obscura, a better, more performant Cluely. Obscura helps you during your meeting and interview calls, in real time, with more context and a built-in agent to websearch in less than a second. it is completely INVISIBLE to your screen recorder, screen share and cannot be detected during your google calls or zoom calls no matter what. some functionalities: 1. toggle visibility: so that only you can see it 2. panic kill: if you wanna kill it in a second 3. screenshot ocr: for screenshot capabilities 4. context window: pass any context in .md or .docx file for it to read before your calls and Obscura will understand it 5. performative built-in agent: to search for anything in less than a second. it uses Deepgram for STT model and Anthropic / OpenAI api keys to generate answers. it takes $1 for 20 hours of your calls as compared to cluely's $20/mo or $75/mo plan (but that depends on tokens consumed on both models). releasing a .dmg for people for people to use. access will be given to people who share this with their friends. I will roll out a referral program so that people can share it with others. feel free to give some feedback. thanks to rahat for being on call with me and testing it.
2:46

holy shit, i built a better Cluely. introducing: Obscura, a better, more performant Cluely. Obscura helps you during your meeting and interview calls, in real time, with more context and a built-in agent to websearch in less than a second. it is completely INVISIBLE to your screen recorder, screen share and cannot be detected during your google calls or zoom calls no matter what. some functionalities: 1. toggle visibility: so that only you can see it 2. panic kill: if you wanna kill it in a second 3. screenshot ocr: for screenshot capabilities 4. context window: pass any context in .md or .docx file for it to read before your calls and Obscura will understand it 5. performative built-in agent: to search for anything in less than a second. it uses Deepgram for STT model and Anthropic / OpenAI api keys to generate answers. it takes $1 for 20 hours of your calls as compared to cluely's $20/mo or $75/mo plan (but that depends on tokens consumed on both models). releasing a .dmg for people for people to use. access will be given to people who share this with their friends. I will roll out a referral program so that people can share it with others. feel free to give some feedback. thanks to rahat for being on call with me and testing it.

Naruto11.eth

15,760 views โ€ข 5 months ago

This is, by far, one of the best uses of modern AI. If you don't use embeddings when querying your database, you are definitely leaving a lot on the table. In this video, I'll show you how to run semantic searches using OpenAI and PostgreSQL. It's all thanks to Pgai, an open-source PostgreSQL extension: Here's what will happen: 1. We'll create a simple table with news articles 2. We'll generate embeddings for those articles 3. We'll run queries on top of those embeddings For this video, I generated the embeddings using a simple query, but pgai Vectorizer would do the same automatically as new information makes it into the database. This is awesome! If you have a PostgreSQL database with data you are searching over, you should start experimenting with semantic searches immediately. For most use cases, a combination of full-text search + semantic search is the best approach. If you don't have a PostgreSQL database around, you can try free for 30 days using Timescale: Thanks to the TimescaleDB (by Tiger Data) team for partnering with me on this post!
19:19

This is, by far, one of the best uses of modern AI. If you don't use embeddings when querying your database, you are definitely leaving a lot on the table. In this video, I'll show you how to run semantic searches using OpenAI and PostgreSQL. It's all thanks to Pgai, an open-source PostgreSQL extension: Here's what will happen: 1. We'll create a simple table with news articles 2. We'll generate embeddings for those articles 3. We'll run queries on top of those embeddings For this video, I generated the embeddings using a simple query, but pgai Vectorizer would do the same automatically as new information makes it into the database. This is awesome! If you have a PostgreSQL database with data you are searching over, you should start experimenting with semantic searches immediately. For most use cases, a combination of full-text search + semantic search is the best approach. If you don't have a PostgreSQL database around, you can try free for 30 days using Timescale: Thanks to the TimescaleDB (by Tiger Data) team for partnering with me on this post!

Santiago

110,133 views โ€ข 1 year ago

Alex Karp's take on the Department of War designating Anthropic as a supply chain risk: "In the warfighting context, the Department of War has to be the arbiter of what gets deployed." "I want to split domestic and foreign. In this country have God-given rights [like the First, Second, and Fourth Amendments]...Adversaries trying to kill us do not have those rights. And I've never believed in extending our rights to foreign countries that are adversarial to us." "On domestic stuff...there are real issues. I'm super sympathetic with restrictions around the use of these products in the domestic context." "Just to give you an example, there are datasets that are publicly available in the US market that I don't think should be used against you and me in a law enforcement context with the help of AI agents and ontology." "But if you don't use [this] on the battlefield, obviously Iran's going to use [it]. You don't think they can go online and buy those products?" "Without going into somewhat classified data, those things in combination with other things โ€” [they're] lethal. A lot of people who want to hurt America end up dead because of our ability to aggregate and then figure out what's going on in the battlefield before they can figure out what we're doing." "So I'm very much in favor of it for moral reasons. But I'm also in favor of it because I don't know how else you explain this to the American people. We're going to take your job, we're going to eviscerate your ability to have money and power, but we're not going to defend you on the battlefield?"
4:53

Alex Karp's take on the Department of War designating Anthropic as a supply chain risk: "In the warfighting context, the Department of War has to be the arbiter of what gets deployed." "I want to split domestic and foreign. In this country have God-given rights [like the First, Second, and Fourth Amendments]...Adversaries trying to kill us do not have those rights. And I've never believed in extending our rights to foreign countries that are adversarial to us." "On domestic stuff...there are real issues. I'm super sympathetic with restrictions around the use of these products in the domestic context." "Just to give you an example, there are datasets that are publicly available in the US market that I don't think should be used against you and me in a law enforcement context with the help of AI agents and ontology." "But if you don't use [this] on the battlefield, obviously Iran's going to use [it]. You don't think they can go online and buy those products?" "Without going into somewhat classified data, those things in combination with other things โ€” [they're] lethal. A lot of people who want to hurt America end up dead because of our ability to aggregate and then figure out what's going on in the battlefield before they can figure out what we're doing." "So I'm very much in favor of it for moral reasons. But I'm also in favor of it because I don't know how else you explain this to the American people. We're going to take your job, we're going to eviscerate your ability to have money and power, but we're not going to defend you on the battlefield?"

TBPN

57,264 views โ€ข 5 months ago