Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Lets build `Auto-RAG` where we let the LLM pull the data it needs from different sources. 🔎 The user asks a question. 🤔 LLM decides whether to search its knowledge, memory, internet or make an API call. ✍️ LLM answers with the context. Code:

Ashpreet Bedi

19,959 subscribers

174,580 views • 2 years ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

10 Comments

Vexxter2 years ago

is there any way to run a local quantized LLM via ollama in this?? amazing project btw!

Ashpreet Bedi2 years ago

@XPhyxer1 absolutely the Hermes2-llama3 might work well here :)

Jordan A. Metzner2 years ago

Just read the read me. Any plans for Groq on Llama 3.

Ashpreet Bedi2 years ago

@mrjmetz on it!

Ameriki Singh 🈳2 years ago

Would love to see groq and Llmma3 on it

Emma.Ai2 years ago

wow, can't wait to try this out

CoinCollector2 years ago

Ashpreet is coooooking

0xba0e7f9d2 years ago

🧑‍🚀this is awesome demo!

Aws Abdo, Ph.D.2 years ago

This work on automating retrieval and generation tasks is incredibly helpful. Thanks you! #MachineLearning #DataScience

Petamber2 years ago

You can also try Brave’s Search API for web search

Related Videos

Introducing `Personalized Agentic RAG` - where the LLM remembers key details about the user and automatically chooses the tool for RAG. We'll build: 👏 ChatGPT like memory ⚙️ Function calling 🧙 Multi-Agent orchestration code:

Introducing `Personalized Agentic RAG` - where the LLM remembers key details about the user and automatically chooses the tool for RAG. We'll build: 👏 ChatGPT like memory ⚙️ Function calling 🧙 Multi-Agent orchestration code:

Ashpreet Bedi

45,035 views • 2 years ago

Our new short course, Knowledge Graphs for RAG, is now available! Knowledge graphs are a data structure that is great at capturing complex relationships between data of multiple types. By enabling more sophisticated retrieval of text than similarity search alone, knowledge graphs can improve the context you pass to the LLM and the performance of your RAG applications. In this course, taught by Andreas Kollegger of Neo4j, you’ll - Explore how knowledge graphs work by building a graph of public financial documents from scratch - Learn to write queries that retrieve text and data from the graph and use it to enhance the context you pass to an LLM chatbot - Combine a knowledge graph with a question-answer chain to build better RAG-powered chat systems Sign up here!

Our new short course, Knowledge Graphs for RAG, is now available! Knowledge graphs are a data structure that is great at capturing complex relationships between data of multiple types. By enabling more sophisticated retrieval of text than similarity search alone, knowledge graphs can improve the context you pass to the LLM and the performance of your RAG applications. In this course, taught by Andreas Kollegger of Neo4j, you’ll - Explore how knowledge graphs work by building a graph of public financial documents from scratch - Learn to write queries that retrieve text and data from the graph and use it to enhance the context you pass to an LLM chatbot - Combine a knowledge graph with a question-answer chain to build better RAG-powered chat systems Sign up here!

Andrew Ng

244,435 views • 2 years ago

Verba is an open source Retrieval Augmented Generation (RAG) application that performs RAG on your own data. To showcase its capabilities, we've customized it as an Airbnb chatbot using Airbnb’s customer documentation. How it works: • Ask any questions, related to your booking, policies, or anything related to your Airbnb experience. • Get relevant, human-like responses: Verba provides natural and informative answers. • Access original sources: One of the standout features of RAG is its ability to directly indicate the sources it used to generate each response. Under the hood, Verba uses a RAG pipeline to deliver these exceptional results. Your query is transformed into a numerical representation (vector) and be used to search through our vector database for the most similar context using Hybrid Search. The most relevant context is then combined with your original question and fed into a powerful large language model (LLM). The LLM will then use all of that information to generate a conversational response. Et voilà! 💫 Try Verba: Verba on GitHub: Learn more in our video:

Verba is an open source Retrieval Augmented Generation (RAG) application that performs RAG on your own data. To showcase its capabilities, we've customized it as an Airbnb chatbot using Airbnb’s customer documentation. How it works: • Ask any questions, related to your booking, policies, or anything related to your Airbnb experience. • Get relevant, human-like responses: Verba provides natural and informative answers. • Access original sources: One of the standout features of RAG is its ability to directly indicate the sources it used to generate each response. Under the hood, Verba uses a RAG pipeline to deliver these exceptional results. Your query is transformed into a numerical representation (vector) and be used to search through our vector database for the most similar context using Hybrid Search. The most relevant context is then combined with your original question and fed into a powerful large language model (LLM). The LLM will then use all of that information to generate a conversational response. Et voilà! 💫 Try Verba: Verba on GitHub: Learn more in our video:

Femke Plantinga

120,565 views • 2 years ago

Our first Generative AI short course in JavaScript! GitHub recently reported that JavaScript is again the world’s most popular programming language. To support web developers exploring and developing with generative AI, we just launched a new short course in JavaScript taught by Jacob Lee, founding engineer at . In Build LLM Apps with LangChain.js you’ll learn elements common in AI development, including: (i) Using data loaders to pull data from common sources such as PDFs, websites, and databases (ii) Prompts, which are used to provide the LLM context (iii) Modules to support RAG such as text splitters and integrations with vector stores (iv) Working with different models to write applications that are not vendor-specific (v) Parsers, which extract and format the output for your downstream code to process You’ll also build with the LangChain Expression Language, which lets you easily compose sequences (also called chains) of modules to perform complex tasks using LLMs. Putting all this together, you’ll also work on a conversational question-answering LLM application capable of using external data as context. Please sign up here:

Our first Generative AI short course in JavaScript! GitHub recently reported that JavaScript is again the world’s most popular programming language. To support web developers exploring and developing with generative AI, we just launched a new short course in JavaScript taught by Jacob Lee, founding engineer at . In Build LLM Apps with LangChain.js you’ll learn elements common in AI development, including: (i) Using data loaders to pull data from common sources such as PDFs, websites, and databases (ii) Prompts, which are used to provide the LLM context (iii) Modules to support RAG such as text splitters and integrations with vector stores (iv) Working with different models to write applications that are not vendor-specific (v) Parsers, which extract and format the output for your downstream code to process You’ll also build with the LangChain Expression Language, which lets you easily compose sequences (also called chains) of modules to perform complex tasks using LLMs. Putting all this together, you’ll also work on a conversational question-answering LLM application capable of using external data as context. Please sign up here:

Andrew Ng

284,472 views • 2 years ago

He is 18 built an LLM and sold it to Anthropic for $3.2M - and at Stanford shared how to repeat it from scratch: 00:30 - how to build an LLM worth $100k in one evening 21:34 - how one LLM controls 1000 processes at the same time 43:47 - from first prompt to $3.2M from Anthropic after watching I built my first LLM in 60 minutes- a week later Anthropic offered me $100k for the system. save & watch - the article below shows how to build an LLM and sell it to Anthropic for $3.2M from scratch

He is 18 built an LLM and sold it to Anthropic for $3.2M - and at Stanford shared how to repeat it from scratch: 00:30 - how to build an LLM worth $100k in one evening 21:34 - how one LLM controls 1000 processes at the same time 43:47 - from first prompt to $3.2M from Anthropic after watching I built my first LLM in 60 minutes- a week later Anthropic offered me $100k for the system. save & watch - the article below shows how to build an LLM and sell it to Anthropic for $3.2M from scratch

Noisy

46,671 views • 17 days ago

I built a chat bot with jev, no LLM at all! Responses are instant, no hallucinations. I hooked it up to web search, wikipedia, weather, todoist and home assistant. Jev decides what tool to call and what args to use based on the prompt. Instant answers cite sources as well!

I built a chat bot with jev, no LLM at all! Responses are instant, no hallucinations. I hooked it up to web search, wikipedia, weather, todoist and home assistant. Jev decides what tool to call and what args to use based on the prompt. Instant answers cite sources as well!

CJ (Coding Garden)

114,643 views • 14 days ago

He is a 1st year AI engineering student at Stanford - and already launched an automated system making $2.1M a year: 00:20 - how Anthropic engineers make millions with LLM and Transformers 38:34 - In-Context Learning makes AI smarter for free 59:47 - 1st year student made $2.1M using LLM from scratch after watching I gave LLM the idea I had been holding in my head for 3 years - it built it in one evening. save & watch - the article below is a step by step guide to build your automated LLM system.

He is a 1st year AI engineering student at Stanford - and already launched an automated system making $2.1M a year: 00:20 - how Anthropic engineers make millions with LLM and Transformers 38:34 - In-Context Learning makes AI smarter for free 59:47 - 1st year student made $2.1M using LLM from scratch after watching I gave LLM the idea I had been holding in my head for 3 years - it built it in one evening. save & watch - the article below is a step by step guide to build your automated LLM system.

Noisy

43,713 views • 2 months ago

Transforming Invoice Data into JSON: Local LLM with LlamaIndex & Pydantic 🚀 Complete video: Code: I explain how to get structured JSON output with LlamaIndex and dynamic Pydantic class. This helps to implement the use case of data extraction from invoice documents. The solution runs on the local machine, thanks to Ollama. I'm using a MacBook Air M1 with 8GB RAM. LlamaIndex 🦙 Pydantic ollama #Python #LLM #RAG

Transforming Invoice Data into JSON: Local LLM with LlamaIndex & Pydantic 🚀 Complete video: Code: I explain how to get structured JSON output with LlamaIndex and dynamic Pydantic class. This helps to implement the use case of data extraction from invoice documents. The solution runs on the local machine, thanks to Ollama. I'm using a MacBook Air M1 with 8GB RAM. LlamaIndex 🦙 Pydantic ollama #Python #LLM #RAG

Andrej Baranovskij

147,991 views • 2 years ago

JSX Tool (@jsxtool) is an in-browser IDE for building React UIs. Click on any element in the browser to jump straight to the relevant line of code and edit it yourself or provide precise context to an LLM to make updates.

JSX Tool (@jsxtool) is an in-browser IDE for building React UIs. Click on any element in the browser to jump straight to the relevant line of code and edit it yourself or provide precise context to an LLM to make updates.

Y Combinator

49,755 views • 10 months ago

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

201,127 views • 1 year ago

New short course on sophisticated RAG (Retrieval Augmented Generation) techniques is out! Taught by Jerry Liu and Anupam Datta of LlamaIndex 🦙 and TruEra , this teaches advanced techniques that help your LLM generate good answers. Topics include: - Sentence-window retrieval, which retrieves not just the most relevant sentence, but a window of sentences around it for higher quality context. - Auto-merging retrieval, which organizes your document into a hierarchical tree structure, where each parent node's text is split among its child nodes. Based on the relevance of the child nodes to a user query, this lets you better decide whether the entire parent node should be provided as context to the LLM. - Evaluation methodology for separately evaluating the quality of the key steps of RAG (context relevance, answer relevance, groundedness) so that you can perform error analysis, identify which part of your pipeline needs work, and tune components systematically. Please check out the course!

New short course on sophisticated RAG (Retrieval Augmented Generation) techniques is out! Taught by Jerry Liu and Anupam Datta of LlamaIndex 🦙 and TruEra , this teaches advanced techniques that help your LLM generate good answers. Topics include: - Sentence-window retrieval, which retrieves not just the most relevant sentence, but a window of sentences around it for higher quality context. - Auto-merging retrieval, which organizes your document into a hierarchical tree structure, where each parent node's text is split among its child nodes. Based on the relevance of the child nodes to a user query, this lets you better decide whether the entire parent node should be provided as context to the LLM. - Evaluation methodology for separately evaluating the quality of the key steps of RAG (context relevance, answer relevance, groundedness) so that you can perform error analysis, identify which part of your pipeline needs work, and tune components systematically. Please check out the course!

Andrew Ng

655,818 views • 2 years ago

He is 17 sold an LLM system to Anthropic for $2.2M - and at Stanford shared how to build and sell one from scratch: 00:20 - how to build an LLM system that Anthropic will buy for millions 08:34 - one prompt in Opus 5 and the LLM system is ready to sell in 60 minutes 23:47 - how to sell an LLM to Anthropic at 17 with no experience and reach $2.2M a year after watching I built my first LLM system in 60 minutes - and sold it to Anthropic for $120k. save & watch - the article below is one file that builds an LLM system and sells it to Anthropic for $2.2M.

He is 17 sold an LLM system to Anthropic for $2.2M - and at Stanford shared how to build and sell one from scratch: 00:20 - how to build an LLM system that Anthropic will buy for millions 08:34 - one prompt in Opus 5 and the LLM system is ready to sell in 60 minutes 23:47 - how to sell an LLM to Anthropic at 17 with no experience and reach $2.2M a year after watching I built my first LLM system in 60 minutes - and sold it to Anthropic for $120k. save & watch - the article below is one file that builds an LLM system and sells it to Anthropic for $2.2M.

Noisy

25,396 views • 19 days ago

LLM + Memory + Planning + Tools = Agents 🤖 Last month, Job and I discussed how generative AI is shifting how companies offer customer support. How can we add more layers to our RAG apps to make it more agentic? LLM: Large language model alone Memory: Short-term and long-term memory Planning: Breaking down questions into sub-questions, plan the steps needed before taking action, etc. Tools: Calculator, code executor, weather API, etc. I hope you find this small clip interesting! The link to the full recording is below.

LLM + Memory + Planning + Tools = Agents 🤖 Last month, Job and I discussed how generative AI is shifting how companies offer customer support. How can we add more layers to our RAG apps to make it more agentic? LLM: Large language model alone Memory: Short-term and long-term memory Planning: Breaking down questions into sub-questions, plan the steps needed before taking action, etc. Tools: Calculator, code executor, weather API, etc. I hope you find this small clip interesting! The link to the full recording is below.

Erika Shorten

32,443 views • 2 years ago

new project: cleo (kindle + llm) 📚 cleo is an ios app that pairs whatever book you're reading on kindle with an llm (o3) — ask questions, get recaps/summaries, or listen/discuss (think audiobooks but interactive). the llm has context on exactly where you are + the book contents. best part? no complex setup needed. just link your kindle account and that's it. it just works. reply / rt for testflight invite 👇

new project: cleo (kindle + llm) 📚 cleo is an ios app that pairs whatever book you're reading on kindle with an llm (o3) — ask questions, get recaps/summaries, or listen/discuss (think audiobooks but interactive). the llm has context on exactly where you are + the book contents. best part? no complex setup needed. just link your kindle account and that's it. it just works. reply / rt for testflight invite 👇

jpa

631,088 views • 1 year ago

Introducing /search - the simplest way for agents and devs to discover the web 👀 Our #1 most requested endpoint is finally here. Search the web AND scrape all results in an LLM-ready format with one API call. Now live on our API, MCP, and all integrations 🔥

Introducing /search - the simplest way for agents and devs to discover the web 👀 Our #1 most requested endpoint is finally here. Search the web AND scrape all results in an LLM-ready format with one API call. Now live on our API, MCP, and all integrations 🔥

Firecrawl

390,772 views • 1 year ago

RLM is the most import foundation of my Pi Harness (other than Pi of course). It's seeded with late interaction retrieval results (thanks to @lightonai for pylate). The Agent initiates it with query then.. 𝐒𝐞𝐭𝐮𝐩 A python REPL is created and seeded with: 1. Late interaction search to pre-filter. Instead of doing top 3/5/10, it's top hundreds of documents. This is set into a `context` variable. 2. Python functions are loaded in to do more searches if `context` variable isn't enough. And to make llm calls with cheaper models in parallel batches. 𝐈𝐭𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐋𝐨𝐨𝐩 From there, an LLM iterates in the REPL based on the query. It's just like exploring in a jupyter notebook. The LLM writes prose (like a markdown cell) and code to be run in the REPL each turn. This allows the LLM to sort, filter, and synthesize information. It can fan out and ask smaller models to summarize, combine, contrast, or do anything else to documents to help it understand the data. After several turns the LLM reponds with the final answer. Either because it found the answer, or hit the budget limit. Context as a Python variable, LLM as the programmer, REPL as the runtime. 𝐖𝐡𝐲 𝐃𝐨𝐞𝐬 𝐓𝐡𝐢𝐬 𝐖𝐨𝐫𝐤 1. Richer Shell. Agents (and subagents) work by intermixing code and prose/thinking. But they use static scripts or bash that run and exit and start over each tool call. That's not ideal for exploration and synthesis of data. For that, state is useful to continue building and exploring the data as you learn more. There's a reason jupyter notebooks have been popular with data scientists. 2. Keeps main agent context clean. The better context you have the better the agent will perform (duh!). This means three thing: better human input, less missing search results, and less incorrect search results. Letting the agent iterate allows it to synthesize just what is needed and nothing else. All bad paths or peeks at something that turns out to be irrelevant stays out of main agent context. 3. Stack the good ideas! People often compare late interaction search vs RLM. Or static vs dynamic languages. Or agentic search vs semantic search. But...You can just use them all together for what they're each good at. Use them all for the area they're really great for. Read the full post which has more detail about how and why.

RLM is the most import foundation of my Pi Harness (other than Pi of course). It's seeded with late interaction retrieval results (thanks to @lightonai for pylate). The Agent initiates it with query then.. 𝐒𝐞𝐭𝐮𝐩 A python REPL is created and seeded with: 1. Late interaction search to pre-filter. Instead of doing top 3/5/10, it's top hundreds of documents. This is set into a `context` variable. 2. Python functions are loaded in to do more searches if `context` variable isn't enough. And to make llm calls with cheaper models in parallel batches. 𝐈𝐭𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐋𝐨𝐨𝐩 From there, an LLM iterates in the REPL based on the query. It's just like exploring in a jupyter notebook. The LLM writes prose (like a markdown cell) and code to be run in the REPL each turn. This allows the LLM to sort, filter, and synthesize information. It can fan out and ask smaller models to summarize, combine, contrast, or do anything else to documents to help it understand the data. After several turns the LLM reponds with the final answer. Either because it found the answer, or hit the budget limit. Context as a Python variable, LLM as the programmer, REPL as the runtime. 𝐖𝐡𝐲 𝐃𝐨𝐞𝐬 𝐓𝐡𝐢𝐬 𝐖𝐨𝐫𝐤 1. Richer Shell. Agents (and subagents) work by intermixing code and prose/thinking. But they use static scripts or bash that run and exit and start over each tool call. That's not ideal for exploration and synthesis of data. For that, state is useful to continue building and exploring the data as you learn more. There's a reason jupyter notebooks have been popular with data scientists. 2. Keeps main agent context clean. The better context you have the better the agent will perform (duh!). This means three thing: better human input, less missing search results, and less incorrect search results. Letting the agent iterate allows it to synthesize just what is needed and nothing else. All bad paths or peeks at something that turns out to be irrelevant stays out of main agent context. 3. Stack the good ideas! People often compare late interaction search vs RLM. Or static vs dynamic languages. Or agentic search vs semantic search. But...You can just use them all together for what they're each good at. Use them all for the area they're really great for. Read the full post which has more detail about how and why.

Isaac Flath

42,874 views • 5 months ago