Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

A quick introduction for those who'd like a 𝗿𝗲𝗳𝗿𝗲𝘀𝗵𝗲𝗿 𝗼𝗻 𝗥𝗔𝗚 or missed THE 2024 AI hype 😆 (Since 2024 isn't over yet, I'll take my chance to explain it) Mary wrote this amazing introductory blog post for Weaviate AI Database, and I thought, why not make a video... show more

Femke Plantinga

13,204 subscribers

20,558 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Check it out: An entire lesson from BloomTech's AI for Developer Productivity course! Fundamentals of RAG (Retrieval-Augmented-Generation). How we enhance accuracy and reliability of generative AI models. This is the foundation we build on to give AI important context.

Check it out: An entire lesson from BloomTech's AI for Developer Productivity course! Fundamentals of RAG (Retrieval-Augmented-Generation). How we enhance accuracy and reliability of generative AI models. This is the foundation we build on to give AI important context.

Austen Allred

79,529 views • 2 years ago

In this short video, I explain the concept of RAG (Retrieval-Augmented Generation) in a simple and easy-to-understand way. I hope it helps! Let me know your thoughts. The material is based on Prof. Tom Yeh's 'AI by Hand' resources.

In this short video, I explain the concept of RAG (Retrieval-Augmented Generation) in a simple and easy-to-understand way. I hope it helps! Let me know your thoughts. The material is based on Prof. Tom Yeh's 'AI by Hand' resources.

Cornelius Adejoro

28,210 views • 1 year ago

We're unlocking next-level AI capabilities with the launch of our Retrieval Augmentation Generation (RAG) architecture! 🌟 By integrating a sophisticated information retrieval system, RAG empowers our Large Language Model (LLM), Qwen, with an extra layer of context, allowing it to access industry-specific insights or a company's proprietary data on demand. At the heart of RAG's knowledgebase lies the Vector Engine – a powerhouse of semantic recall ensuring all data is meticulously preserved and readily accessible. Plus, with Alibaba Cloud's suite of vector database solutions, we're at the forefront of augmented data retrieval, offering unprecedented precision and speed. Get ready to transform your data strategy with RAG! #AlibabaCloud #AI #Innovation #DataStrategy #VectorEngine #RAG #AIandBigDataSummit2024 #AIForward

We're unlocking next-level AI capabilities with the launch of our Retrieval Augmentation Generation (RAG) architecture! 🌟 By integrating a sophisticated information retrieval system, RAG empowers our Large Language Model (LLM), Qwen, with an extra layer of context, allowing it to access industry-specific insights or a company's proprietary data on demand. At the heart of RAG's knowledgebase lies the Vector Engine – a powerhouse of semantic recall ensuring all data is meticulously preserved and readily accessible. Plus, with Alibaba Cloud's suite of vector database solutions, we're at the forefront of augmented data retrieval, offering unprecedented precision and speed. Get ready to transform your data strategy with RAG! #AlibabaCloud #AI #Innovation #DataStrategy #VectorEngine #RAG #AIandBigDataSummit2024 #AIForward

Alibaba Cloud

9,636,416 views • 2 years ago

Announcing a new Coursera course: Retrieval Augmented Generation (RAG) You'll learn to build high performance, production-ready RAG systems in this hands-on, in-depth course created by and taught by , experienced AI and ML engineer, researcher, and educator. RAG is a critical component today of many LLM-based applications in customer support, internal company Q&A systems, even many of the leading chatbots that use web search to answer your questions. This course teaches you in-depth how to make RAG work well. LLMs can produce generic or outdated responses, especially when asked specialized questions not covered in its training data. RAG is the most widely used technique for addressing this. It brings in data from new data sources, such as internal documents or recent news, to give the LLM the relevant context to private, recent, or specialized information. This lets it generate more grounded and accurate responses. In this course, you’ll learn to design and implement every part of a RAG system, from retrievers to vector databases to generation to evals. You’ll learn about the fundamental principles behind RAG and how to optimize it at both the component and whole-system levels. As AI evolves, RAG is evolving too. New models can handle longer context windows, reason more effectively, and can be parts of complex agentic workflows. One exciting growth area is Agentic RAG, in which an AI agent at runtime (rather than it being hardcoded at development time) autonomously decides what data to retrieve, and when/how to go deeper. Even with this evolution, access to high-quality data at runtime is essential, which is why RAG is a key part of so many applications. You'll learn via hands-on experiences to: - Build a RAG system with retrieval and prompt augmentation - Compare retrieval methods like BM25, semantic search, and Reciprocal Rank Fusion - Chunk, index, and retrieve documents using a Weaviate vector database and a news dataset - Develop a chatbot, using open-source LLMs hosted by Together AI, for a fictional store that answers product and FAQ questions - Use evals to drive improving reliability, and incorporate multi-modal data RAG is an important foundational technique. Become good at it through this course! Please sign up here:

Announcing a new Coursera course: Retrieval Augmented Generation (RAG) You'll learn to build high performance, production-ready RAG systems in this hands-on, in-depth course created by and taught by , experienced AI and ML engineer, researcher, and educator. RAG is a critical component today of many LLM-based applications in customer support, internal company Q&A systems, even many of the leading chatbots that use web search to answer your questions. This course teaches you in-depth how to make RAG work well. LLMs can produce generic or outdated responses, especially when asked specialized questions not covered in its training data. RAG is the most widely used technique for addressing this. It brings in data from new data sources, such as internal documents or recent news, to give the LLM the relevant context to private, recent, or specialized information. This lets it generate more grounded and accurate responses. In this course, you’ll learn to design and implement every part of a RAG system, from retrievers to vector databases to generation to evals. You’ll learn about the fundamental principles behind RAG and how to optimize it at both the component and whole-system levels. As AI evolves, RAG is evolving too. New models can handle longer context windows, reason more effectively, and can be parts of complex agentic workflows. One exciting growth area is Agentic RAG, in which an AI agent at runtime (rather than it being hardcoded at development time) autonomously decides what data to retrieve, and when/how to go deeper. Even with this evolution, access to high-quality data at runtime is essential, which is why RAG is a key part of so many applications. You'll learn via hands-on experiences to: - Build a RAG system with retrieval and prompt augmentation - Compare retrieval methods like BM25, semantic search, and Reciprocal Rank Fusion - Chunk, index, and retrieve documents using a Weaviate vector database and a news dataset - Develop a chatbot, using open-source LLMs hosted by Together AI, for a fictional store that answers product and FAQ questions - Use evals to drive improving reliability, and incorporate multi-modal data RAG is an important foundational technique. Become good at it through this course! Please sign up here:

Andrew Ng

124,656 views • 1 year ago

Today, we’re excited to announce the general availability of the Contextual AI Platform. This is the first enterprise platform designed for building specialized RAG agents to support expert knowledge work. What is a specialized RAG agent? First, a general-purpose AI agent is one designed to automate simple daily tasks like scheduling a meeting or responding to an email. On the other hand, a specialized RAG agent is one designed to augment subject-matter experts performing complex domain-specific work. The Contextual AI Platform allows you to create these agents easily and achieve SOTA accuracy right out of the box. Check out what the Contextual AI Platform can do.

Today, we’re excited to announce the general availability of the Contextual AI Platform. This is the first enterprise platform designed for building specialized RAG agents to support expert knowledge work. What is a specialized RAG agent? First, a general-purpose AI agent is one designed to automate simple daily tasks like scheduling a meeting or responding to an email. On the other hand, a specialized RAG agent is one designed to augment subject-matter experts performing complex domain-specific work. The Contextual AI Platform allows you to create these agents easily and achieve SOTA accuracy right out of the box. Check out what the Contextual AI Platform can do.

Contextual AI

879,586 views • 1 year ago

everybody talks about building AI chatbots, but nobody tells you HOW to do it that's why I made a full practical walkthrough on how to build an AI chatbot that's hooked up to your own custom knowledgebase inside of the walk-through i go over: – data collection: gathering all relevant documents, conversations, and info - preprocessing: cleaning up and formatting the collected data - chunking: break down the cleaned data into smaller, manageable pieces - embedding & storing in a vector database - RAG & chatbot integration: using RAG to allow the chatbot to retrieve relevant information from the vector database based on a user's question reply to this tweet w/ the word “RAG” & I’ll send it to you (must be following so I can DM)

everybody talks about building AI chatbots, but nobody tells you HOW to do it that's why I made a full practical walkthrough on how to build an AI chatbot that's hooked up to your own custom knowledgebase inside of the walk-through i go over: – data collection: gathering all relevant documents, conversations, and info - preprocessing: cleaning up and formatting the collected data - chunking: break down the cleaned data into smaller, manageable pieces - embedding & storing in a vector database - RAG & chatbot integration: using RAG to allow the chatbot to retrieve relevant information from the vector database based on a user's question reply to this tweet w/ the word “RAG” & I’ll send it to you (must be following so I can DM)

Tyler

83,619 views • 1 year ago

Zoey Uso confirms that white people do not use rags or soap to clean themselves after using the toilet "It is true that white people do not use a rag" "So what do you use? Just a bar of soap?" "No, I use a rag. What do average Caucasian person use? Their hands!" "I had a friend recently and I was like, 'You don't use a rag? You use your hands?' and I was like, 'What are you talking about?' and I'm like, this is actually a thing!"

Zoey Uso confirms that white people do not use rags or soap to clean themselves after using the toilet "It is true that white people do not use a rag" "So what do you use? Just a bar of soap?" "No, I use a rag. What do average Caucasian person use? Their hands!" "I had a friend recently and I was like, 'You don't use a rag? You use your hands?' and I was like, 'What are you talking about?' and I'm like, this is actually a thing!"

Riddler

18,832 views • 1 month ago

How to Make AI Models More Useful? Discover the potential of AI and blockchain, and how features like RAG and TEEs can help NEAR unlock its full potential to build the world's best truly open AI. TL;DR - RAG (Retrieval-Augmented Generation) enhances AI agents by personalizing responses with relevant context. - Blockchain ensures transactions are processed transparently, efficiently, and securely. - NEAR Protocol is designed for AI, featuring a sharding architecture that supports high transaction volumes at low fees without compromising performance. It also offers privacy, user-owned data, and monetization opportunities. Study the NEAR stack, explore how to make AI models useful:

How to Make AI Models More Useful? Discover the potential of AI and blockchain, and how features like RAG and TEEs can help NEAR unlock its full potential to build the world's best truly open AI. TL;DR - RAG (Retrieval-Augmented Generation) enhances AI agents by personalizing responses with relevant context. - Blockchain ensures transactions are processed transparently, efficiently, and securely. - NEAR Protocol is designed for AI, featuring a sharding architecture that supports high transaction volumes at low fees without compromising performance. It also offers privacy, user-owned data, and monetization opportunities. Study the NEAR stack, explore how to make AI models useful:

NEARWEEK

23,509 views • 1 year ago

Here is what openai's new voice mode looks like with RAG. This lets you connect the voice agent to external sources to like company files, a website or a database. The world of customer support just changed forever Check it out

Here is what openai's new voice mode looks like with RAG. This lets you connect the voice agent to external sources to like company files, a website or a database. The world of customer support just changed forever Check it out

Yasser

300,407 views • 2 years ago

Introducing RAG CLI 🧑‍💻🔎 - a dead-simple command-line tool that allows you to RAG literally any file on your local machine. Index any files including glob patterns, such as `$ llamaindex-cli rag --files "./docs/**/*.rst”` To search simply do `$ llamaindex-cli rag --question "What is LlamaIndex?”` Built with our IngestionPipeline and QueryPipeline abstractions that allow efficient, performant data indexing and retrieval. Also built with Chroma. That’s not all! 💡Want to use local models like ollama? 💡Want to bake in advanced ingestion and retrieval techniques? The guide also shows you how to customize the logic for your own needs! Blog post: Docs:

Introducing RAG CLI 🧑‍💻🔎 - a dead-simple command-line tool that allows you to RAG literally any file on your local machine. Index any files including glob patterns, such as `$ llamaindex-cli rag --files "./docs/**/*.rst”` To search simply do `$ llamaindex-cli rag --question "What is LlamaIndex?”` Built with our IngestionPipeline and QueryPipeline abstractions that allow efficient, performant data indexing and retrieval. Also built with Chroma. That’s not all! 💡Want to use local models like ollama? 💡Want to bake in advanced ingestion and retrieval techniques? The guide also shows you how to customize the logic for your own needs! Blog post: Docs:

LlamaIndex 🦙

67,888 views • 2 years ago

Let's build a dashboard to evaluate and monitor your Agentic and RAG apps! . . In this video, I'll guide you through creating an evaluation and observability pipeline for your AI apps using a 100% open-source tool! Tech Stack: - Comet's Opik to eval and monitor - LlamaIndex to build a RAG pipeline - Ragas for synthetic datagen You'll learn: - Setting up Opik - Building a RAG pipeline - Creating an eval dataset - Evaluating the RAG pipeline - Monitoring all activities during the process It's a hands on demo with code and step-by-step guide to do everything listed above. CometML's Opik is fully open-source, offering the most Pythonic and easiest way to monitor LLM apps. I have shared link to their repo in next tweet!

Let's build a dashboard to evaluate and monitor your Agentic and RAG apps! . . In this video, I'll guide you through creating an evaluation and observability pipeline for your AI apps using a 100% open-source tool! Tech Stack: - Comet's Opik to eval and monitor - LlamaIndex to build a RAG pipeline - Ragas for synthetic datagen You'll learn: - Setting up Opik - Building a RAG pipeline - Creating an eval dataset - Evaluating the RAG pipeline - Monitoring all activities during the process It's a hands on demo with code and step-by-step guide to do everything listed above. CometML's Opik is fully open-source, offering the most Pythonic and easiest way to monitor LLM apps. I have shared link to their repo in next tweet!

Akshay 🚀

20,058 views • 1 year ago

Verba is an open source Retrieval Augmented Generation (RAG) application that performs RAG on your own data. To showcase its capabilities, we've customized it as an Airbnb chatbot using Airbnb’s customer documentation. How it works: • Ask any questions, related to your booking, policies, or anything related to your Airbnb experience. • Get relevant, human-like responses: Verba provides natural and informative answers. • Access original sources: One of the standout features of RAG is its ability to directly indicate the sources it used to generate each response. Under the hood, Verba uses a RAG pipeline to deliver these exceptional results. Your query is transformed into a numerical representation (vector) and be used to search through our vector database for the most similar context using Hybrid Search. The most relevant context is then combined with your original question and fed into a powerful large language model (LLM). The LLM will then use all of that information to generate a conversational response. Et voilà! 💫 Try Verba: Verba on GitHub: Learn more in our video:

Verba is an open source Retrieval Augmented Generation (RAG) application that performs RAG on your own data. To showcase its capabilities, we've customized it as an Airbnb chatbot using Airbnb’s customer documentation. How it works: • Ask any questions, related to your booking, policies, or anything related to your Airbnb experience. • Get relevant, human-like responses: Verba provides natural and informative answers. • Access original sources: One of the standout features of RAG is its ability to directly indicate the sources it used to generate each response. Under the hood, Verba uses a RAG pipeline to deliver these exceptional results. Your query is transformed into a numerical representation (vector) and be used to search through our vector database for the most similar context using Hybrid Search. The most relevant context is then combined with your original question and fed into a powerful large language model (LLM). The LLM will then use all of that information to generate a conversational response. Et voilà! 💫 Try Verba: Verba on GitHub: Learn more in our video:

Femke Plantinga

120,565 views • 2 years ago

New short course on advanced retrieval for RAG (retrieval augmented generation)! RAG fetches relevant documents to give context to an LLM. In Advanced Retrieval for AI with Chroma, taught by Chroma founder anton 🇺🇸, you’ll learn: (i) Query expansion using an LLM to rewrite and improve a query, by either generating either additional relevant queries or a hypothetical answer to the query. (ii) Reranking using a cross-encoder - a model trained to measure similarity between two inputs presented simultaneously. Reranking reorders retrieved documents based on the cross-encoder similarity measure. (iii) Constructing and training an Embedding Adaptor, which is a model that adapts the embedding values to be more relevant to your use case. Each of these techniques can help you build much better RAG systems. Please sign up for the course here:

New short course on advanced retrieval for RAG (retrieval augmented generation)! RAG fetches relevant documents to give context to an LLM. In Advanced Retrieval for AI with Chroma, taught by Chroma founder anton 🇺🇸, you’ll learn: (i) Query expansion using an LLM to rewrite and improve a query, by either generating either additional relevant queries or a hypothetical answer to the query. (ii) Reranking using a cross-encoder - a model trained to measure similarity between two inputs presented simultaneously. Reranking reorders retrieved documents based on the cross-encoder similarity measure. (iii) Constructing and training an Embedding Adaptor, which is a model that adapts the embedding values to be more relevant to your use case. Each of these techniques can help you build much better RAG systems. Please sign up for the course here:

Andrew Ng

191,219 views • 2 years ago

We’re excited to introduce RAGs v2 - build, customize, and use multiple ChatGPTs over your data, all with natural language 💬 A huge upgrade vs. the initial launch: 💫 Easily create multiple RAG pipelines and save them 💫 Easily swap between and customize each one (e.g. over different data, or w/ different system prompts) 💫 Delete unused RAG pipelines 💫 (dev quality) added much-needed linting/CI Check out the video 🎥 for details. It’s super easy to setup and use. Some additional features: 🧠 Supports a lot of LLMs both for building RAG and within each RAG pipeline 🌐 Supports loading load files or web pages. Check out our repo here:

We’re excited to introduce RAGs v2 - build, customize, and use multiple ChatGPTs over your data, all with natural language 💬 A huge upgrade vs. the initial launch: 💫 Easily create multiple RAG pipelines and save them 💫 Easily swap between and customize each one (e.g. over different data, or w/ different system prompts) 💫 Delete unused RAG pipelines 💫 (dev quality) added much-needed linting/CI Check out the video 🎥 for details. It’s super easy to setup and use. Some additional features: 🧠 Supports a lot of LLMs both for building RAG and within each RAG pipeline 🌐 Supports loading load files or web pages. Check out our repo here:

LlamaIndex 🦙

124,202 views • 2 years ago

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. 2. The connector ecosystem to load data from unstructured data sources is very immature. 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. The goal of a RAG Pipeline is to solve these problems. The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. At a high level, there are four different stages in the architecture of a RAG pipeline: 1. Ingestion: Here is where the pipeline loads the information from the data source. 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. 3. Transform: Where the pipeline chunks the data and generates document embeddings. 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. There are different rabbit holes at each one of these stages. Here are three of them: 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. If you have a few documents lying around, set up a free account and give it a try.

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. 2. The connector ecosystem to load data from unstructured data sources is very immature. 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. The goal of a RAG Pipeline is to solve these problems. The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. At a high level, there are four different stages in the architecture of a RAG pipeline: 1. Ingestion: Here is where the pipeline loads the information from the data source. 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. 3. Transform: Where the pipeline chunks the data and generates document embeddings. 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. There are different rabbit holes at each one of these stages. Here are three of them: 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. If you have a few documents lying around, set up a free account and give it a try.

Santiago

40,627 views • 1 year ago

I’m excited to announce a new course with DeepLearning.AI - Building Agentic RAG 💫 In this course, you’ll learn how to build a research assistant that can reason over multiple documents and answer complex questions. You’ll also learn how to step through the execution of the agent and steer it with human feedback. This represents a big step beyond any standard RAG pipeline, which is mostly good for simple questions over a small set of documents. Learn the layers first and then put them together: ✅ Routing ✅ Tool Use ✅ Multi-step reasoning with Memory ✅ Tool retrieval ✅ Debugging + user input Check it out!

I’m excited to announce a new course with DeepLearning.AI - Building Agentic RAG 💫 In this course, you’ll learn how to build a research assistant that can reason over multiple documents and answer complex questions. You’ll also learn how to step through the execution of the agent and steer it with human feedback. This represents a big step beyond any standard RAG pipeline, which is mostly good for simple questions over a small set of documents. Learn the layers first and then put them together: ✅ Routing ✅ Tool Use ✅ Multi-step reasoning with Memory ✅ Tool retrieval ✅ Debugging + user input Check it out!

Jerry Liu

76,298 views • 2 years ago

Okay, this is probably the best RAG UI I have ever seen. Just wow. I got a local version running in about 30 seconds with my existing Weaviate data. Here are some of the things I loved right away: 1) You can examine the raw data by collection 2) You can chat 3) You can see a tree for how the responses were generated 4) You can provide real-time feedback that can be used to train a model to improve the RAG workflow. 5) Still testing, but lots more! Well done Weaviate AI Database team!

Okay, this is probably the best RAG UI I have ever seen. Just wow. I got a local version running in about 30 seconds with my existing Weaviate data. Here are some of the things I loved right away: 1) You can examine the raw data by collection 2) You can chat 3) You can see a tree for how the responses were generated 4) You can provide real-time feedback that can be used to train a model to improve the RAG workflow. 5) Still testing, but lots more! Well done Weaviate AI Database team!

William J.B. Mattingly

33,340 views • 1 year ago

I love seeing Manchester United fans and rag loving media cry like babies over the offside goal yesterday. Remember when Manchester City didn't get a penalty for this? Should have been a penalty and a straight red card. Didn't see the media make a massive deal out of it.

I love seeing Manchester United fans and rag loving media cry like babies over the offside goal yesterday. Remember when Manchester City didn't get a penalty for this? Should have been a penalty and a straight red card. Didn't see the media make a massive deal out of it.

Luke🏆🏆🏆

66,816 views • 15 days ago