Загрузка видео...

Не удалось загрузить видео

На главную

Parsing a document accurately is one thing. Proving where every value came from is another. When a compliance team reviews an AI extraction, or an auditor needs to sign off on a figure pulled from a financial filing, "it came from this document" isn't enough. They need to see...

29,090 просмотров • 3 месяцев назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: ​ 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. ​ 2. The connector ecosystem to load data from unstructured data sources is very immature. ​ 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. ​ The goal of a RAG Pipeline is to solve these problems. ​ The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. ​ At a high level, there are four different stages in the architecture of a RAG pipeline: ​ 1. Ingestion: Here is where the pipeline loads the information from the data source. ​ 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. ​ 3. Transform: Where the pipeline chunks the data and generates document embeddings. ​ 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. ​ There are different rabbit holes at each one of these stages. Here are three of them: ​ 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. ​ 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. ​ 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. ​ In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. ​ I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. ​ ​ If you have a few documents lying around, set up a free account and give it a try.

Santiago

40,627 просмотров • 1 год назад

THIS IS F*CKING INSANE. picture asking your AI agent why it made a call six months ago, and actually getting the full answer: the data it pulled, the policies that applied, and the exact chain of reasoning that led there. that is Semantica, an open source Palantir for AI agents. every other setup forgets. an LLM plus a vector store makes a decision, moves on, and leaves you no trail of why. Semantica is a graph native layer that sits underneath your models and agents and turns your data into a living context graph, where every fact, relationship, and decision carries full provenance you can query and audit. why it changes memory engineering: > every fact remembers where it came from, W3C PROV-O compliant, exportable as JSON, RDF, or CSV > record a decision with its reasoning and confidence, then trace the whole causal chain back whenever you want > the reasoning layer is deterministic, Datalog, SPARQL, Rete, no LLM in the loop, so every path is explainable instead of guessed > bitemporal facts and time travel snapshots let you ask what the system knew at any point in the past > polyglot storage across RDF stores, property graphs like Neo4j, and vector stores like Qdrant this is memory built for the rooms where a black box gets you sued: > finance, healthcare, legal, government, compliance, anywhere a decision needs a paper trail and it drops into what you already run: > CrewAI, Agno, an MCP server, LiteLLM for OpenAI, Anthropic, Gemini, and Ollama, plus Claude Code and Cursor one line to start: pip install semantica bookmark this. the next leap in agent memory is not more embeddings, it is memory that can prove why.

NO1ennn

11,750 просмотров • 1 месяц назад