Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Your Postgres is 100x slower than traditional OLAP engines. A deceptively simple OSS extension fixes this. Here's an interview where we dive into the deep engineering around how this is achieved. Joining me (and leading the conversation) is Marco Slot: an engineer with an EXTENSIVE and impressive career history... around PostgreSQL: 👉 Created pg_cron in 2017 (3.7k stars) - a tool to run cron-jobs in Postgres 👉 Built pg_incremental - fast, reliable, incremental batch processing inside PostgreSQL itself 👉 co-created pg_lake (after working on Crunchy Data's Warehouse, and getting acquired into Snowflake) 👉 Helped get pg_documentdb (MongoDB-on-Postgres) off the ground Marco Slot is a world-class expert in Postgres extensions. He seriously impressed me with his knowledge over the course of a private LinkedIn conversation, and now that I type out his resume - I understand where it came from. He should be on everyone's radar. So I brought him on the pod. In our full 2-hour deep-dive, we went over: • 🔥 how pg_lake makes analytics 100x faster (literally) • 🔥 perf internals like vectorized execution & CPU branching • 🤔 practical differences between OLTP and OLAP database development (and the age-old mission in uniting both) • 🤔 how (and why) pg_lake intercepts query plans and delegates parts of the query tree to DuckDB • 💡 why Postgres is architecturally terrible at analytical queries (and how vectorized execution fixes this) • 💡 Marco's hard-won experience through a decade+ career in Postgres • 🏆 Iceberg's role as the TCP/IP for tables • 🏆 what the real moat of PostgreSQL is Developments like pg_lake are a real reason why "Just Use Postgres" is much more than a meme, and it'll continue to dominate discourse. I promise you will learn a lot from this episode. Timestamps: (0:02) What is pg_lake? (2:23) Postgres' 100x slower problem and columnar storage experiments they had to make Postgres fast for analytics (6:00) practical examples and internals (16:20) perf internals - vectorized execution & CPU optimization (23:00) pg_lake architecture (why DuckDB isn't embedded) and the connection-per-process issue (29:16) how pg_lake intercepts the query plan tree and delegates parts to DuckDB (41:09) Iceberg catalogs (48:24) postgres to iceberg ingestion patterns (and pg_incremental) (53:40) Marco's (long) career: early AWS, Citus, Microsoft, Crunchy Data & Snowflake (1:04:20) Marco's observations around the merging between OLTP and OLAP (and the subtle dev differences there) (1:15:30) reverse ETL (1:33:08) Iceberg as the TCP/IP for tables (1:35:00) Marco's thoughts on the "Just Use Postgres" fevershow more

Stanislav Kozlovski

18,444 subscribers

16,910 Aufrufe • vor 7 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

3 Kommentare

Profilbild von Stanislav Kozlovski

Stanislav Kozlovskivor 7 Monaten

also on other platforms: - YouTube: - Spotify: - Apple Podcasts: - Your favorite podcast player (via RSS):

Profilbild von wanziforever

wanziforevervor 7 Monaten

is there a youtube link for this

Profilbild von PsudoMike 🇨🇦

PsudoMike 🇨🇦vor 5 Monaten

The 100x gap isn't the real story. Postgres was built for OLTP and its execution model reflects that. Row-by-row is fine at transactional scale, terrible for scans across millions of rows. For serious analytics, a dedicated columnar store isn't premature optimization.

Ähnliche Videos

Let Postgres own the Iceberg catalog and delegate analytics to DuckDB. The result => transactional lakehouse updates with fast analytical queries. This isn’t a concept. It’s exactly what pg_lake delivers today. pg_lake combines a set of extensions and components that let you query and modify Iceberg tables (and other lakehouse formats) directly from Postgres. DuckDB is used to accelerate analytical queries and runs in a sidecar process called pgduck_server, which communicates with Postgres during query execution. How it works (diagram below): 1. An application sends a query to Postgres to calculate unrealized PnL (Profit and Loss) for the Disney ticker. 2. Postgres parses the query and identifies the part that computes the average price from historical lakehouse data. 3. That part is forwarded to pgduck_server for accelerated execution. 4. pgduck_server delegates execution to DuckDB, which queries the lakehouse (reusing cached data if available). 5. DuckDB computes the average price and returns it to Postgres. 6. Postgres joins the result with local portfolio data and computes the unrealized PnL. 7. The final result is returned to the application.

Let Postgres own the Iceberg catalog and delegate analytics to DuckDB. The result => transactional lakehouse updates with fast analytical queries. This isn’t a concept. It’s exactly what pg_lake delivers today. pg_lake combines a set of extensions and components that let you query and modify Iceberg tables (and other lakehouse formats) directly from Postgres. DuckDB is used to accelerate analytical queries and runs in a sidecar process called pgduck_server, which communicates with Postgres during query execution. How it works (diagram below): 1. An application sends a query to Postgres to calculate unrealized PnL (Profit and Loss) for the Disney ticker. 2. Postgres parses the query and identifies the part that computes the average price from historical lakehouse data. 3. That part is forwarded to pgduck_server for accelerated execution. 4. pgduck_server delegates execution to DuckDB, which queries the lakehouse (reusing cached data if available). 5. DuckDB computes the average price and returns it to Postgres. 6. Postgres joins the result with local portfolio data and computes the unrealized PnL. 7. The final result is returned to the application.

Denis Magda

28,597 Aufrufe • vor 9 Monaten

Just Use Postgres feat. Denis Magda Denis literally just published a book called “Just Use Postgres”. In this conversation, that turned out to be super fun, we talked about: • what MySQL got wrong w.r.t community • the recent explosion of Postgres development in the world (Supabase, Neon, Crunchy, ClickHouse) • when Postgres instead of Kafka • the significance of the meme “just use postgres” • Postgres for data analytics Chapters (00:00) Intro (04:26) The Just Use Postgres Movement (08:36) The Small Data Trend and Its Implications (15:49) The Shift from Distributed Systems to Simplicity (17:15) Updating your priors re: PG capabilities (20:28) MySQL's OSS community split & our experiences with such things (39:48) Postgres for Analytics (55:49) PG in the lake house (59:27) AI Marketing Slop (01:02:19) How you'd build a brand new app on Postgres (01:05:24) Speedrun through PG extensions (01:11:56) PG as a Message Queue

Stanislav Kozlovski

13,036 Aufrufe • vor 9 Monaten

MySQL vs Postgres: storage engine addition. Clustered indexes (MySQL) and heap tables (Postgres) each have unique advantages. Covered the differences and practical implications in the latest stream.

MySQL vs Postgres: storage engine addition. Clustered indexes (MySQL) and heap tables (Postgres) each have unique advantages. Covered the differences and practical implications in the latest stream.

Ben Dicken

19,991 Aufrufe • vor 5 Monaten

Built a full-text search over all of Hacker News (~49M stories, comments, jobs & polls, syncing live per hour) to be 𝐬𝐮𝐫𝐩𝐫𝐢𝐬𝐢𝐧𝐠𝐥𝐲 𝐟𝐚𝐬𝐭 with no Next.js level cache and all magic at the Neon Postgres layer: 🎉 • Every keystroke is a live query against Neon Postgres • BM25 (powered by Lakebase Search) ranking runs inside the Postgres • No connection pool, just SQL-over-HTTP round trips • pg_prewarm loads the table + every index into cache • partial BM25 index per type, predicate matching the WHERE clause • Deployed on Vercel Just Postgres doing Postgres things, close to the compute 🐘

Built a full-text search over all of Hacker News (~49M stories, comments, jobs & polls, syncing live per hour) to be 𝐬𝐮𝐫𝐩𝐫𝐢𝐬𝐢𝐧𝐠𝐥𝐲 𝐟𝐚𝐬𝐭 with no Next.js level cache and all magic at the Neon Postgres layer: 🎉 • Every keystroke is a live query against Neon Postgres • BM25 (powered by Lakebase Search) ranking runs inside the Postgres • No connection pool, just SQL-over-HTTP round trips • pg_prewarm loads the table + every index into cache • partial BM25 index per type, predicate matching the WHERE clause • Deployed on Vercel Just Postgres doing Postgres things, close to the compute 🐘

Rishi Raj Jain

56,033 Aufrufe • vor 2 Monaten

How Postgres 17 improves IN queries In prior releases, postgres would loop and do multiple "primitive" index scans as many as your IN elements. In Postgres 17 it takes an array and eliminates the need for multiple scans. This only applies for constant values

How Postgres 17 improves IN queries In prior releases, postgres would loop and do multiple "primitive" index scans as many as your IN elements. In Postgres 17 it takes an array and eliminates the need for multiple scans. This only applies for constant values

Hussein Nasser

143,703 Aufrufe • vor 2 Jahren

Big moment for Postgres! AI agents broke the idea of what a database is supposed to do. Traditional databases were built for humans, and Agents broke that model. - They branch endlessly. - They run ten experiments at once. - They need isolation, context, memory, structured reasoning, and safe sandboxes. Letting agents touch production systems is terrifying because the old model of Postgres was never built for this kind of behavior. Agentic Postgres is an agent-ready version of Postgres by TimescaleDB (by Tiger Data) that solves this. I think it is one of the biggest upgrades to the Agent stack this year and Tiger Data is working with me on this post to share what they did. Some key features: > It instantly creates branches of an entire database, which is perfect for parallel agent evals, safe experiments, migrations, or isolated testing. Forks take seconds and cost almost nothing. > It comes with a built-in MCP server, which agents can use to get schema guidance, best practices, and safe, structured access to Postgres. This is also helpful to run migrations with a real understanding. > It comes with actual hybrid search (vector search and BM25), so Agents can retrieve data directly inside the database. > The database is Memory native. This gives a persistent context for Agents to evolve. This is one of the first times I have seen Postgres feel ready for the AI native era.

Big moment for Postgres! AI agents broke the idea of what a database is supposed to do. Traditional databases were built for humans, and Agents broke that model. - They branch endlessly. - They run ten experiments at once. - They need isolation, context, memory, structured reasoning, and safe sandboxes. Letting agents touch production systems is terrifying because the old model of Postgres was never built for this kind of behavior. Agentic Postgres is an agent-ready version of Postgres by TimescaleDB (by Tiger Data) that solves this. I think it is one of the biggest upgrades to the Agent stack this year and Tiger Data is working with me on this post to share what they did. Some key features: > It instantly creates branches of an entire database, which is perfect for parallel agent evals, safe experiments, migrations, or isolated testing. Forks take seconds and cost almost nothing. > It comes with a built-in MCP server, which agents can use to get schema guidance, best practices, and safe, structured access to Postgres. This is also helpful to run migrations with a real understanding. > It comes with actual hybrid search (vector search and BM25), so Agents can retrieve data directly inside the database. > The database is Memory native. This gives a persistent context for Agents to evolve. This is one of the first times I have seen Postgres feel ready for the AI native era.

Avi Chawla

94,406 Aufrufe • vor 11 Monaten

Finish the week with a Postgres video 🌟 All about using JSON in Postgres: - when you should use it - how to use it efficiently - how JSONs actually get stored in postgres.

Finish the week with a Postgres video 🌟 All about using JSON in Postgres: - when you should use it - how to use it efficiently - how JSONs actually get stored in postgres.

Gwen (Chen) Shapira

27,636 Aufrufe • vor 2 Jahren

🐘 Introducing Postgres Internals in 3D. 🌟 Stop reading boring diagrams. Start exploring PostgreSQL. An interactive 3D map to understand how Postgres actually works. 100% free. Built for developers.

🐘 Introducing Postgres Internals in 3D. 🌟 Stop reading boring diagrams. Start exploring PostgreSQL. An interactive 3D map to understand how Postgres actually works. 100% free. Built for developers.

alex shapalov

46,945 Aufrufe • vor 28 Tagen

Meet the new look of Neon Postgres. A total of ~1,200 hours of teamwork and five months from the initial kickoff to bring this design beauty to prod. Rebranding, design, dev, and motion. Ship faster with Postgres!

Meet the new look of Neon Postgres. A total of ~1,200 hours of teamwork and five months from the initial kickoff to bring this design beauty to prod. Rebranding, design, dev, and motion. Ship faster with Postgres!

Alex Barashkov

105,285 Aufrufe • vor 10 Monaten

sqlit is a lazygit style SQL TUI for the terminal. It supports 10+ databases (Postgres, SQLite, Supabase, ClickHouse, DuckDB, etc), has query history, autocomplete and more. Peter Adams (Maxteabag on GitHub) made sqlit using textualize.io and is Terminal Tool of the Week! ⭐️

sqlit is a lazygit style SQL TUI for the terminal. It supports 10+ databases (Postgres, SQLite, Supabase, ClickHouse, DuckDB, etc), has query history, autocomplete and more. Peter Adams (Maxteabag on GitHub) made sqlit using textualize.io and is Terminal Tool of the Week! ⭐️

Terminal Trove

138,230 Aufrufe • vor 10 Monaten

🚨 Tiger Data - Creators of TimescaleDB (Yes! the team behind TimescaleDB) is quietly revolutionizing how we build agents. The Old Way: Glueing together Postgres + Vector DBs + Search tools = complexity. The New Way: Tiger Data’s Agentic Postgres handles it all natively 🔥 What’s now possible 🧵 ↓ ✦ Hybrid retrieval → combine keyword filtering and vector similarity in a single query ✦ Forkable databases → retrieval and safe experimentation without external systems ✦ Native Postgres semantics → everything stays queryable, inspectable, and debuggable in SQL Why this matters: → Fewer moving parts in RAG pipelines → No extra infrastructure to stitch together → Agents can iterate and learn safely → Production data stays isolated and protected This is an awesome shift! Stop bolting workflows onto your database. Let your database be the workflow 🔥

🚨 Tiger Data - Creators of TimescaleDB (Yes! the team behind TimescaleDB) is quietly revolutionizing how we build agents. The Old Way: Glueing together Postgres + Vector DBs + Search tools = complexity. The New Way: Tiger Data’s Agentic Postgres handles it all natively 🔥 What’s now possible 🧵 ↓ ✦ Hybrid retrieval → combine keyword filtering and vector similarity in a single query ✦ Forkable databases → retrieval and safe experimentation without external systems ✦ Native Postgres semantics → everything stays queryable, inspectable, and debuggable in SQL Why this matters: → Fewer moving parts in RAG pipelines → No extra infrastructure to stitch together → Agents can iterate and learn safely → Production data stays isolated and protected This is an awesome shift! Stop bolting workflows onto your database. Let your database be the workflow 🔥

Charly Wargnier ♨️

20,190 Aufrufe • vor 9 Monaten

Use Neon and ClickHouse to get the best of both worlds: the flexibility of Postgres for transactions + the performance of ClickHouse for analytics With PeerDB, data streaming is simple 👇

Use Neon and ClickHouse to get the best of both worlds: the flexibility of Postgres for transactions + the performance of ClickHouse for analytics With PeerDB, data streaming is simple 👇

Neon

11,464 Aufrufe • vor 2 Jahren

After decades of building databases, Databricks VP of Engineering Nikita | Scaling Postgres believes it's time to rethink one of their fundamental assumptions. In this episode of the 𝘐𝘵'𝘴 𝘈𝘣𝘰𝘶𝘵 𝘋𝘢𝘵𝘢 podcast, Nikita discusses why separating compute and storage is the next major architectural shift for operational databases, how storing data on object storage opens it to downstream systems, and why the rise of AI agents is changing how developers build software—and what they need from a database. Watch the full conversation:

After decades of building databases, Databricks VP of Engineering Nikita | Scaling Postgres believes it's time to rethink one of their fundamental assumptions. In this episode of the 𝘐𝘵'𝘴 𝘈𝘣𝘰𝘶𝘵 𝘋𝘢𝘵𝘢 podcast, Nikita discusses why separating compute and storage is the next major architectural shift for operational databases, how storing data on object storage opens it to downstream systems, and why the rise of AI agents is changing how developers build software—and what they need from a database. Watch the full conversation:

Databricks

10,520 Aufrufe • vor 4 Monaten

Inverted indexes are the perfect structure for text search. "Inverted" here doesn't mean upside-down. Rather that it's an inversion of the original data. We take every distinct word from a set of documents and build a "posting list" of references back to every location where that word is used. When structured right, this lets databases like Postgres do fast lookup of words and phrases. TIN takes all this to the next level for Postgres. Span queries, fuzzy matching, wildcard matching, regex. All while strictly holding to Postgres durability, replication, and transactional semantics. Also, it's FAST. Who doesn't love a good benchmark?

Inverted indexes are the perfect structure for text search. "Inverted" here doesn't mean upside-down. Rather that it's an inversion of the original data. We take every distinct word from a set of documents and build a "posting list" of references back to every location where that word is used. When structured right, this lets databases like Postgres do fast lookup of words and phrases. TIN takes all this to the next level for Postgres. Span queries, fuzzy matching, wildcard matching, regex. All while strictly holding to Postgres durability, replication, and transactional semantics. Also, it's FAST. Who doesn't love a good benchmark?

Ben Dicken

32,297 Aufrufe • vor 1 Monat

We just shipped Managed DBs in Mastra platform. Over the last couple months, a number of folks getting started asked us for off-the-shelf storage So we added support for Postgres (backed by Neon Postgres) and LibSQL/Turso.

We just shipped Managed DBs in Mastra platform. Over the last couple months, a number of folks getting started asked us for off-the-shelf storage So we added support for Postgres (backed by Neon Postgres) and LibSQL/Turso.

Sam Bhagwat

27,789 Aufrufe • vor 4 Monaten

How Cloudflare cut query times by 35x without leaving Postgres: Their Postgres tables hit billions of rows, and every time-range query started getting slower. Plain Postgres scans the entire table on every query, regardless of the time window. They tried the manual route by building precomputed aggregates with cron jobs and evaluating ClickHouse, which needed a full ingestion pipeline just to handle their write pattern. Here's where most teams get stuck at exactly this point. They utilize manual partitioning, splitting data into child tables by day or month. Then they wire up cron jobs to refresh aggregate tables. Every schema change after that requires updating the cron logic and coordinating across teams. The infrastructure becomes the project. Tiger Cloud is managed TimescaleDB by Tiger Data - Creators of TimescaleDB on the Postgres you already know, with automatic time-based partitioning, continuous aggregates, and compression built in from the start. Cloudflare moved to TimescaleDB after exhausting the manual route and saw 5-35x query performance improvement on the same data. Here is how it works: → Hypertables partition data by timestamp automatically. Every time-range query hits only the relevant chunk, not the full table. → Continuous aggregates refresh incrementally in the background, with no cron jobs to maintain. To show what this looks like in practice, I built a real-time earthquake intelligence dashboard on a 3D globe using Claude Code and Tiger Cloud in a single session. The USGS earthquake catalog has 400,000+ events since 1900, each a timestamped row. The dashboard plots them as ripple animations sized by magnitude and colored by depth, with a time slider that scrubs through 120 years of seismic history. Every slider position fires a live query against the hypertable, and the side panel pulls from continuous aggregates. Claude Code connected to Tiger Cloud through the Tiger CLI MCP server, provisioned the database, pulled the USGS catalog, and assembled the full Next.js and Three.js frontend without leaving the session. The video below shows the final build in action, and I worked with the Tiger Data team to put this together. Tiger CLI is open-source (Apache 2.0) and works with Claude Code, Cursor, Codex, Gemini CLI, and VS Code. To try this yourself: → Sign up for Tiger Cloud here: New accounts get $1,000 in free credits, no credit card required. → Install Tiger CLI: curl -fsSL https(:)//cli(.)tigerdata(.)com | sh → Run tiger mcp install claude-code → Give Claude Code a prompt and let it build. I also wrote a full walkthrough on how you can turn any coding agent into a production-grade data engineer that can manage over a billion-row Postgres workloads. It covers everything from the database setup to the final build. Read it below.

How Cloudflare cut query times by 35x without leaving Postgres: Their Postgres tables hit billions of rows, and every time-range query started getting slower. Plain Postgres scans the entire table on every query, regardless of the time window. They tried the manual route by building precomputed aggregates with cron jobs and evaluating ClickHouse, which needed a full ingestion pipeline just to handle their write pattern. Here's where most teams get stuck at exactly this point. They utilize manual partitioning, splitting data into child tables by day or month. Then they wire up cron jobs to refresh aggregate tables. Every schema change after that requires updating the cron logic and coordinating across teams. The infrastructure becomes the project. Tiger Cloud is managed TimescaleDB by Tiger Data - Creators of TimescaleDB on the Postgres you already know, with automatic time-based partitioning, continuous aggregates, and compression built in from the start. Cloudflare moved to TimescaleDB after exhausting the manual route and saw 5-35x query performance improvement on the same data. Here is how it works: → Hypertables partition data by timestamp automatically. Every time-range query hits only the relevant chunk, not the full table. → Continuous aggregates refresh incrementally in the background, with no cron jobs to maintain. To show what this looks like in practice, I built a real-time earthquake intelligence dashboard on a 3D globe using Claude Code and Tiger Cloud in a single session. The USGS earthquake catalog has 400,000+ events since 1900, each a timestamped row. The dashboard plots them as ripple animations sized by magnitude and colored by depth, with a time slider that scrubs through 120 years of seismic history. Every slider position fires a live query against the hypertable, and the side panel pulls from continuous aggregates. Claude Code connected to Tiger Cloud through the Tiger CLI MCP server, provisioned the database, pulled the USGS catalog, and assembled the full Next.js and Three.js frontend without leaving the session. The video below shows the final build in action, and I worked with the Tiger Data team to put this together. Tiger CLI is open-source (Apache 2.0) and works with Claude Code, Cursor, Codex, Gemini CLI, and VS Code. To try this yourself: → Sign up for Tiger Cloud here: New accounts get $1,000 in free credits, no credit card required. → Install Tiger CLI: curl -fsSL https(:)//cli(.)tigerdata(.)com | sh → Run tiger mcp install claude-code → Give Claude Code a prompt and let it build. I also wrote a full walkthrough on how you can turn any coding agent into a production-grade data engineer that can manage over a billion-row Postgres workloads. It covers everything from the database setup to the final build. Read it below.

Avi Chawla

22,094 Aufrufe • vor 2 Monaten

With PGlite + Electric sync you can build an issue tracker handling 100k issues + 500k comments, synced from Postgres to PGlite in the browser! Super fast UI updates with all writes going though the local database, and Postgres FTS in PGlite for search 🔥

With PGlite + Electric sync you can build an issue tracker handling 100k issues + 500k comments, synced from Postgres to PGlite in the browser! Super fast UI updates with all writes going though the local database, and Postgres FTS in PGlite for search 🔥

Sam Willis

55,412 Aufrufe • vor 2 Jahren

Your Launchpad for Postgres: Get a Postgres connection URL in seconds, use your DB, claim it later. Perfect for building products like Netlify DB, OSS projects, and agents.

Your Launchpad for Postgres: Get a Postgres connection URL in seconds, use your DB, claim it later. Perfect for building products like Netlify DB, OSS projects, and agents.

Neon

21,090 Aufrufe • vor 1 Jahr

Mike Stonebraker is a Turing award winner famous for his fundamental contributions to databases (e.g. Postgres, C-Store and much more). I interviewed him recently about: • The story behind Postgres & the hardest technical challenge in building it • Where he disagreed with Google's technical decisions • Future problems in databases • Literature recommendations to learn databases • Why LLMs score 0% on his text-SQL benchmark • What if you replaced all state in an OS with a DB Timestamps: 0:00 - Intro 1:03 - How he got into databases 6:43 - Competing with Oracle 9:07 - What made Postgres special 15:55 - One size fits none 21:37 - Why he disagreed with Google 29:14 - Why he chose academia over big tech 30:58 - Replacing state in an OS with a DB 42:02 - Future problems in databases 51:36 - Technical book recommendations to learn databases 52:20 - Advice for younger self 55:52 - Outro Where to watch: • YouTube: • Spotify: • Apple Podcasts: • Transcript:

Mike Stonebraker is a Turing award winner famous for his fundamental contributions to databases (e.g. Postgres, C-Store and much more). I interviewed him recently about: • The story behind Postgres & the hardest technical challenge in building it • Where he disagreed with Google's technical decisions • Future problems in databases • Literature recommendations to learn databases • Why LLMs score 0% on his text-SQL benchmark • What if you replaced all state in an OS with a DB Timestamps: 0:00 - Intro 1:03 - How he got into databases 6:43 - Competing with Oracle 9:07 - What made Postgres special 15:55 - One size fits none 21:37 - Why he disagreed with Google 29:14 - Why he chose academia over big tech 30:58 - Replacing state in an OS with a DB 42:02 - Future problems in databases 51:36 - Technical book recommendations to learn databases 52:20 - Advice for younger self 55:52 - Outro Where to watch: • YouTube: • Spotify: • Apple Podcasts: • Transcript:

Ryan Peterman

118,875 Aufrufe • vor 6 Monaten

Your agents can't keep up with real-time data. Especially when it's scattered across dozens of sources. Most teams waste weeks building custom connectors for every database, API, and data warehouse. Then they build ETL pipelines to sync everything. By the time your agent retrieves the data, it's already outdated. Picture this: Your Postgres database updated 5 minutes ago. Your MongoDB collection changed 2 minutes ago. Your agent is still pulling from yesterday's snapshot. This is why most production RAG systems fail. There's a better approach: MindsDB is an open-source AI platform with a federated data engine that lets you query multiple data sources in real-time using SQL - without moving any data. Here's what makes it different: ↳ Your data stays in place. No ETL pipelines or data duplication ↳ Query Postgres, MongoDB, REST APIs, and more using consistent SQL ↳ JOIN across different sources in real-time with a unified interface ↳ Works with both structured and un-structured data And here's the best part: You don't even need to write SQL. Just describe what you want in plain English, and MindsDB converts it to SQL automatically. The system does all the heavy lifting. The breakthrough for AI agents is simple: When data updates at the source, your agent gets fresh results immediately. No sync delays. No stale embeddings. No custom code for each integration. You can literally write a SQL query that joins a Postgres table with a MongoDB collection and gets live results. This is what production AI applications need but rarely get. In this video, I give you a complete walkthrough of what we just discussed and how to actually do it. Make sure you watch this till the end. I've shared the link to MindsDB's GitHub repo in the next tweet!

Your agents can't keep up with real-time data. Especially when it's scattered across dozens of sources. Most teams waste weeks building custom connectors for every database, API, and data warehouse. Then they build ETL pipelines to sync everything. By the time your agent retrieves the data, it's already outdated. Picture this: Your Postgres database updated 5 minutes ago. Your MongoDB collection changed 2 minutes ago. Your agent is still pulling from yesterday's snapshot. This is why most production RAG systems fail. There's a better approach: MindsDB is an open-source AI platform with a federated data engine that lets you query multiple data sources in real-time using SQL - without moving any data. Here's what makes it different: ↳ Your data stays in place. No ETL pipelines or data duplication ↳ Query Postgres, MongoDB, REST APIs, and more using consistent SQL ↳ JOIN across different sources in real-time with a unified interface ↳ Works with both structured and un-structured data And here's the best part: You don't even need to write SQL. Just describe what you want in plain English, and MindsDB converts it to SQL automatically. The system does all the heavy lifting. The breakthrough for AI agents is simple: When data updates at the source, your agent gets fresh results immediately. No sync delays. No stale embeddings. No custom code for each integration. You can literally write a SQL query that joins a Postgres table with a MongoDB collection and gets live results. This is what production AI applications need but rarely get. In this video, I give you a complete walkthrough of what we just discussed and how to actually do it. Make sure you watch this till the end. I've shared the link to MindsDB's GitHub repo in the next tweet!

Akshay 🚀

65,672 Aufrufe • vor 1 Jahr