Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Let Postgres own the Iceberg catalog and delegate analytics to DuckDB. The result => transactional lakehouse updates with fast analytical queries. This isn’t a concept. It’s exactly what pg_lake delivers today. pg_lake combines a set of extensions and components that let you query and modify Iceberg tables (and other... lakehouse formats) directly from Postgres. DuckDB is used to accelerate analytical queries and runs in a sidecar process called pgduck_server, which communicates with Postgres during query execution. How it works (diagram below): 1. An application sends a query to Postgres to calculate unrealized PnL (Profit and Loss) for the Disney ticker. 2. Postgres parses the query and identifies the part that computes the average price from historical lakehouse data. 3. That part is forwarded to pgduck_server for accelerated execution. 4. pgduck_server delegates execution to DuckDB, which queries the lakehouse (reusing cached data if available). 5. DuckDB computes the average price and returns it to Postgres. 6. Postgres joins the result with local portfolio data and computes the unrealized PnL. 7. The final result is returned to the application.show more

Denis Magda

5,050 subscribers

28,597 Aufrufe • vor 8 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Data is Life

Data is Lifevor 8 Monaten

Just use DuckLake

Profilbild von Denis Magda

Denis Magdavor 8 Monaten

Does it let me query and join transactional data stored in Postgres with historical data in the lakehouse?

Profilbild von joaah

joaahvor 8 Monaten

@Data_ls_Life YES

Profilbild von Denis Magda

Denis Magdavor 8 Monaten

@Data_ls_Life I’l check it out. Thanks for sharing

Profilbild von Dipankar Sarkar

Dipankar Sarkarvor 8 Monaten

Sounds great!

Profilbild von Alexander Schröder

Alexander Schrödervor 8 Monaten

Classic federation. Works well if query can be pushed down and cardinality of fetched data is small. Will be painfull as soon as there is a big cross-database join or so.

Profilbild von Tobias_Petry.sql

Tobias_Petry.sqlvor 8 Monaten

> 4. pgduck_server delegates execution to DuckDB, which queries the lakehouse (reusing cached data if available). How does it reuse cached data? Did you setup a file-based cache for the lakehouse data?

Profilbild von fictiousxxl

fictiousxxlvor 8 Monaten

can we see the implementation of this so we understand the actual setup

Profilbild von Lars Hansen

Lars Hansenvor 8 Monaten

So compared to using DuckLake directly, here the DuckLake tables becomes first-class Postgres citizens? (instead of using special client)

Profilbild von Denis Magda

Denis Magdavor 8 Monaten

I'm not familiar with DuckLake yet. So, can't compare architectural differences.

Ähnliche Videos

Your Postgres is 100x slower than traditional OLAP engines. A deceptively simple OSS extension fixes this. Here's an interview where we dive into the deep engineering around how this is achieved. Joining me (and leading the conversation) is Marco Slot: an engineer with an EXTENSIVE and impressive career history around PostgreSQL: 👉 Created pg_cron in 2017 (3.7k stars) - a tool to run cron-jobs in Postgres 👉 Built pg_incremental - fast, reliable, incremental batch processing inside PostgreSQL itself 👉 co-created pg_lake (after working on Crunchy Data's Warehouse, and getting acquired into Snowflake) 👉 Helped get pg_documentdb (MongoDB-on-Postgres) off the ground Marco Slot is a world-class expert in Postgres extensions. He seriously impressed me with his knowledge over the course of a private LinkedIn conversation, and now that I type out his resume - I understand where it came from. He should be on everyone's radar. So I brought him on the pod. In our full 2-hour deep-dive, we went over: • 🔥 how pg_lake makes analytics 100x faster (literally) • 🔥 perf internals like vectorized execution & CPU branching • 🤔 practical differences between OLTP and OLAP database development (and the age-old mission in uniting both) • 🤔 how (and why) pg_lake intercepts query plans and delegates parts of the query tree to DuckDB • 💡 why Postgres is architecturally terrible at analytical queries (and how vectorized execution fixes this) • 💡 Marco's hard-won experience through a decade+ career in Postgres • 🏆 Iceberg's role as the TCP/IP for tables • 🏆 what the real moat of PostgreSQL is Developments like pg_lake are a real reason why "Just Use Postgres" is much more than a meme, and it'll continue to dominate discourse. I promise you will learn a lot from this episode. Timestamps: (0:02) What is pg_lake? (2:23) Postgres' 100x slower problem and columnar storage experiments they had to make Postgres fast for analytics (6:00) practical examples and internals (16:20) perf internals - vectorized execution & CPU optimization (23:00) pg_lake architecture (why DuckDB isn't embedded) and the connection-per-process issue (29:16) how pg_lake intercepts the query plan tree and delegates parts to DuckDB (41:09) Iceberg catalogs (48:24) postgres to iceberg ingestion patterns (and pg_incremental) (53:40) Marco's (long) career: early AWS, Citus, Microsoft, Crunchy Data & Snowflake (1:04:20) Marco's observations around the merging between OLTP and OLAP (and the subtle dev differences there) (1:15:30) reverse ETL (1:33:08) Iceberg as the TCP/IP for tables (1:35:00) Marco's thoughts on the "Just Use Postgres" fever

Your Postgres is 100x slower than traditional OLAP engines. A deceptively simple OSS extension fixes this. Here's an interview where we dive into the deep engineering around how this is achieved. Joining me (and leading the conversation) is Marco Slot: an engineer with an EXTENSIVE and impressive career history around PostgreSQL: 👉 Created pg_cron in 2017 (3.7k stars) - a tool to run cron-jobs in Postgres 👉 Built pg_incremental - fast, reliable, incremental batch processing inside PostgreSQL itself 👉 co-created pg_lake (after working on Crunchy Data's Warehouse, and getting acquired into Snowflake) 👉 Helped get pg_documentdb (MongoDB-on-Postgres) off the ground Marco Slot is a world-class expert in Postgres extensions. He seriously impressed me with his knowledge over the course of a private LinkedIn conversation, and now that I type out his resume - I understand where it came from. He should be on everyone's radar. So I brought him on the pod. In our full 2-hour deep-dive, we went over: • 🔥 how pg_lake makes analytics 100x faster (literally) • 🔥 perf internals like vectorized execution & CPU branching • 🤔 practical differences between OLTP and OLAP database development (and the age-old mission in uniting both) • 🤔 how (and why) pg_lake intercepts query plans and delegates parts of the query tree to DuckDB • 💡 why Postgres is architecturally terrible at analytical queries (and how vectorized execution fixes this) • 💡 Marco's hard-won experience through a decade+ career in Postgres • 🏆 Iceberg's role as the TCP/IP for tables • 🏆 what the real moat of PostgreSQL is Developments like pg_lake are a real reason why "Just Use Postgres" is much more than a meme, and it'll continue to dominate discourse. I promise you will learn a lot from this episode. Timestamps: (0:02) What is pg_lake? (2:23) Postgres' 100x slower problem and columnar storage experiments they had to make Postgres fast for analytics (6:00) practical examples and internals (16:20) perf internals - vectorized execution & CPU optimization (23:00) pg_lake architecture (why DuckDB isn't embedded) and the connection-per-process issue (29:16) how pg_lake intercepts the query plan tree and delegates parts to DuckDB (41:09) Iceberg catalogs (48:24) postgres to iceberg ingestion patterns (and pg_incremental) (53:40) Marco's (long) career: early AWS, Citus, Microsoft, Crunchy Data & Snowflake (1:04:20) Marco's observations around the merging between OLTP and OLAP (and the subtle dev differences there) (1:15:30) reverse ETL (1:33:08) Iceberg as the TCP/IP for tables (1:35:00) Marco's thoughts on the "Just Use Postgres" fever

Stanislav Kozlovski

16,910 Aufrufe • vor 6 Monaten

We're excited to announce duckdb-skills, a DuckDB plugin for Claude Code! We think the embedded nature of DuckDB makes it a perfect companion for Claude in your local workflows. The skills supported include: + read-file and query – uses DuckDB's CLI to query data locally, unlocking easy access to any file that DuckDB can read. + read-memories – a clever idea to store your Claude memories in DuckDB and query them at blazing speed. These are powered by two additional skills: + attach-db – gives Claude a mechanism to manage DuckDB state through a .sql file linked to your project. + duckdb-docs – uses a remote DuckDB full-text search database to query the DuckDB docs and answer all of your (and Claude's own) questions.

We're excited to announce duckdb-skills, a DuckDB plugin for Claude Code! We think the embedded nature of DuckDB makes it a perfect companion for Claude in your local workflows. The skills supported include: + read-file and query – uses DuckDB's CLI to query data locally, unlocking easy access to any file that DuckDB can read. + read-memories – a clever idea to store your Claude memories in DuckDB and query them at blazing speed. These are powered by two additional skills: + attach-db – gives Claude a mechanism to manage DuckDB state through a .sql file linked to your project. + duckdb-docs – uses a remote DuckDB full-text search database to query the DuckDB docs and answer all of your (and Claude's own) questions.

DuckDB

60,855 Aufrufe • vor 7 Monaten

Inverted indexes are the perfect structure for text search. "Inverted" here doesn't mean upside-down. Rather that it's an inversion of the original data. We take every distinct word from a set of documents and build a "posting list" of references back to every location where that word is used. When structured right, this lets databases like Postgres do fast lookup of words and phrases. TIN takes all this to the next level for Postgres. Span queries, fuzzy matching, wildcard matching, regex. All while strictly holding to Postgres durability, replication, and transactional semantics. Also, it's FAST. Who doesn't love a good benchmark?

Inverted indexes are the perfect structure for text search. "Inverted" here doesn't mean upside-down. Rather that it's an inversion of the original data. We take every distinct word from a set of documents and build a "posting list" of references back to every location where that word is used. When structured right, this lets databases like Postgres do fast lookup of words and phrases. TIN takes all this to the next level for Postgres. Span queries, fuzzy matching, wildcard matching, regex. All while strictly holding to Postgres durability, replication, and transactional semantics. Also, it's FAST. Who doesn't love a good benchmark?

Ben Dicken

32,297 Aufrufe • vor 1 Monat

sqlit is a lazygit style SQL TUI for the terminal. It supports 10+ databases (Postgres, SQLite, Supabase, ClickHouse, DuckDB, etc), has query history, autocomplete and more. Peter Adams (Maxteabag on GitHub) made sqlit using textualize.io and is Terminal Tool of the Week! ⭐️

sqlit is a lazygit style SQL TUI for the terminal. It supports 10+ databases (Postgres, SQLite, Supabase, ClickHouse, DuckDB, etc), has query history, autocomplete and more. Peter Adams (Maxteabag on GitHub) made sqlit using textualize.io and is Terminal Tool of the Week! ⭐️

Terminal Trove

138,230 Aufrufe • vor 10 Monaten

Flamegraphs are a great way to narrow in on performance issues + broadly to explore program execution. I built a custom viewer for Postgres queries, inspired Jan Nidzwetzki's excellent blog on the subject. Highly recommend analysis tools like this for learning query execution.

Flamegraphs are a great way to narrow in on performance issues + broadly to explore program execution. I built a custom viewer for Postgres queries, inspired Jan Nidzwetzki's excellent blog on the subject. Highly recommend analysis tools like this for learning query execution.

Ben Dicken

47,895 Aufrufe • vor 7 Monaten

Just Use Postgres feat. Denis Magda Denis literally just published a book called “Just Use Postgres”. In this conversation, that turned out to be super fun, we talked about: • what MySQL got wrong w.r.t community • the recent explosion of Postgres development in the world (Supabase, Neon, Crunchy, ClickHouse) • when Postgres instead of Kafka • the significance of the meme “just use postgres” • Postgres for data analytics Chapters (00:00) Intro (04:26) The Just Use Postgres Movement (08:36) The Small Data Trend and Its Implications (15:49) The Shift from Distributed Systems to Simplicity (17:15) Updating your priors re: PG capabilities (20:28) MySQL's OSS community split & our experiences with such things (39:48) Postgres for Analytics (55:49) PG in the lake house (59:27) AI Marketing Slop (01:02:19) How you'd build a brand new app on Postgres (01:05:24) Speedrun through PG extensions (01:11:56) PG as a Message Queue

Stanislav Kozlovski

13,036 Aufrufe • vor 9 Monaten

How Postgres 17 improves IN queries In prior releases, postgres would loop and do multiple "primitive" index scans as many as your IN elements. In Postgres 17 it takes an array and eliminates the need for multiple scans. This only applies for constant values

How Postgres 17 improves IN queries In prior releases, postgres would loop and do multiple "primitive" index scans as many as your IN elements. In Postgres 17 it takes an array and eliminates the need for multiple scans. This only applies for constant values

Hussein Nasser

143,703 Aufrufe • vor 2 Jahren

Quick demo of Zero’s new background queries. Zero’s sync is query-based. Rather than specifying what data you want using rules or some other separate system, you just use queries. Right inside the client app, you do a query using a full sql-style language. You get filters, subqueries , limits, etc. Zero syncs the data backing these queries to the client. It’s important to realize Zero isn’t really a cache. It’s a replica. It’s eagerly replicating a precise snapshot of a slice of your database - the slice covered by the queries you have open. So there is never stale data in Zero. When you close a query we delete the rows uniquely returned by that query because we can no longer keep them up to date. Of course that kind of sucks for the common case of doing a query, navigating, then pressing “back”. Ideally we want that back nav to be fast. To address this Zero 0.17 adds background queries. You can add a ttl to a query and it will keep running and syncing in the background. This is much different than normal caching because this data stays up to date. If you make the same query again, the results will be *instantly* available *and already up to date with server*. If you make a different query the data from the background query is used client-side to answer the new query instantly if possible. This all happens completely automatically. Just by adding the ttl flag.

Quick demo of Zero’s new background queries. Zero’s sync is query-based. Rather than specifying what data you want using rules or some other separate system, you just use queries. Right inside the client app, you do a query using a full sql-style language. You get filters, subqueries , limits, etc. Zero syncs the data backing these queries to the client. It’s important to realize Zero isn’t really a cache. It’s a replica. It’s eagerly replicating a precise snapshot of a slice of your database - the slice covered by the queries you have open. So there is never stale data in Zero. When you close a query we delete the rows uniquely returned by that query because we can no longer keep them up to date. Of course that kind of sucks for the common case of doing a query, navigating, then pressing “back”. Ideally we want that back nav to be fast. To address this Zero 0.17 adds background queries. You can add a ttl to a query and it will keep running and syncing in the background. This is much different than normal caching because this data stays up to date. If you make the same query again, the results will be instantly available and already up to date with server. If you make a different query the data from the background query is used client-side to answer the new query instantly if possible. This all happens completely automatically. Just by adding the ttl flag.

Aaron Boodman

18,808 Aufrufe • vor 1 Jahr

Meet the new look of Neon Postgres. A total of ~1,200 hours of teamwork and five months from the initial kickoff to bring this design beauty to prod. Rebranding, design, dev, and motion. Ship faster with Postgres!

Meet the new look of Neon Postgres. A total of ~1,200 hours of teamwork and five months from the initial kickoff to bring this design beauty to prod. Rebranding, design, dev, and motion. Ship faster with Postgres!

Alex Barashkov

105,285 Aufrufe • vor 9 Monaten

Big moment for Postgres! AI agents broke the idea of what a database is supposed to do. Traditional databases were built for humans, and Agents broke that model. - They branch endlessly. - They run ten experiments at once. - They need isolation, context, memory, structured reasoning, and safe sandboxes. Letting agents touch production systems is terrifying because the old model of Postgres was never built for this kind of behavior. Agentic Postgres is an agent-ready version of Postgres by TimescaleDB (by Tiger Data) that solves this. I think it is one of the biggest upgrades to the Agent stack this year and Tiger Data is working with me on this post to share what they did. Some key features: > It instantly creates branches of an entire database, which is perfect for parallel agent evals, safe experiments, migrations, or isolated testing. Forks take seconds and cost almost nothing. > It comes with a built-in MCP server, which agents can use to get schema guidance, best practices, and safe, structured access to Postgres. This is also helpful to run migrations with a real understanding. > It comes with actual hybrid search (vector search and BM25), so Agents can retrieve data directly inside the database. > The database is Memory native. This gives a persistent context for Agents to evolve. This is one of the first times I have seen Postgres feel ready for the AI native era.

Big moment for Postgres! AI agents broke the idea of what a database is supposed to do. Traditional databases were built for humans, and Agents broke that model. - They branch endlessly. - They run ten experiments at once. - They need isolation, context, memory, structured reasoning, and safe sandboxes. Letting agents touch production systems is terrifying because the old model of Postgres was never built for this kind of behavior. Agentic Postgres is an agent-ready version of Postgres by TimescaleDB (by Tiger Data) that solves this. I think it is one of the biggest upgrades to the Agent stack this year and Tiger Data is working with me on this post to share what they did. Some key features: > It instantly creates branches of an entire database, which is perfect for parallel agent evals, safe experiments, migrations, or isolated testing. Forks take seconds and cost almost nothing. > It comes with a built-in MCP server, which agents can use to get schema guidance, best practices, and safe, structured access to Postgres. This is also helpful to run migrations with a real understanding. > It comes with actual hybrid search (vector search and BM25), so Agents can retrieve data directly inside the database. > The database is Memory native. This gives a persistent context for Agents to evolve. This is one of the first times I have seen Postgres feel ready for the AI native era.

Avi Chawla

94,406 Aufrufe • vor 11 Monaten

Use Neon and ClickHouse to get the best of both worlds: the flexibility of Postgres for transactions + the performance of ClickHouse for analytics With PeerDB, data streaming is simple 👇

Use Neon and ClickHouse to get the best of both worlds: the flexibility of Postgres for transactions + the performance of ClickHouse for analytics With PeerDB, data streaming is simple 👇

Neon

11,464 Aufrufe • vor 2 Jahren

Announcing the Supabase ETL! Supabase ETL is a change-data-capture pipeline that replicates your Postgres tables to analytical destinations in near real time! Available as private alpha starting today!

Announcing the Supabase ETL! Supabase ETL is a change-data-capture pipeline that replicates your Postgres tables to analytical destinations in near real time! Available as private alpha starting today!

Tyler Shukert

24,973 Aufrufe • vor 11 Monaten

How Cloudflare cut query times by 35x without leaving Postgres: Their Postgres tables hit billions of rows, and every time-range query started getting slower. Plain Postgres scans the entire table on every query, regardless of the time window. They tried the manual route by building precomputed aggregates with cron jobs and evaluating ClickHouse, which needed a full ingestion pipeline just to handle their write pattern. Here's where most teams get stuck at exactly this point. They utilize manual partitioning, splitting data into child tables by day or month. Then they wire up cron jobs to refresh aggregate tables. Every schema change after that requires updating the cron logic and coordinating across teams. The infrastructure becomes the project. Tiger Cloud is managed TimescaleDB by Tiger Data - Creators of TimescaleDB on the Postgres you already know, with automatic time-based partitioning, continuous aggregates, and compression built in from the start. Cloudflare moved to TimescaleDB after exhausting the manual route and saw 5-35x query performance improvement on the same data. Here is how it works: → Hypertables partition data by timestamp automatically. Every time-range query hits only the relevant chunk, not the full table. → Continuous aggregates refresh incrementally in the background, with no cron jobs to maintain. To show what this looks like in practice, I built a real-time earthquake intelligence dashboard on a 3D globe using Claude Code and Tiger Cloud in a single session. The USGS earthquake catalog has 400,000+ events since 1900, each a timestamped row. The dashboard plots them as ripple animations sized by magnitude and colored by depth, with a time slider that scrubs through 120 years of seismic history. Every slider position fires a live query against the hypertable, and the side panel pulls from continuous aggregates. Claude Code connected to Tiger Cloud through the Tiger CLI MCP server, provisioned the database, pulled the USGS catalog, and assembled the full Next.js and Three.js frontend without leaving the session. The video below shows the final build in action, and I worked with the Tiger Data team to put this together. Tiger CLI is open-source (Apache 2.0) and works with Claude Code, Cursor, Codex, Gemini CLI, and VS Code. To try this yourself: → Sign up for Tiger Cloud here: New accounts get $1,000 in free credits, no credit card required. → Install Tiger CLI: curl -fsSL https(:)//cli(.)tigerdata(.)com | sh → Run tiger mcp install claude-code → Give Claude Code a prompt and let it build. I also wrote a full walkthrough on how you can turn any coding agent into a production-grade data engineer that can manage over a billion-row Postgres workloads. It covers everything from the database setup to the final build. Read it below.

How Cloudflare cut query times by 35x without leaving Postgres: Their Postgres tables hit billions of rows, and every time-range query started getting slower. Plain Postgres scans the entire table on every query, regardless of the time window. They tried the manual route by building precomputed aggregates with cron jobs and evaluating ClickHouse, which needed a full ingestion pipeline just to handle their write pattern. Here's where most teams get stuck at exactly this point. They utilize manual partitioning, splitting data into child tables by day or month. Then they wire up cron jobs to refresh aggregate tables. Every schema change after that requires updating the cron logic and coordinating across teams. The infrastructure becomes the project. Tiger Cloud is managed TimescaleDB by Tiger Data - Creators of TimescaleDB on the Postgres you already know, with automatic time-based partitioning, continuous aggregates, and compression built in from the start. Cloudflare moved to TimescaleDB after exhausting the manual route and saw 5-35x query performance improvement on the same data. Here is how it works: → Hypertables partition data by timestamp automatically. Every time-range query hits only the relevant chunk, not the full table. → Continuous aggregates refresh incrementally in the background, with no cron jobs to maintain. To show what this looks like in practice, I built a real-time earthquake intelligence dashboard on a 3D globe using Claude Code and Tiger Cloud in a single session. The USGS earthquake catalog has 400,000+ events since 1900, each a timestamped row. The dashboard plots them as ripple animations sized by magnitude and colored by depth, with a time slider that scrubs through 120 years of seismic history. Every slider position fires a live query against the hypertable, and the side panel pulls from continuous aggregates. Claude Code connected to Tiger Cloud through the Tiger CLI MCP server, provisioned the database, pulled the USGS catalog, and assembled the full Next.js and Three.js frontend without leaving the session. The video below shows the final build in action, and I worked with the Tiger Data team to put this together. Tiger CLI is open-source (Apache 2.0) and works with Claude Code, Cursor, Codex, Gemini CLI, and VS Code. To try this yourself: → Sign up for Tiger Cloud here: New accounts get $1,000 in free credits, no credit card required. → Install Tiger CLI: curl -fsSL https(:)//cli(.)tigerdata(.)com | sh → Run tiger mcp install claude-code → Give Claude Code a prompt and let it build. I also wrote a full walkthrough on how you can turn any coding agent into a production-grade data engineer that can manage over a billion-row Postgres workloads. It covers everything from the database setup to the final build. Read it below.

Avi Chawla

22,094 Aufrufe • vor 2 Monaten

With PGlite + Electric sync you can build an issue tracker handling 100k issues + 500k comments, synced from Postgres to PGlite in the browser! Super fast UI updates with all writes going though the local database, and Postgres FTS in PGlite for search 🔥

With PGlite + Electric sync you can build an issue tracker handling 100k issues + 500k comments, synced from Postgres to PGlite in the browser! Super fast UI updates with all writes going though the local database, and Postgres FTS in PGlite for search 🔥

Sam Willis

55,412 Aufrufe • vor 2 Jahren

Finish the week with a Postgres video 🌟 All about using JSON in Postgres: - when you should use it - how to use it efficiently - how JSONs actually get stored in postgres.

Finish the week with a Postgres video 🌟 All about using JSON in Postgres: - when you should use it - how to use it efficiently - how JSONs actually get stored in postgres.

Gwen (Chen) Shapira

27,636 Aufrufe • vor 1 Jahr

Another insane Jev use case! Traditional database filters need precise, predefined conditions. But many questions are semantic: - Is this article mainly about software engineering? - Which topic best describes it? - How technically deep does it appear? These usually require moving rows into application code, invoking a model, parsing its output, and writing the result back. pg-jev is an open-source Postgres extension that exposes Jev through SQL functions. - jev() works inside WHERE - jev_prob() returns a probability - jev_choice() selects a label - jev_score() ranks rows across ordered levels. It needs no vector column or embedding index. Under the hood, pg-jev batches rows, sends them to Jev, and returns typed answers that SQL can filter, sort, group, and combine with exact predicates. The recording below runs it against real Hacker News stories stored in Postgres. It first shows the rows, then asks Jev to find software-engineering stories, classify them by topic, and rank them by technical depth. The final query reports requests, tokens, cost, and cache hits from the database session. GitHub repo: (don't forget to star it ⭐) Postgres keeps the data and controls the query, while Jev handles the part SQL cannot express as a deterministic condition. If you want to understand what Jev is doing under the hood, I also wrote a hands-on guide to building a Jev-style model with open models, entirely locally. Read it below.

Another insane Jev use case! Traditional database filters need precise, predefined conditions. But many questions are semantic: - Is this article mainly about software engineering? - Which topic best describes it? - How technically deep does it appear? These usually require moving rows into application code, invoking a model, parsing its output, and writing the result back. pg-jev is an open-source Postgres extension that exposes Jev through SQL functions. - jev() works inside WHERE - jev_prob() returns a probability - jev_choice() selects a label - jev_score() ranks rows across ordered levels. It needs no vector column or embedding index. Under the hood, pg-jev batches rows, sends them to Jev, and returns typed answers that SQL can filter, sort, group, and combine with exact predicates. The recording below runs it against real Hacker News stories stored in Postgres. It first shows the rows, then asks Jev to find software-engineering stories, classify them by topic, and rank them by technical depth. The final query reports requests, tokens, cost, and cache hits from the database session. GitHub repo: (don't forget to star it ⭐) Postgres keeps the data and controls the query, while Jev handles the part SQL cannot express as a deterministic condition. If you want to understand what Jev is doing under the hood, I also wrote a hands-on guide to building a Jev-style model with open models, entirely locally. Read it below.

Avi Chawla

299,936 Aufrufe • vor 29 Tagen

Your agents can't keep up with real-time data. Especially when it's scattered across dozens of sources. Most teams waste weeks building custom connectors for every database, API, and data warehouse. Then they build ETL pipelines to sync everything. By the time your agent retrieves the data, it's already outdated. Picture this: Your Postgres database updated 5 minutes ago. Your MongoDB collection changed 2 minutes ago. Your agent is still pulling from yesterday's snapshot. This is why most production RAG systems fail. There's a better approach: MindsDB is an open-source AI platform with a federated data engine that lets you query multiple data sources in real-time using SQL - without moving any data. Here's what makes it different: ↳ Your data stays in place. No ETL pipelines or data duplication ↳ Query Postgres, MongoDB, REST APIs, and more using consistent SQL ↳ JOIN across different sources in real-time with a unified interface ↳ Works with both structured and un-structured data And here's the best part: You don't even need to write SQL. Just describe what you want in plain English, and MindsDB converts it to SQL automatically. The system does all the heavy lifting. The breakthrough for AI agents is simple: When data updates at the source, your agent gets fresh results immediately. No sync delays. No stale embeddings. No custom code for each integration. You can literally write a SQL query that joins a Postgres table with a MongoDB collection and gets live results. This is what production AI applications need but rarely get. In this video, I give you a complete walkthrough of what we just discussed and how to actually do it. Make sure you watch this till the end. I've shared the link to MindsDB's GitHub repo in the next tweet!

Your agents can't keep up with real-time data. Especially when it's scattered across dozens of sources. Most teams waste weeks building custom connectors for every database, API, and data warehouse. Then they build ETL pipelines to sync everything. By the time your agent retrieves the data, it's already outdated. Picture this: Your Postgres database updated 5 minutes ago. Your MongoDB collection changed 2 minutes ago. Your agent is still pulling from yesterday's snapshot. This is why most production RAG systems fail. There's a better approach: MindsDB is an open-source AI platform with a federated data engine that lets you query multiple data sources in real-time using SQL - without moving any data. Here's what makes it different: ↳ Your data stays in place. No ETL pipelines or data duplication ↳ Query Postgres, MongoDB, REST APIs, and more using consistent SQL ↳ JOIN across different sources in real-time with a unified interface ↳ Works with both structured and un-structured data And here's the best part: You don't even need to write SQL. Just describe what you want in plain English, and MindsDB converts it to SQL automatically. The system does all the heavy lifting. The breakthrough for AI agents is simple: When data updates at the source, your agent gets fresh results immediately. No sync delays. No stale embeddings. No custom code for each integration. You can literally write a SQL query that joins a Postgres table with a MongoDB collection and gets live results. This is what production AI applications need but rarely get. In this video, I give you a complete walkthrough of what we just discussed and how to actually do it. Make sure you watch this till the end. I've shared the link to MindsDB's GitHub repo in the next tweet!

Akshay 🚀

65,672 Aufrufe • vor 11 Monaten

`bunx pg-here` global one-liner to run a standalone postgres instance with all data within the current folder throw-away postgres anytime anywhere. cli args for version/user/pass/db configs. docs: stop using docker just to launch a postgres!!

`bunx pg-here` global one-liner to run a standalone postgres instance with all data within the current folder throw-away postgres anytime anywhere. cli args for version/user/pass/db configs. docs: stop using docker just to launch a postgres!!

murat 🍥

50,537 Aufrufe • vor 8 Monaten

Introducing pg_strict for Postgres. Our new extension adds a safety net to Postgres, catching dangerous queries before they run.

Introducing pg_strict for Postgres. Our new extension adds a safety net to Postgres, catching dangerous queries before they run.

PlanetScale

114,704 Aufrufe • vor 9 Monaten