Загрузка видео...

Не удалось загрузить видео

На главную

What Postgres can’t do alone, it can tackle with DuckDB. The pg_duckdb extension shows how two databases can work hand in hand by querying analytical data with DuckDB and joining it with transactional data in Postgres. The extension embeds DuckDB directly into Postgres. DuckDB literally runs inside a Postgres...

denismagda's profile picture

Denis Magda

5,025 subscribers

28,678 просмотров • 5 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

This is how long it takes to load data from your Prisma Postgres Postgres DB using Accelerate Stay tuned, video drops tomorrow!
0:14

This is how long it takes to load data from your Prisma Postgres Postgres DB using Accelerate Stay tuned, video drops tomorrow!

Beto

11,781 просмотров • 1 год назад

Big moment for Postgres! Search has always been Postgres' weak spot, and everyone just accepted it. If you needed a real relevance-ranked keyword search, the default answer was to spin up Elasticsearch or add Algolia and deal with the data sync headaches forever. The problem isn't that Postgres can't do text search. It can. But the built-in `ts_rank` function uses a basic term frequency algorithm that doesn't come close to what modern search engines deliver. So teams end up: - Running a separate Elasticsearch cluster just for search - Building sync pipelines that inevitably drift out of consistency - Paying for managed search services that charge per query - Accepting mediocre search relevance because "good enough" ships faster But this is actually a solvable problem. You can realistically bring industry-standard search ranking directly into Postgres, which eliminates the need for external infra entirely. This exact solution is now available with the newly open-sourced pg_textsearch by Tiger Data - Creators of TimescaleDB, a Postgres extension that brings true BM25 relevance ranking into the database. BM25 is the algorithm behind Elasticsearch, Lucene, and most modern search engines. Now it runs natively in Postgres. Here's what pg_textsearch enables: - True BM25 ranking with configurable parameters (the same algorithm powering production search systems) - Simple SQL syntax: `ORDER BY content 'search terms'` - Works with Postgres text search configurations for multiple languages - Pairs naturally with pgvector for hybrid keyword + semantic search That last point matters a lot for RAG apps. The video below shows this in action, and I worked with the team to put this together. You can now do hybrid retrieval (combining keyword matching with vector similarity) in a single database, without stitching together multiple systems. The syntax is clean enough that you can add relevance-ranked search to existing queries in minutes. pg_textsearch is fully open-source under the PostgreSQL license. You can find a link to their GitHub repo in the next tweet.
0:10

Big moment for Postgres! Search has always been Postgres' weak spot, and everyone just accepted it. If you needed a real relevance-ranked keyword search, the default answer was to spin up Elasticsearch or add Algolia and deal with the data sync headaches forever. The problem isn't that Postgres can't do text search. It can. But the built-in `ts_rank` function uses a basic term frequency algorithm that doesn't come close to what modern search engines deliver. So teams end up: - Running a separate Elasticsearch cluster just for search - Building sync pipelines that inevitably drift out of consistency - Paying for managed search services that charge per query - Accepting mediocre search relevance because "good enough" ships faster But this is actually a solvable problem. You can realistically bring industry-standard search ranking directly into Postgres, which eliminates the need for external infra entirely. This exact solution is now available with the newly open-sourced pg_textsearch by Tiger Data - Creators of TimescaleDB, a Postgres extension that brings true BM25 relevance ranking into the database. BM25 is the algorithm behind Elasticsearch, Lucene, and most modern search engines. Now it runs natively in Postgres. Here's what pg_textsearch enables: - True BM25 ranking with configurable parameters (the same algorithm powering production search systems) - Simple SQL syntax: `ORDER BY content 'search terms'` - Works with Postgres text search configurations for multiple languages - Pairs naturally with pgvector for hybrid keyword + semantic search That last point matters a lot for RAG apps. The video below shows this in action, and I worked with the team to put this together. You can now do hybrid retrieval (combining keyword matching with vector similarity) in a single database, without stitching together multiple systems. The syntax is clean enough that you can add relevance-ranked search to existing queries in minutes. pg_textsearch is fully open-source under the PostgreSQL license. You can find a link to their GitHub repo in the next tweet.

Akshay 🚀

215,227 просмотров • 5 месяцев назад

First try with Vercel's new Postgres database! Was so quick to set up 🤯 here I've got a collaborative form built with Liveblocks, and I'm having it automatically sync data to the database. More info 🧵
0:17

First try with Vercel's new Postgres database! Was so quick to set up 🤯 here I've got a collaborative form built with Liveblocks, and I'm having it automatically sync data to the database. More info 🧵

Chris Nicholas

114,238 просмотров • 3 лет назад

Go from single-node Postgres to highly-available Postgres in one click 🐘 NOT production ready. Every time you use it in production, the Product Engineer who built it starts sweating Try it out and give us feedback
0:19

Go from single-node Postgres to highly-available Postgres in one click 🐘 NOT production ready. Every time you use it in production, the Product Engineer who built it starts sweating Try it out and give us feedback

Mahmoud

24,778 просмотров • 3 месяцев назад

you can now get data from auth'd APIs with DuckDB 🤯 ya know, to get your stripe data or whatever (it's not documented yet)
0:12

you can now get data from auth'd APIs with DuckDB 🤯 ya know, to get your stripe data or whatever (it's not documented yet)

archie 🦋

69,170 просмотров • 1 год назад

Operational databases have long relied on tightly coupled compute and storage. This architecture creates resource contention and pushes teams to manage infrastructure rather than build. As applications become more real time and automated, the transactional layer needs to adapt. Databricks Lakebase is built for that evolution: • Familiar Postgres semantics for app developers • Compute separated from durable state • Operational data running directly on the lakehouse • Serverless autoscaling (including scale to zero), branching, and recovery to match agent-driven workload Now generally available:
0:16

Operational databases have long relied on tightly coupled compute and storage. This architecture creates resource contention and pushes teams to manage infrastructure rather than build. As applications become more real time and automated, the transactional layer needs to adapt. Databricks Lakebase is built for that evolution: • Familiar Postgres semantics for app developers • Compute separated from durable state • Operational data running directly on the lakehouse • Serverless autoscaling (including scale to zero), branching, and recovery to match agent-driven workload Now generally available:

Databricks

18,611 просмотров • 4 месяцев назад

Analyzing large vector datasets on Source Cooperative with #DuckDB 🦆Using the 75 GB National Wetlands Inventory as an example. It used to take me hours to get summary statistics of the dataset, now it takes seconds with DuckDB 🚀 Dataset: Colab: #geospatial #dataviz #opendata
0:14

Analyzing large vector datasets on Source Cooperative with #DuckDB 🦆Using the 75 GB National Wetlands Inventory as an example. It used to take me hours to get summary statistics of the dataset, now it takes seconds with DuckDB 🚀 Dataset: Colab: #geospatial #dataviz #opendata

Qiusheng Wu

13,309 просмотров • 2 лет назад

Craft. Not AI. Hero section design behind the scenes Neon Postgres.
0:14

Craft. Not AI. Hero section design behind the scenes Neon Postgres.

Yarik

59,401 просмотров • 5 месяцев назад

the reality when Supabase disappoints with their launch on Monday shipping just another boring postgres plugin #SupaAss
0:15

the reality when Supabase disappoints with their launch on Monday shipping just another boring postgres plugin #SupaAss

Peer Richelsen

32,705 просмотров • 2 лет назад

Postgres 17 is now available on Neon!
0:17

Postgres 17 is now available on Neon!

Neon Postgres

72,306 просмотров • 1 год назад

Real, actual Postgres right from your browser! Learning a lot. This is fun
0:12

Real, actual Postgres right from your browser! Learning a lot. This is fun

Aaron Francis

21,237 просмотров • 6 месяцев назад

Talking with someone the other day that estimated they had about 25,000 idle connections to Postgres. My actual response to them: "holy shit". They double checked, it was only about 12,000 Same day had a conversation with someone saying they didn't need pgbouncer because of activerecord's connection pooling. Let's dig into connection pooling in Postgres. Prior to Postgres 14 every connection to the database consumed memory, roughly 10MB, it may be slightly less but it still wasn't free. Even beyond Postgres 14 there is still various contention that happens when Postgres starts to use a connection. An application pooler maintains a set of connections and hands them out when needed on the application side. These are idle and real connections against the database that indeed do impact performance negatively. In contrast pgbouncer speaks the wire protocol, waits for the begin part of the transaction and then uses a connection. It more strictly manages how many idle ones it's having instead of per web server you're running. pgbouncer up until recently really needed to be run in transaction mode (which meant disabling prepared statements in your application framework). prepared_statement support in pgbouncer was added recently, and now you don't have to disable. Even when running with an older version of pgbouncer with prepared_statements disabled you'd still see a big performance gain. A quick check to know if you'd benefit from pgbouncer, run this query: SELECT count(*), state FROM pg_stat_activity GROUP BY 2; If you're idle account is high (yes this is dependent on your view, but to me if it's above 25-30 range, and especially if active is until half that) then you'd already start to benefit from pgbouncer. If it's at 10,000 then post haste get pgbouncer in place. Finally, you don't have to not use a framework pooler, they're fine, but don't think it replaces a native Postgres connection pooler.
0:17

Talking with someone the other day that estimated they had about 25,000 idle connections to Postgres. My actual response to them: "holy shit". They double checked, it was only about 12,000 Same day had a conversation with someone saying they didn't need pgbouncer because of activerecord's connection pooling. Let's dig into connection pooling in Postgres. Prior to Postgres 14 every connection to the database consumed memory, roughly 10MB, it may be slightly less but it still wasn't free. Even beyond Postgres 14 there is still various contention that happens when Postgres starts to use a connection. An application pooler maintains a set of connections and hands them out when needed on the application side. These are idle and real connections against the database that indeed do impact performance negatively. In contrast pgbouncer speaks the wire protocol, waits for the begin part of the transaction and then uses a connection. It more strictly manages how many idle ones it's having instead of per web server you're running. pgbouncer up until recently really needed to be run in transaction mode (which meant disabling prepared statements in your application framework). prepared_statement support in pgbouncer was added recently, and now you don't have to disable. Even when running with an older version of pgbouncer with prepared_statements disabled you'd still see a big performance gain. A quick check to know if you'd benefit from pgbouncer, run this query: SELECT count(*), state FROM pg_stat_activity GROUP BY 2; If you're idle account is high (yes this is dependent on your view, but to me if it's above 25-30 range, and especially if active is until half that) then you'd already start to benefit from pgbouncer. If it's at 10,000 then post haste get pgbouncer in place. Finally, you don't have to not use a framework pooler, they're fine, but don't think it replaces a native Postgres connection pooler.

Craig Kerstiens

12,357 просмотров • 1 год назад

Starting a new project today, building an end to end system to forecast traffic of flights across cities (starting with Mumbai) The idea is to implement > ingestion service with kafka > data etl with polars > feast for feature store // mlflow for model registry > batch inferencing // dashboard > s3 // postgres for data storage All this orchestrated across multiple DAGs built with Airflow. This year has just been Agents and LLMs all along. Not a bad idea to keep revisiting the traditional format :) Will be posting more of this in the coming days, stay tuned
0:14

Starting a new project today, building an end to end system to forecast traffic of flights across cities (starting with Mumbai) The idea is to implement > ingestion service with kafka > data etl with polars > feast for feature store // mlflow for model registry > batch inferencing // dashboard > s3 // postgres for data storage All this orchestrated across multiple DAGs built with Airflow. This year has just been Agents and LLMs all along. Not a bad idea to keep revisiting the traditional format :) Will be posting more of this in the coming days, stay tuned

Aarno

19,466 просмотров • 6 месяцев назад

I’m happy to share that I’m starting a new position as Developer Advocate at Neon - Serverless Postgres!
0:19

I’m happy to share that I’m starting a new position as Developer Advocate at Neon - Serverless Postgres!

Ryan Vogel

11,723 просмотров • 1 год назад

oh shut up Aaron Francis when you change the website to dark mode it changes the video to dark mode as well what is this dark magic in Mastering Postgres did you build this course platform??
0:16

oh shut up Aaron Francis when you change the website to dark mode it changes the video to dark mode as well what is this dark magic in Mastering Postgres did you build this course platform??

Jon Kurtis ⚡

240,221 просмотров • 1 год назад

Supabase now has Agent Skills! Install it on your favorite AI agent/ editor to have it follow the best practices for handling Postgres! $ npx skills add supabase/agent-skills
0:17

Supabase now has Agent Skills! Install it on your favorite AI agent/ editor to have it follow the best practices for handling Postgres! $ npx skills add supabase/agent-skills

Tyler Shukert

108,870 просмотров • 5 месяцев назад

OpenAI has deployed interactive charts and tables in ChatGPT. You can now interact with your data and process it with GPT-4o. Here's how it works and what you can do with it:
0:12

OpenAI has deployed interactive charts and tables in ChatGPT. You can now interact with your data and process it with GPT-4o. Here's how it works and what you can do with it:

Paul Couvert

513,332 просмотров • 2 лет назад

I've been invited by the wonderful TANSTACK team to help them build out framework-agnostic devtools not for only them, but for everyone! Here's a sneek peak of TanStack router, query, and Prisma Postgres embedded studio running in @tanstack/devtools
0:19

I've been invited by the wonderful TANSTACK team to help them build out framework-agnostic devtools not for only them, but for everyone! Here's a sneek peak of TanStack router, query, and Prisma Postgres embedded studio running in @tanstack/devtools

Alem Tuzlak 🇧🇦

48,634 просмотров • 11 месяцев назад