Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Does your team rely on Snowflake or BigQuery as your data warehouse? We built the step that every other onchain data provider skips. Transform first, deliver only what you need. dbt Connector → Datashare is now live. Query against 130+ chains ➡️ Transform with dbt on DuneSQL ➡️ Deliver...

31,422 görüntüleme • 7 ay önce •via X (Twitter)

8 Yorum

Phil profil fotoğrafı
Phil7 ay önce

@Snowflake Databricks when?

0xfarmed profil fotoğrafı
0xfarmed7 ay önce

@Snowflake very cool !

Ember 🇰🇷 profil fotoğrafı
Ember 🇰🇷7 ay önce

@Snowflake Discover endless possibilities with dbt Connector Datashare feature.

Dune profil fotoğrafı
Dune7 ay önce

Read more on dbt Connector → Datashare

𝚲𝖓𝛂𝖑უي𝖙 profil fotoğrafı
𝚲𝖓𝛂𝖑უي𝖙7 ay önce

@Snowflake Are you guys hiring still?

WW3 SOON profil fotoğrafı
WW3 SOON7 ay önce

@Snowflake bigquery se daata warehouse karte hain hum tb connector aur datashare ke madhyam se 130+ chain par data analytics kar sakte hain

agentXchain.dev profil fotoğrafı
agentXchain.dev7 ay önce

@Snowflake Nice to see a transform-first approach powering cross-chain queries on DuneSQL 🚀🧭 How do teams decide when to lean into this vs their current data warehouse setup, and what early wins are you hearing from users?

JennyLiu profil fotoğrafı
JennyLiu6 ay önce

@Snowflake wait so this means we can skip the manual extract+load circus entirely? that's kinda wild if it holds up

Benzer Videolar

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?

Zach Wilson

16,851 görüntüleme • 1 yıl önce