正在加载视频...

视频加载失败

返回首页

Here’s how I would learn data engineering in 2025: 1. The basics: - learn SQL — SELECT, FROM, WHERE, GROUP BY, JOIN, HAVING, etc - learn Python — data structures: objects, arrays, tuples, namedtuples — algorithms: recursion, loops 2. Intermediate - learn distributed compute — pick up PySpark or...

EcZachly's profile picture

Zach Wilson

50,747 subscribers

29,164 次观看 • 11 个月前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

9 条评论

vingança da rainha anne 的头像
vingança da rainha anne11 个月前

Roadmap

Ayobami.Ola 的头像
Ayobami.Ola11 个月前

Roadmap

W0lfgeng 的头像
W0lfgeng11 个月前

Roadmap

Martin Shein 的头像
Martin Shein11 个月前

Roadmap

asa of tech 的头像
asa of tech11 个月前

Roadmap

4Eyed Ìfẹ́luyì Asojú 的头像
4Eyed Ìfẹ́luyì Asojú11 个月前

Roadmap

esp 的头像
esp11 个月前

Roadmap

Azningnam 的头像
Azningnam11 个月前

Roadmap

Kshiteej Pitta 的头像
Kshiteej Pitta11 个月前

Roadmap

相关视频

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?
0:51

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?

Zach Wilson

20,363 次观看 • 11 个月前

Is your JSON data getting hung up with trailing commas or incorrect data types? Just ask GitHub Copilot Chat what’s wrong and how to fix it 🛠️ Learn more in the Copilot Chat Cookbook.
0:34

Is your JSON data getting hung up with trailing commas or incorrect data types? Just ask GitHub Copilot Chat what’s wrong and how to fix it 🛠️ Learn more in the Copilot Chat Cookbook.

GitHub

29,212 次观看 • 1 年前

Effective Table Data Extraction from PDF without LLM Sparrow Parse helps to read tabular data from PDFs, relying on various libraries, such as Unstructured or PyMuPDF4LLM. This allows us to avoid data hallucination errors often produced by LLMs when processing complex data structures. Learn more: ✅ ✅ Katana
2:26

Effective Table Data Extraction from PDF without LLM Sparrow Parse helps to read tabular data from PDFs, relying on various libraries, such as Unstructured or PyMuPDF4LLM. This allows us to avoid data hallucination errors often produced by LLMs when processing complex data structures. Learn more: ✅ ✅ Katana

Andrej Baranovskij

27,886 次观看 • 2 年前

Sharing our latest short course: Building and Evaluating Data Agents, created in collaboration with Snowflake and taught by Anupam Datta (Anupam Datta) and Josh Reini (Josh Reini). A data agent extracts data from sources such as files or databases, analyzes it, and provides insights and visualizes its findings. But most data agents struggle with reliability or can't handle multi-step reasoning. In this course, you'll learn to build, trace, and evaluate a multi-agent workflow that plans tasks, pulls context from structured and unstructured data, performs web search, and summarizes or visualizes the final results. Learn more and enroll for free!
3:20

Sharing our latest short course: Building and Evaluating Data Agents, created in collaboration with Snowflake and taught by Anupam Datta (Anupam Datta) and Josh Reini (Josh Reini). A data agent extracts data from sources such as files or databases, analyzes it, and provides insights and visualizes its findings. But most data agents struggle with reliability or can't handle multi-step reasoning. In this course, you'll learn to build, trace, and evaluate a multi-agent workflow that plans tasks, pulls context from structured and unstructured data, performs web search, and summarizes or visualizes the final results. Learn more and enroll for free!

DeepLearning.AI

40,745 次观看 • 8 个月前

We just launched a major new Data Engineering Professional Certificate on Coursera! Data underlies all modern AI systems, and engineers who know how to build systems to store and serve it are in high demand. If you're interested in learning this skill, please check out this 4-course sequence, which is designed to make you job-ready to be a Data Engineer. This is a new specialization taught by Joe Reis, the co-author of the best-selling book “Fundamentals of Data Engineering," in collaboration with AWS. (Disclosure, I serve on Amazon's board.) For many AI systems, data engineering is 80% of the work, and modeling is 20%. But people’s attention on these two topics is often flipped. This makes the job of the data engineer particularly important. In this professional certificate, you'll learn foundational data engineering skills while implementing modern data architectures using open-source tools: - Learn the key steps of the data lifecycle, to generate, ingest, store, transform, and serve data. - Learn to align with organizational goals to design the data pipeline right for your business' needs. - Understand how to make necessary trade-offs between speed, scalability, security, and cost. Joe has distilled into this specialization decades of experience helping startups and large companies with data infrastructure. He is also joined by 17 other industry leaders in the data field, who will help you learn in-demand skills for the growing field of data engineering. Please sign up here:
1:33

We just launched a major new Data Engineering Professional Certificate on Coursera! Data underlies all modern AI systems, and engineers who know how to build systems to store and serve it are in high demand. If you're interested in learning this skill, please check out this 4-course sequence, which is designed to make you job-ready to be a Data Engineer. This is a new specialization taught by Joe Reis, the co-author of the best-selling book “Fundamentals of Data Engineering," in collaboration with AWS. (Disclosure, I serve on Amazon's board.) For many AI systems, data engineering is 80% of the work, and modeling is 20%. But people’s attention on these two topics is often flipped. This makes the job of the data engineer particularly important. In this professional certificate, you'll learn foundational data engineering skills while implementing modern data architectures using open-source tools: - Learn the key steps of the data lifecycle, to generate, ingest, store, transform, and serve data. - Learn to align with organizational goals to design the data pipeline right for your business' needs. - Understand how to make necessary trade-offs between speed, scalability, security, and cost. Joe has distilled into this specialization decades of experience helping startups and large companies with data infrastructure. He is also joined by 17 other industry leaders in the data field, who will help you learn in-demand skills for the growing field of data engineering. Please sign up here:

Andrew Ng

118,937 次观看 • 1 年前

Pick up the phone, you could learn something new
0:35

Pick up the phone, you could learn something new

Niamocha🐻✨

67,907 次观看 • 1 年前

Shut the fuck up, watch, learn and maybe you might pick up a thing or two. 145kilos x 4 at 87kilos bw.
0:27

Shut the fuck up, watch, learn and maybe you might pick up a thing or two. 145kilos x 4 at 87kilos bw.

blaakbaki

351,749 次观看 • 2 个月前

Announcing The Open Data Market by Stork Where onchain innovation is no longer suppressed by access to data. p.s. learn how to participate below 👀
0:31

Announcing The Open Data Market by Stork Where onchain innovation is no longer suppressed by access to data. p.s. learn how to participate below 👀

Stork

45,039 次观看 • 1 年前

Someone made the most addictive game to learn how real data centers work.
0:26

Someone made the most addictive game to learn how real data centers work.

mitsuri

20,294 次观看 • 2 个月前

Starting today, the FBI will publish reported crime data to the Crime Data Explorer every month. See the newest data and learn how this timelier data can help law enforcement crush violent crime:
0:48

Starting today, the FBI will publish reported crime data to the Crime Data Explorer every month. See the newest data and learn how this timelier data can help law enforcement crush violent crime:

FBI

22,485 次观看 • 9 个月前

Learn to train an LLM with distributed data while ensuring privacy using federated learning in a new two-part short course, Intro to Federated Learning and Federated Fine-tuning of LLMs with Private Data, created with Flower and taught by Daniel J. Beutel and nic lane. Federated learning allows a single model to be trained across multiple devices, such as phones, or multiple organizations, such as hospitals, without the need to share data to a central server. This two-part course gives you an introduction to federated learning, and then teaches you how to fine-tune your large language model with distributed data using Flower Lab’s open source federated learning framework. You’ll learn: - How to use federated learning to train a variety of models, ranging from speech and vision models to LLMs, across distributed data while offering data privacy options to users and organizations. - Privacy Enhancing Technologies like differential privacy (DP), which obscures individual data by adding calibrated noise to query results. - Two variants of differential privacy - Central and Local - and how to choose depending on your use case. - How to measure and decrease bandwidth usage to make federated learning more practical and efficient with techniques like using pre-trained models and Parameter-Efficient Fine-Tuning - How federated LLM fine-tuning reduces the risk of leaking training data. Sign up here!
4:34

Learn to train an LLM with distributed data while ensuring privacy using federated learning in a new two-part short course, Intro to Federated Learning and Federated Fine-tuning of LLMs with Private Data, created with Flower and taught by Daniel J. Beutel and nic lane. Federated learning allows a single model to be trained across multiple devices, such as phones, or multiple organizations, such as hospitals, without the need to share data to a central server. This two-part course gives you an introduction to federated learning, and then teaches you how to fine-tune your large language model with distributed data using Flower Lab’s open source federated learning framework. You’ll learn: - How to use federated learning to train a variety of models, ranging from speech and vision models to LLMs, across distributed data while offering data privacy options to users and organizations. - Privacy Enhancing Technologies like differential privacy (DP), which obscures individual data by adding calibrated noise to query results. - Two variants of differential privacy - Central and Local - and how to choose depending on your use case. - How to measure and decrease bandwidth usage to make federated learning more practical and efficient with techniques like using pre-trained models and Parameter-Efficient Fine-Tuning - How federated LLM fine-tuning reduces the risk of leaking training data. Sign up here!

Andrew Ng

64,538 次观看 • 1 年前

🚀 Framer Forms are here! Enjoy 10+ input types, custom states, and secure data handling. Send data via email, Google Sheets, or custom Webhook. Learn more below.
0:47

🚀 Framer Forms are here! Enjoy 10+ input types, custom states, and secure data handling. Send data via email, Google Sheets, or custom Webhook. Learn more below.

Framer

100,853 次观看 • 1 年前

Learn why oracles need low latency data 🔮
0:27

Learn why oracles need low latency data 🔮

Pyth Network 🔮

24,981 次观看 • 2 年前

Announcing the expansion of Microsoft Sentinel with a modern data lake and dynamic threat intel capabilities. Learn how it: 💰Optimizes costs 📈Simplifies data management ⚡Accelerates AI adoption
0:45

Announcing the expansion of Microsoft Sentinel with a modern data lake and dynamic threat intel capabilities. Learn how it: 💰Optimizes costs 📈Simplifies data management ⚡Accelerates AI adoption

Microsoft Security

28,901 次观看 • 10 个月前

Dear Data Analyst, learn how to display percentages on a column chart in Power BI 🙉 DAX formula used: %MaritalStatus = DIVIDE( COUNTROWS('customer data'), CALCULATE(COUNTROWS('customer data'), ALL('customer data'[Marital_Status])))
0:50

Dear Data Analyst, learn how to display percentages on a column chart in Power BI 🙉 DAX formula used: %MaritalStatus = DIVIDE( COUNTROWS('customer data'), CALCULATE(COUNTROWS('customer data'), ALL('customer data'[Marital_Status])))

Zehida

22,557 次观看 • 1 年前

DATA AVAILABILITY SAMPLING EXPLAINED! learn about -Modular Blockchains -probabilistic sampling techniques -Erasure Encoding Full vid below👇
0:44

DATA AVAILABILITY SAMPLING EXPLAINED! learn about -Modular Blockchains -probabilistic sampling techniques -Erasure Encoding Full vid below👇

gogoDiego

93,106 次观看 • 2 年前

AI Czar David Sacks says American companies will learn efficiency techniques from China's DeepSeek AI model, but big AI data centers are still needed and scaling the biggest data centers is still an advantage
0:41

AI Czar David Sacks says American companies will learn efficiency techniques from China's DeepSeek AI model, but big AI data centers are still needed and scaling the biggest data centers is still an advantage

Tsarathustra

360,750 次观看 • 1 年前

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?
1:59

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?

Zach Wilson

16,688 次观看 • 1 年前

How do computers understand data? With semantic search! Instead of just matching keywords, it understands context using vector embeddings. Here’s how: 1) Convert data (text, images, etc.) into vectors (embeddings) 2) Store these vectors in a vector database 3) Search by meaning, not just the keywords Semantic search makes finding data across formats easier. Learn more in this blog post by Leonie, my all-time favorite:
1:05

How do computers understand data? With semantic search! Instead of just matching keywords, it understands context using vector embeddings. Here’s how: 1) Convert data (text, images, etc.) into vectors (embeddings) 2) Store these vectors in a vector database 3) Search by meaning, not just the keywords Semantic search makes finding data across formats easier. Learn more in this blog post by Leonie, my all-time favorite:

Femke Plantinga

23,911 次观看 • 1 年前