Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Here’s how I would learn data engineering in 2025: 1. The basics: - learn SQL — SELECT, FROM, WHERE, GROUP BY, JOIN, HAVING, etc - learn Python — data structures: objects, arrays, tuples, namedtuples — algorithms: recursion, loops 2. Intermediate - learn distributed compute — pick up PySpark or...

EcZachly's profile picture

Zach Wilson

50,747 subscribers

29,164 Aufrufe • vor 11 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

9 Kommentare

Profilbild von vingança da rainha anne
vingança da rainha annevor 11 Monaten

Roadmap

Profilbild von Ayobami.Ola
Ayobami.Olavor 11 Monaten

Roadmap

Profilbild von W0lfgeng
W0lfgengvor 11 Monaten

Roadmap

Profilbild von Martin Shein
Martin Sheinvor 11 Monaten

Roadmap

Profilbild von asa of tech
asa of techvor 11 Monaten

Roadmap

Profilbild von 4Eyed Ìfẹ́luyì Asojú
4Eyed Ìfẹ́luyì Asojúvor 11 Monaten

Roadmap

Profilbild von esp
espvor 11 Monaten

Roadmap

Profilbild von Azningnam
Azningnamvor 11 Monaten

Roadmap

Profilbild von Kshiteej Pitta
Kshiteej Pittavor 11 Monaten

Roadmap

Ähnliche Videos

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?
0:51

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?

Zach Wilson

20,363 Aufrufe • vor 11 Monaten

Effective Table Data Extraction from PDF without LLM Sparrow Parse helps to read tabular data from PDFs, relying on various libraries, such as Unstructured or PyMuPDF4LLM. This allows us to avoid data hallucination errors often produced by LLMs when processing complex data structures. Learn more: ✅ ✅ Katana
2:26

Effective Table Data Extraction from PDF without LLM Sparrow Parse helps to read tabular data from PDFs, relying on various libraries, such as Unstructured or PyMuPDF4LLM. This allows us to avoid data hallucination errors often produced by LLMs when processing complex data structures. Learn more: ✅ ✅ Katana

Andrej Baranovskij

27,886 Aufrufe • vor 2 Jahren

Sharing our latest short course: Building and Evaluating Data Agents, created in collaboration with Snowflake and taught by Anupam Datta (Anupam Datta) and Josh Reini (Josh Reini). A data agent extracts data from sources such as files or databases, analyzes it, and provides insights and visualizes its findings. But most data agents struggle with reliability or can't handle multi-step reasoning. In this course, you'll learn to build, trace, and evaluate a multi-agent workflow that plans tasks, pulls context from structured and unstructured data, performs web search, and summarizes or visualizes the final results. Learn more and enroll for free!
3:20

Sharing our latest short course: Building and Evaluating Data Agents, created in collaboration with Snowflake and taught by Anupam Datta (Anupam Datta) and Josh Reini (Josh Reini). A data agent extracts data from sources such as files or databases, analyzes it, and provides insights and visualizes its findings. But most data agents struggle with reliability or can't handle multi-step reasoning. In this course, you'll learn to build, trace, and evaluate a multi-agent workflow that plans tasks, pulls context from structured and unstructured data, performs web search, and summarizes or visualizes the final results. Learn more and enroll for free!

DeepLearning.AI

40,745 Aufrufe • vor 8 Monaten

We just launched a major new Data Engineering Professional Certificate on Coursera! Data underlies all modern AI systems, and engineers who know how to build systems to store and serve it are in high demand. If you're interested in learning this skill, please check out this 4-course sequence, which is designed to make you job-ready to be a Data Engineer. This is a new specialization taught by Joe Reis, the co-author of the best-selling book “Fundamentals of Data Engineering," in collaboration with AWS. (Disclosure, I serve on Amazon's board.) For many AI systems, data engineering is 80% of the work, and modeling is 20%. But people’s attention on these two topics is often flipped. This makes the job of the data engineer particularly important. In this professional certificate, you'll learn foundational data engineering skills while implementing modern data architectures using open-source tools: - Learn the key steps of the data lifecycle, to generate, ingest, store, transform, and serve data. - Learn to align with organizational goals to design the data pipeline right for your business' needs. - Understand how to make necessary trade-offs between speed, scalability, security, and cost. Joe has distilled into this specialization decades of experience helping startups and large companies with data infrastructure. He is also joined by 17 other industry leaders in the data field, who will help you learn in-demand skills for the growing field of data engineering. Please sign up here:
1:33

We just launched a major new Data Engineering Professional Certificate on Coursera! Data underlies all modern AI systems, and engineers who know how to build systems to store and serve it are in high demand. If you're interested in learning this skill, please check out this 4-course sequence, which is designed to make you job-ready to be a Data Engineer. This is a new specialization taught by Joe Reis, the co-author of the best-selling book “Fundamentals of Data Engineering," in collaboration with AWS. (Disclosure, I serve on Amazon's board.) For many AI systems, data engineering is 80% of the work, and modeling is 20%. But people’s attention on these two topics is often flipped. This makes the job of the data engineer particularly important. In this professional certificate, you'll learn foundational data engineering skills while implementing modern data architectures using open-source tools: - Learn the key steps of the data lifecycle, to generate, ingest, store, transform, and serve data. - Learn to align with organizational goals to design the data pipeline right for your business' needs. - Understand how to make necessary trade-offs between speed, scalability, security, and cost. Joe has distilled into this specialization decades of experience helping startups and large companies with data infrastructure. He is also joined by 17 other industry leaders in the data field, who will help you learn in-demand skills for the growing field of data engineering. Please sign up here:

Andrew Ng

118,937 Aufrufe • vor 1 Jahr

Pick up the phone, you could learn something new
0:35

Pick up the phone, you could learn something new

Niamocha🐻✨

67,907 Aufrufe • vor 1 Jahr

Shut the fuck up, watch, learn and maybe you might pick up a thing or two. 145kilos x 4 at 87kilos bw.
0:27

Shut the fuck up, watch, learn and maybe you might pick up a thing or two. 145kilos x 4 at 87kilos bw.

blaakbaki

351,749 Aufrufe • vor 2 Monaten

Announcing The Open Data Market by Stork Where onchain innovation is no longer suppressed by access to data. p.s. learn how to participate below 👀
0:31

Announcing The Open Data Market by Stork Where onchain innovation is no longer suppressed by access to data. p.s. learn how to participate below 👀

Stork

45,039 Aufrufe • vor 1 Jahr

Someone made the most addictive game to learn how real data centers work.
0:26

Someone made the most addictive game to learn how real data centers work.

mitsuri

20,294 Aufrufe • vor 2 Monaten

Starting today, the FBI will publish reported crime data to the Crime Data Explorer every month. See the newest data and learn how this timelier data can help law enforcement crush violent crime:
0:48

Starting today, the FBI will publish reported crime data to the Crime Data Explorer every month. See the newest data and learn how this timelier data can help law enforcement crush violent crime:

FBI

22,485 Aufrufe • vor 9 Monaten

Learn to train an LLM with distributed data while ensuring privacy using federated learning in a new two-part short course, Intro to Federated Learning and Federated Fine-tuning of LLMs with Private Data, created with Flower and taught by Daniel J. Beutel and nic lane. Federated learning allows a single model to be trained across multiple devices, such as phones, or multiple organizations, such as hospitals, without the need to share data to a central server. This two-part course gives you an introduction to federated learning, and then teaches you how to fine-tune your large language model with distributed data using Flower Lab’s open source federated learning framework. You’ll learn: - How to use federated learning to train a variety of models, ranging from speech and vision models to LLMs, across distributed data while offering data privacy options to users and organizations. - Privacy Enhancing Technologies like differential privacy (DP), which obscures individual data by adding calibrated noise to query results. - Two variants of differential privacy - Central and Local - and how to choose depending on your use case. - How to measure and decrease bandwidth usage to make federated learning more practical and efficient with techniques like using pre-trained models and Parameter-Efficient Fine-Tuning - How federated LLM fine-tuning reduces the risk of leaking training data. Sign up here!
4:34

Learn to train an LLM with distributed data while ensuring privacy using federated learning in a new two-part short course, Intro to Federated Learning and Federated Fine-tuning of LLMs with Private Data, created with Flower and taught by Daniel J. Beutel and nic lane. Federated learning allows a single model to be trained across multiple devices, such as phones, or multiple organizations, such as hospitals, without the need to share data to a central server. This two-part course gives you an introduction to federated learning, and then teaches you how to fine-tune your large language model with distributed data using Flower Lab’s open source federated learning framework. You’ll learn: - How to use federated learning to train a variety of models, ranging from speech and vision models to LLMs, across distributed data while offering data privacy options to users and organizations. - Privacy Enhancing Technologies like differential privacy (DP), which obscures individual data by adding calibrated noise to query results. - Two variants of differential privacy - Central and Local - and how to choose depending on your use case. - How to measure and decrease bandwidth usage to make federated learning more practical and efficient with techniques like using pre-trained models and Parameter-Efficient Fine-Tuning - How federated LLM fine-tuning reduces the risk of leaking training data. Sign up here!

Andrew Ng

64,517 Aufrufe • vor 1 Jahr

🚀 Framer Forms are here! Enjoy 10+ input types, custom states, and secure data handling. Send data via email, Google Sheets, or custom Webhook. Learn more below.
0:47

🚀 Framer Forms are here! Enjoy 10+ input types, custom states, and secure data handling. Send data via email, Google Sheets, or custom Webhook. Learn more below.

Framer

100,853 Aufrufe • vor 1 Jahr

Learn why oracles need low latency data 🔮
0:27

Learn why oracles need low latency data 🔮

Pyth Network 🔮

24,981 Aufrufe • vor 2 Jahren

Announcing the expansion of Microsoft Sentinel with a modern data lake and dynamic threat intel capabilities. Learn how it: 💰Optimizes costs 📈Simplifies data management ⚡Accelerates AI adoption
0:45

Announcing the expansion of Microsoft Sentinel with a modern data lake and dynamic threat intel capabilities. Learn how it: 💰Optimizes costs 📈Simplifies data management ⚡Accelerates AI adoption

Microsoft Security

28,901 Aufrufe • vor 10 Monaten

Dear Data Analyst, learn how to display percentages on a column chart in Power BI 🙉 DAX formula used: %MaritalStatus = DIVIDE( COUNTROWS('customer data'), CALCULATE(COUNTROWS('customer data'), ALL('customer data'[Marital_Status])))
0:50

Dear Data Analyst, learn how to display percentages on a column chart in Power BI 🙉 DAX formula used: %MaritalStatus = DIVIDE( COUNTROWS('customer data'), CALCULATE(COUNTROWS('customer data'), ALL('customer data'[Marital_Status])))

Zehida

22,557 Aufrufe • vor 1 Jahr

DATA AVAILABILITY SAMPLING EXPLAINED! learn about -Modular Blockchains -probabilistic sampling techniques -Erasure Encoding Full vid below👇
0:44

DATA AVAILABILITY SAMPLING EXPLAINED! learn about -Modular Blockchains -probabilistic sampling techniques -Erasure Encoding Full vid below👇

gogoDiego

93,106 Aufrufe • vor 2 Jahren

AI Czar David Sacks says American companies will learn efficiency techniques from China's DeepSeek AI model, but big AI data centers are still needed and scaling the biggest data centers is still an advantage
0:41

AI Czar David Sacks says American companies will learn efficiency techniques from China's DeepSeek AI model, but big AI data centers are still needed and scaling the biggest data centers is still an advantage

Tsarathustra

360,750 Aufrufe • vor 1 Jahr

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?
1:59

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?

Zach Wilson

16,688 Aufrufe • vor 1 Jahr

Make data work smarter. 💡💪 Storage Browser for #AmazonS3 in Salesforce Data Cloud gives teams direct access to customer data, right where they need it. Learn more ➡️
0:29

Make data work smarter. 💡💪 Storage Browser for #AmazonS3 in Salesforce Data Cloud gives teams direct access to customer data, right where they need it. Learn more ➡️

Amazon Web Services

13,077 Aufrufe • vor 1 Jahr