Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Getting a big tech data engineer job in 2016: - do you know SQL? - yes - here’s $500k Getting a big tech data engineer job in 2024: - do you know Spark, Kafka, Iceberg? - yes - did you shake hands with Bill Inmon when he invented the... show more

Zach Wilson

48,608 subscribers

43,547 views • 1 year ago •via X (Twitter)

Comedy Science & Technology Education

Anya Rossi• Live Now

Private livecam show

5 Comments

DataProfessorX1 year ago

People still underestimate the AI effect, In a couple years more, it will become more clear

MicroSectors3 years ago

Are you a sophisticated investor that is bullish on big tech stocks? Learn more about $BULZ here:

XyzzyPoof1 year ago

4) the y401B Visa workers are paid 50% or less of what you are

Zach Morris Wilson1 year ago

@SeamansRC No they aren’t 😂

silverlightwa1 year ago

Damn you are still grifting?

Related Videos

Are you ready to take your career to the highest level? Boost your career prospects by acquiring highly sought-after Data skills. Jumpstart your tech career as a certified Data Analyst, Data Scientist, or Data Engineer. Start Now!

Are you ready to take your career to the highest level? Boost your career prospects by acquiring highly sought-after Data skills. Jumpstart your tech career as a certified Data Analyst, Data Scientist, or Data Engineer. Start Now!

AltSchool Africa

8,790,738 views • 2 years ago

Here’s how I would learn data engineering in 2025: 1. The basics: - learn SQL — SELECT, FROM, WHERE, GROUP BY, JOIN, HAVING, etc - learn Python — data structures: objects, arrays, tuples, namedtuples — algorithms: recursion, loops 2. Intermediate - learn distributed compute — pick up PySpark or Snowflake or BigQuery - learn data make architecture — pick up iceberg or delta lake - learn job orchestration — pick up Airflow or Mage - learn data quality — pick up Great expectations 3. Advanced - learn the data modeling techniques — one big table vs kimball vs Inmon vs data vault techniques - learn machine learning features and vector databases — pick up pinecone and how to fine tune LLMs with high quality data My newsletter has a deeper roadmap here:

Here’s how I would learn data engineering in 2025: 1. The basics: - learn SQL — SELECT, FROM, WHERE, GROUP BY, JOIN, HAVING, etc - learn Python — data structures: objects, arrays, tuples, namedtuples — algorithms: recursion, loops 2. Intermediate - learn distributed compute — pick up PySpark or Snowflake or BigQuery - learn data make architecture — pick up iceberg or delta lake - learn job orchestration — pick up Airflow or Mage - learn data quality — pick up Great expectations 3. Advanced - learn the data modeling techniques — one big table vs kimball vs Inmon vs data vault techniques - learn machine learning features and vector databases — pick up pinecone and how to fine tune LLMs with high quality data My newsletter has a deeper roadmap here:

Zach Wilson

29,164 views • 1 year ago

We just launched a major new Data Engineering Professional Certificate on Coursera! Data underlies all modern AI systems, and engineers who know how to build systems to store and serve it are in high demand. If you're interested in learning this skill, please check out this 4-course sequence, which is designed to make you job-ready to be a Data Engineer. This is a new specialization taught by Joe Reis, the co-author of the best-selling book “Fundamentals of Data Engineering," in collaboration with AWS. (Disclosure, I serve on Amazon's board.) For many AI systems, data engineering is 80% of the work, and modeling is 20%. But people’s attention on these two topics is often flipped. This makes the job of the data engineer particularly important. In this professional certificate, you'll learn foundational data engineering skills while implementing modern data architectures using open-source tools: - Learn the key steps of the data lifecycle, to generate, ingest, store, transform, and serve data. - Learn to align with organizational goals to design the data pipeline right for your business' needs. - Understand how to make necessary trade-offs between speed, scalability, security, and cost. Joe has distilled into this specialization decades of experience helping startups and large companies with data infrastructure. He is also joined by 17 other industry leaders in the data field, who will help you learn in-demand skills for the growing field of data engineering. Please sign up here:

We just launched a major new Data Engineering Professional Certificate on Coursera! Data underlies all modern AI systems, and engineers who know how to build systems to store and serve it are in high demand. If you're interested in learning this skill, please check out this 4-course sequence, which is designed to make you job-ready to be a Data Engineer. This is a new specialization taught by Joe Reis, the co-author of the best-selling book “Fundamentals of Data Engineering," in collaboration with AWS. (Disclosure, I serve on Amazon's board.) For many AI systems, data engineering is 80% of the work, and modeling is 20%. But people’s attention on these two topics is often flipped. This makes the job of the data engineer particularly important. In this professional certificate, you'll learn foundational data engineering skills while implementing modern data architectures using open-source tools: - Learn the key steps of the data lifecycle, to generate, ingest, store, transform, and serve data. - Learn to align with organizational goals to design the data pipeline right for your business' needs. - Understand how to make necessary trade-offs between speed, scalability, security, and cost. Joe has distilled into this specialization decades of experience helping startups and large companies with data infrastructure. He is also joined by 17 other industry leaders in the data field, who will help you learn in-demand skills for the growing field of data engineering. Please sign up here:

Andrew Ng

118,937 views • 1 year ago

Did you know that Facebook made $134.9 billion in 2023 from selling your data? Our society has come to the point where we accepted an unfair model in which privacy is being exploited and data gifted to big tech that is selling it for billions. We intend to break the norm of how data is shared on the internet because it is outdated and unfair. You should be empowered with more control and monetization models for your data!

Did you know that Facebook made $134.9 billion in 2023 from selling your data? Our society has come to the point where we accepted an unfair model in which privacy is being exploited and data gifted to big tech that is selling it for billions. We intend to break the norm of how data is shared on the internet because it is outdated and unfair. You should be empowered with more control and monetization models for your data!

Solana ID 🪷

30,447 views • 1 year ago

Meet Houston, your AI data engineer. Run natural language queries with prompt-to-SQL and generate instant charts and dashboards. Build a data pipeline in seconds with our OpenAI chatbot. 🚀#MeetHouston

Meet Houston, your AI data engineer. Run natural language queries with prompt-to-SQL and generate instant charts and dashboards. Build a data pipeline in seconds with our OpenAI chatbot. 🚀#MeetHouston

Space and Time

141,557 views • 2 years ago

The future of retail data is decentralized. POPI ensures your data works for you, not just big tech companies. Earn money while contributing to a fairer, more transparent data economy. 💡

The future of retail data is decentralized. POPI ensures your data works for you, not just big tech companies. Earn money while contributing to a fairer, more transparent data economy. 💡

POPAI

15,176 views • 1 year ago

❌ Data Science ✅ Data Engineering is the sexiest job of the 21st century Top 7 skills you can master for Data Engineering /🧵/

❌ Data Science ✅ Data Engineering is the sexiest job of the 21st century Top 7 skills you can master for Data Engineering /🧵/

Darshil | Data Engineer👨🏻‍🔧

69,092 views • 3 years ago

I don't know if SQL is bad or not but losing user data in the middle of a workflow certainly is! Did you know you can block React Router navigations in to prevent losing user data? let blocker = useBlocker(shouldBlock) blocker.state blocker.proceed() blocker.reset()

I don't know if SQL is bad or not but losing user data in the middle of a workflow certainly is! Did you know you can block React Router navigations in to prevent losing user data? let blocker = useBlocker(shouldBlock) blocker.state blocker.proceed() blocker.reset()

Ryan Florence

41,462 views • 1 year ago

🚨Governor DeSantis says he is about to sign legislation that will effectively BLOCK hyperscale AI data centers from coming to Florida! “They come in and these big tech companies are building these massive data centers that take up as much power as a city of half a million people. Well guess what? Supply and demand. Do you think your electricity rates go up when that happens? Yes, it does. So this would BLOCK that basically. No rate payer should have to pay ONE RED CENT because of big tech and we'll be one of the first states to do it!”

🚨Governor DeSantis says he is about to sign legislation that will effectively BLOCK hyperscale AI data centers from coming to Florida! “They come in and these big tech companies are building these massive data centers that take up as much power as a city of half a million people. Well guess what? Supply and demand. Do you think your electricity rates go up when that happens? Yes, it does. So this would BLOCK that basically. No rate payer should have to pay ONE RED CENT because of big tech and we'll be one of the first states to do it!”

Chris Nelson 🏝️🇺🇸

88,312 views • 3 months ago

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?

Zach Wilson

20,368 views • 11 months ago

#Web3 and digital property rights truly excite us! Here’s why: We live in a world where our data is owned and farmed by big tech companies. But web3 presents a future where we can regain ownership of our data and subsequently assume the lead on our freedom.

#Web3 and digital property rights truly excite us! Here’s why: We live in a world where our data is owned and farmed by big tech companies. But web3 presents a future where we can regain ownership of our data and subsequently assume the lead on our freedom.

Animoca Brands

10,009 views • 2 years ago

The real reason why Big Tech needs infinite data centers.

The real reason why Big Tech needs infinite data centers.

Cernovich

493,031 views • 1 month ago

Your data. Your control. Your rewards. Dwinity lets you store, own, and monetize your personal data — without Big Tech or cloud traps.

Your data. Your control. Your rewards. Dwinity lets you store, own, and monetize your personal data — without Big Tech or cloud traps.

Dwinity

738,927 views • 1 year ago

Collins: People are saying you posted the job data early. You’re obviously not supposed to share that until the following morning. Did you do that on purpose? Trump: No. I don’t know. When people give me things, I post them.

Collins: People are saying you posted the job data early. You’re obviously not supposed to share that until the following morning. Did you do that on purpose? Trump: No. I don’t know. When people give me things, I post them.

Acyn

44,500 views • 5 months ago

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?

Zach Wilson

16,688 views • 1 year ago

Data engineering can be completely open source! You don’t need to rely on many platforms aside from maybe S3! - compute: Spark - data lake format: iceberg - orchestrator: Airflow, Mage or Dagster You can get far with just these three!

Data engineering can be completely open source! You don’t need to rely on many platforms aside from maybe S3! - compute: Spark - data lake format: iceberg - orchestrator: Airflow, Mage or Dagster You can get far with just these three!

Zach Morris Wilson

15,628 views • 1 year ago

Honestly, if you work with data, you must know SQL! It’s that simple. No exceptions. #Honestly #WorkSmart #KnowSQL

Honestly, if you work with data, you must know SQL! It’s that simple. No exceptions. #Honestly #WorkSmart #KnowSQL

Marco Russo

45,985 views • 1 year ago