正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Getting a big tech data engineer job in 2016: - do you know SQL? - yes - here’s $500k Getting a big tech data engineer job in 2024: - do you know Spark, Kafka, Iceberg? - yes - did you shake hands with Bill Inmon when he invented the... show more

Zach Wilson

48,608 subscribers

43,547 次观看 • 1 年前 •via X (Twitter)

喜剧科学技术教育

Anya Rossi• Live Now

Private livecam show

5 条评论

DataProfessorX 的头像

DataProfessorX1 年前

People still underestimate the AI effect, In a couple years more, it will become more clear

MicroSectors 的头像

MicroSectors3 年前

Are you a sophisticated investor that is bullish on big tech stocks? Learn more about $BULZ here:

XyzzyPoof 的头像

XyzzyPoof1 年前

4) the y401B Visa workers are paid 50% or less of what you are

Zach Morris Wilson 的头像

Zach Morris Wilson1 年前

@SeamansRC No they aren’t 😂

silverlightwa 的头像

silverlightwa1 年前

Damn you are still grifting?

相关视频

Are you ready to take your career to the highest level? Boost your career prospects by acquiring highly sought-after Data skills. Jumpstart your tech career as a certified Data Analyst, Data Scientist, or Data Engineer. Start Now!

Are you ready to take your career to the highest level? Boost your career prospects by acquiring highly sought-after Data skills. Jumpstart your tech career as a certified Data Analyst, Data Scientist, or Data Engineer. Start Now!

AltSchool Africa

8,790,738 次观看 • 2 年前

Here’s how I would learn data engineering in 2025: 1. The basics: - learn SQL — SELECT, FROM, WHERE, GROUP BY, JOIN, HAVING, etc - learn Python — data structures: objects, arrays, tuples, namedtuples — algorithms: recursion, loops 2. Intermediate - learn distributed compute — pick up PySpark or Snowflake or BigQuery - learn data make architecture — pick up iceberg or delta lake - learn job orchestration — pick up Airflow or Mage - learn data quality — pick up Great expectations 3. Advanced - learn the data modeling techniques — one big table vs kimball vs Inmon vs data vault techniques - learn machine learning features and vector databases — pick up pinecone and how to fine tune LLMs with high quality data My newsletter has a deeper roadmap here:

Here’s how I would learn data engineering in 2025: 1. The basics: - learn SQL — SELECT, FROM, WHERE, GROUP BY, JOIN, HAVING, etc - learn Python — data structures: objects, arrays, tuples, namedtuples — algorithms: recursion, loops 2. Intermediate - learn distributed compute — pick up PySpark or Snowflake or BigQuery - learn data make architecture — pick up iceberg or delta lake - learn job orchestration — pick up Airflow or Mage - learn data quality — pick up Great expectations 3. Advanced - learn the data modeling techniques — one big table vs kimball vs Inmon vs data vault techniques - learn machine learning features and vector databases — pick up pinecone and how to fine tune LLMs with high quality data My newsletter has a deeper roadmap here:

Zach Wilson

29,164 次观看 • 1 年前

We just launched a major new Data Engineering Professional Certificate on Coursera! Data underlies all modern AI systems, and engineers who know how to build systems to store and serve it are in high demand. If you're interested in learning this skill, please check out this 4-course sequence, which is designed to make you job-ready to be a Data Engineer. This is a new specialization taught by Joe Reis, the co-author of the best-selling book “Fundamentals of Data Engineering," in collaboration with AWS. (Disclosure, I serve on Amazon's board.) For many AI systems, data engineering is 80% of the work, and modeling is 20%. But people’s attention on these two topics is often flipped. This makes the job of the data engineer particularly important. In this professional certificate, you'll learn foundational data engineering skills while implementing modern data architectures using open-source tools: - Learn the key steps of the data lifecycle, to generate, ingest, store, transform, and serve data. - Learn to align with organizational goals to design the data pipeline right for your business' needs. - Understand how to make necessary trade-offs between speed, scalability, security, and cost. Joe has distilled into this specialization decades of experience helping startups and large companies with data infrastructure. He is also joined by 17 other industry leaders in the data field, who will help you learn in-demand skills for the growing field of data engineering. Please sign up here:

We just launched a major new Data Engineering Professional Certificate on Coursera! Data underlies all modern AI systems, and engineers who know how to build systems to store and serve it are in high demand. If you're interested in learning this skill, please check out this 4-course sequence, which is designed to make you job-ready to be a Data Engineer. This is a new specialization taught by Joe Reis, the co-author of the best-selling book “Fundamentals of Data Engineering," in collaboration with AWS. (Disclosure, I serve on Amazon's board.) For many AI systems, data engineering is 80% of the work, and modeling is 20%. But people’s attention on these two topics is often flipped. This makes the job of the data engineer particularly important. In this professional certificate, you'll learn foundational data engineering skills while implementing modern data architectures using open-source tools: - Learn the key steps of the data lifecycle, to generate, ingest, store, transform, and serve data. - Learn to align with organizational goals to design the data pipeline right for your business' needs. - Understand how to make necessary trade-offs between speed, scalability, security, and cost. Joe has distilled into this specialization decades of experience helping startups and large companies with data infrastructure. He is also joined by 17 other industry leaders in the data field, who will help you learn in-demand skills for the growing field of data engineering. Please sign up here:

Andrew Ng

118,937 次观看 • 1 年前

Did you know that Facebook made $134.9 billion in 2023 from selling your data? Our society has come to the point where we accepted an unfair model in which privacy is being exploited and data gifted to big tech that is selling it for billions. We intend to break the norm of how data is shared on the internet because it is outdated and unfair. You should be empowered with more control and monetization models for your data!

Did you know that Facebook made $134.9 billion in 2023 from selling your data? Our society has come to the point where we accepted an unfair model in which privacy is being exploited and data gifted to big tech that is selling it for billions. We intend to break the norm of how data is shared on the internet because it is outdated and unfair. You should be empowered with more control and monetization models for your data!

Solana ID 🪷

30,447 次观看 • 1 年前

Meet Houston, your AI data engineer. Run natural language queries with prompt-to-SQL and generate instant charts and dashboards. Build a data pipeline in seconds with our OpenAI chatbot. 🚀#MeetHouston

Meet Houston, your AI data engineer. Run natural language queries with prompt-to-SQL and generate instant charts and dashboards. Build a data pipeline in seconds with our OpenAI chatbot. 🚀#MeetHouston

Space and Time

141,613 次观看 • 2 年前

The future of retail data is decentralized. POPI ensures your data works for you, not just big tech companies. Earn money while contributing to a fairer, more transparent data economy. 💡

The future of retail data is decentralized. POPI ensures your data works for you, not just big tech companies. Earn money while contributing to a fairer, more transparent data economy. 💡

POPAI

15,176 次观看 • 1 年前

❌ Data Science ✅ Data Engineering is the sexiest job of the 21st century Top 7 skills you can master for Data Engineering /🧵/

❌ Data Science ✅ Data Engineering is the sexiest job of the 21st century Top 7 skills you can master for Data Engineering /🧵/

Darshil | Data Engineer👨🏻‍🔧

69,092 次观看 • 3 年前

I don't know if SQL is bad or not but losing user data in the middle of a workflow certainly is! Did you know you can block React Router navigations in to prevent losing user data? let blocker = useBlocker(shouldBlock) blocker.state blocker.proceed() blocker.reset()

I don't know if SQL is bad or not but losing user data in the middle of a workflow certainly is! Did you know you can block React Router navigations in to prevent losing user data? let blocker = useBlocker(shouldBlock) blocker.state blocker.proceed() blocker.reset()

Ryan Florence

41,462 次观看 • 1 年前

🚨Governor DeSantis says he is about to sign legislation that will effectively BLOCK hyperscale AI data centers from coming to Florida! “They come in and these big tech companies are building these massive data centers that take up as much power as a city of half a million people. Well guess what? Supply and demand. Do you think your electricity rates go up when that happens? Yes, it does. So this would BLOCK that basically. No rate payer should have to pay ONE RED CENT because of big tech and we'll be one of the first states to do it!”

🚨Governor DeSantis says he is about to sign legislation that will effectively BLOCK hyperscale AI data centers from coming to Florida! “They come in and these big tech companies are building these massive data centers that take up as much power as a city of half a million people. Well guess what? Supply and demand. Do you think your electricity rates go up when that happens? Yes, it does. So this would BLOCK that basically. No rate payer should have to pay ONE RED CENT because of big tech and we'll be one of the first states to do it!”

Chris Nelson 🏝️🇺🇸

88,312 次观看 • 3 个月前

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?

Zach Wilson

20,368 次观看 • 11 个月前

#Web3 and digital property rights truly excite us! Here’s why: We live in a world where our data is owned and farmed by big tech companies. But web3 presents a future where we can regain ownership of our data and subsequently assume the lead on our freedom.

#Web3 and digital property rights truly excite us! Here’s why: We live in a world where our data is owned and farmed by big tech companies. But web3 presents a future where we can regain ownership of our data and subsequently assume the lead on our freedom.

Animoca Brands

10,009 次观看 • 2 年前

The real reason why Big Tech needs infinite data centers.

The real reason why Big Tech needs infinite data centers.

Cernovich

493,101 次观看 • 1 个月前

Your data. Your control. Your rewards. Dwinity lets you store, own, and monetize your personal data — without Big Tech or cloud traps.

Your data. Your control. Your rewards. Dwinity lets you store, own, and monetize your personal data — without Big Tech or cloud traps.

Dwinity

738,927 次观看 • 1 年前

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?

Zach Wilson

16,688 次观看 • 1 年前

Collins: People are saying you posted the job data early. You’re obviously not supposed to share that until the following morning. Did you do that on purpose? Trump: No. I don’t know. When people give me things, I post them.

Collins: People are saying you posted the job data early. You’re obviously not supposed to share that until the following morning. Did you do that on purpose? Trump: No. I don’t know. When people give me things, I post them.

Acyn

44,500 次观看 • 5 个月前

Data engineering can be completely open source! You don’t need to rely on many platforms aside from maybe S3! - compute: Spark - data lake format: iceberg - orchestrator: Airflow, Mage or Dagster You can get far with just these three!

Data engineering can be completely open source! You don’t need to rely on many platforms aside from maybe S3! - compute: Spark - data lake format: iceberg - orchestrator: Airflow, Mage or Dagster You can get far with just these three!

Zach Morris Wilson

15,628 次观看 • 1 年前

Honestly, if you work with data, you must know SQL! It’s that simple. No exceptions. #Honestly #WorkSmart #KnowSQL

Honestly, if you work with data, you must know SQL! It’s that simple. No exceptions. #Honestly #WorkSmart #KnowSQL

Marco Russo

45,985 次观看 • 1 年前