Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Reworkd is your scraping co-pilot. It understands website structures and auto-generates Playwright code to take actions, visit subpages, scrape, and save data based on your custom schema. With Reworkd you can simplify web data extraction at scale.

Y Combinator

1,637,797 subscribers

63,943 views • 1 year ago •via X (Twitter)

Education Finance Science & Technology

Anya Rossi• Live Now

Private livecam show

11 Comments

anirudh1 year ago

@ReworkdAI stagehand can help when your playwright code breaks!

HUDI2 years ago

🚀🌍Transform your online footprint with #IPDW - a groundbreaking protocol that decentralizes your data storage & exchange. Inspired by BitTorrent, we're creating a new era of data sovereignty with #HUDI. 🛡️ Secure, encrypted, and fully in your control, our P2P technology integrates seamlessly into your browser, making every user a vital node in our expansive Web 4.0 network. Ready to reclaim your digital power? Dive into the revolution at and discover the backbone tech at #DecentralizeYourData #Web4

Artypone1 year ago

@ReworkdAI What a disappointment for the investments after MCP. You can do it by developing your own scraptool in 10 mins and keep your scrapped data in privately. There like 10 opensource projects. Would you like me to share github links?

Israr Ahmed1 year ago

@ReworkdAI Scaling web data extraction is revolutionary. At @qryptum, we’re building decentralized AI and data solutions. What are your thoughts on how Reworkd could integrate with decentralized data sources?

vito1 year ago

@ReworkdAI cool! just what i was looking for.

Steph from OpenVC1 year ago

@ReworkdAI This is awesome!

Raiders1 year ago

@ReworkdAI Can we scrape twitter posts too? I doubt

Ashish1 year ago

This is useless, even for simple single page websites. I tried scrapping 2 very simple web pages, and it failed miserably. Now think of all varying complexities with different sites(captchas, geolocation, sessions), it does not stand a chance. If it is only good for basic sites, it is too costly. I can get that done on chatgpt subscription.

Tyler Clark1 year ago

@ReworkdAI @firecrawl_dev 4 life! 🔥

Vinnie 🌳1 year ago

@ReworkdAI 🌿 ReworkdAI could be a game-changer for eco-data projects! Let's see it simplify nature-data extraction! 🚀 #EcoData

Lantos (e/acc)1 year ago

@ReworkdAI @talos256

Related Videos

Thunderbit is probably the most powerful web scraper right now. It can scrape any website in just 2 clicks, no code, no setup. It’s smart enough to understand complex page layouts and nested structures. Normally, scraping this kind of data manually takes hours of tedious copy-paste work. With Thunderbit, you click twice, and it auto-builds the table for you, scraping in seconds. More details and examples below 👇 #Thunderbit #WebScraping #AITools #BlackFridayDeals

Thunderbit is probably the most powerful web scraper right now. It can scrape any website in just 2 clicks, no code, no setup. It’s smart enough to understand complex page layouts and nested structures. Normally, scraping this kind of data manually takes hours of tedious copy-paste work. With Thunderbit, you click twice, and it auto-builds the table for you, scraping in seconds. More details and examples below 👇 #Thunderbit #WebScraping #AITools #BlackFridayDeals

AshutoshShrivastava

17,857 views • 10 months ago

Excited to more broadly share what we've been working on at Reworkd ⚡️ This past year we've fully invested in building the next generation of web agents. They're already live in production today across a handful of companies serving as the simplest way to extract web data at scale. Come to us with a list of 100s-1000s of websites + a data schema, and we'll handle the rest 🤖

Excited to more broadly share what we've been working on at Reworkd ⚡️ This past year we've fully invested in building the next generation of web agents. They're already live in production today across a handful of companies serving as the simplest way to extract web data at scale. Come to us with a list of 100s-1000s of websites + a data schema, and we'll handle the rest 🤖

asim

94,665 views • 2 years ago

You can now scrape any website with AI in two clicks. Thunderbit is a zero-setup web scraper that extracts data from any site instantly, it even allows you subpage scraping. Here’s how to try free:

You can now scrape any website with AI in two clicks. Thunderbit is a zero-setup web scraper that extracts data from any site instantly, it even allows you subpage scraping. Here’s how to try free:

Alvaro Cintas

27,254 views • 10 months ago

You can now clone any website just by writing a prompt. Simply add the Firecrawl MCP server to your favorite AI coding tool for improved web data extraction, and let Claude code it for you. Here’s how:

You can now clone any website just by writing a prompt. Simply add the Firecrawl MCP server to your favorite AI coding tool for improved web data extraction, and let Claude code it for you. Here’s how:

Alvaro Cintas

83,031 views • 1 year ago

You can now clone any website just by writing a prompt. Simply add the new Firecrawl MCP server to your favorite AI coding tool for improved web data extraction, and let Claude code it for you. Here’s how:

You can now clone any website just by writing a prompt. Simply add the new Firecrawl MCP server to your favorite AI coding tool for improved web data extraction, and let Claude code it for you. Here’s how:

Alvaro Cintas

540,932 views • 1 year ago

Web scraping is a critical skill, and yet nobody talks about it. How do you think companies are training their Large Language Models? Where do you think the data comes from? But web scraping goes beyond all of that. Imagine giving an AI agent access to any public online data in real time! I like to call this "web-scrapping on demand", and I'm pretty sure it's going to unlock unlimited power for AI applications. I recorded a quick video to show you how you can do this using Apify. I've talked about them before, and they are collaborating with me on this post. They have one of the best open-source web scraping and browser automation libraries out there: But it gets much better than this! You can use MCP to connect your AI Agents and applications to the Apify platform and use any specialized actor on demand to scrape and process online data. In the video, I used Cursor to scrape LinkedIn posts with the words "Machine Learning" in real time. Worked like a charm with no code needed! Here is a link to the platform: Think about this: You can now feed your AI applications with any public data on demand! We aren't ready for what's coming.

Web scraping is a critical skill, and yet nobody talks about it. How do you think companies are training their Large Language Models? Where do you think the data comes from? But web scraping goes beyond all of that. Imagine giving an AI agent access to any public online data in real time! I like to call this "web-scrapping on demand", and I'm pretty sure it's going to unlock unlimited power for AI applications. I recorded a quick video to show you how you can do this using Apify. I've talked about them before, and they are collaborating with me on this post. They have one of the best open-source web scraping and browser automation libraries out there: But it gets much better than this! You can use MCP to connect your AI Agents and applications to the Apify platform and use any specialized actor on demand to scrape and process online data. In the video, I used Cursor to scrape LinkedIn posts with the words "Machine Learning" in real time. Worked like a charm with no code needed! Here is a link to the platform: Think about this: You can now feed your AI applications with any public data on demand! We aren't ready for what's coming.

Santiago

101,473 views • 1 year ago

The Firecrawl plugin is now available through Anthropic's official marketplace 🔌 1. Run /plugin 2. Install Firecrawl 3. Ask Claude to fetch the web data you need Scrape and extract data from any website, directly inside Claude Code!

The Firecrawl plugin is now available through Anthropic's official marketplace 🔌 1. Run /plugin 2. Install Firecrawl 3. Ask Claude to fetch the web data you need Scrape and extract data from any website, directly inside Claude Code!

Firecrawl

91,889 views • 7 months ago

🚨 THIS IS ACTUALLY INSANE Your AI agent can have access to the web. But if it can't reliably read what’s actually on the page, that access is almost useless. We looked at Firecrawl as the web layer for AI agents and the numbers are hard to ignore. The setup is simple: Give it a URL, search query, or website. Firecrawl handles the ugly part — crawling, scraping, rendering, extracting, and turning web content into something an AI model can actually use. The headline numbers: → 173,000+ GitHub stars → Search, scrape and interact with the web at scale → Supports web pages, PDFs, DOCX and other content → Structured data extraction for AI workflows → MCP support for connecting it directly to AI agents The workflow looks like this: Search → Scrape → Crawl → Extract → Feed the agent Three things stand out: 1. Scraping becomes an infrastructure layer Instead of maintaining your own pile of HTTP clients, parsers, browser automation and retry logic, you can treat web access as an API. 2. Agents get more than raw HTML The goal isn't just downloading a webpage. It's turning messy web content into clean context that an LLM can reason over. 3. The same layer works across different agent workflows Research agents. RAG pipelines. AI search. Competitive intelligence. Web-data extraction. The interesting shift: AI agents don't just need better models. They need better access to the information those models are supposed to reason about. Firecrawl is building that layer. Save this repo.

🚨 THIS IS ACTUALLY INSANE Your AI agent can have access to the web. But if it can't reliably read what’s actually on the page, that access is almost useless. We looked at Firecrawl as the web layer for AI agents and the numbers are hard to ignore. The setup is simple: Give it a URL, search query, or website. Firecrawl handles the ugly part — crawling, scraping, rendering, extracting, and turning web content into something an AI model can actually use. The headline numbers: → 173,000+ GitHub stars → Search, scrape and interact with the web at scale → Supports web pages, PDFs, DOCX and other content → Structured data extraction for AI workflows → MCP support for connecting it directly to AI agents The workflow looks like this: Search → Scrape → Crawl → Extract → Feed the agent Three things stand out: 1. Scraping becomes an infrastructure layer Instead of maintaining your own pile of HTTP clients, parsers, browser automation and retry logic, you can treat web access as an API. 2. Agents get more than raw HTML The goal isn't just downloading a webpage. It's turning messy web content into clean context that an LLM can reason over. 3. The same layer works across different agent workflows Research agents. RAG pipelines. AI search. Competitive intelligence. Web-data extraction. The interesting shift: AI agents don't just need better models. They need better access to the information those models are supposed to reason about. Firecrawl is building that layer. Save this repo.

Vikas gupta

18,053 views • 12 days ago

🌊TrellisInsights (YC W24) makes unstructured data SQL-ready. It extracts and transforms your unstructured data to SQL-compliant tables with schema you define with natural language. Congrats on the launch, Mac Klinkachorn and Jacky Lin!

🌊TrellisInsights (YC W24) makes unstructured data SQL-ready. It extracts and transforms your unstructured data to SQL-compliant tables with schema you define with natural language. Congrats on the launch, Mac Klinkachorn and Jacky Lin!

Y Combinator

14,213 views • 2 years ago

YOUR AI AGENT CAN NOW COLLECT DATA FROM ALMOST ANY WEBSITE X, YouTube, Reddit, random websites, basically whatever you need for research, analytics or monitoring. Here are 3 GitHub tools that let your agent pull the data for you. 1. Agent-Reach Bundles tools for working with different platforms in one place. X, YouTube, Reddit, GitHub and more. 2. Patchright Enhanced Works through the browser with Playwright. You can listen to the requests a website makes and pull the data you need through a script. So if a website doesn’t have a proper API, you still have a way to get the data. 3. Scrapling A general purpose tool for pulling data from web pages. And the best part is that you can give all of this to your AI agent. For example: “Collect posts from 100 X accounts from the last month, find the most popular topics and analyze what’s working.” Or: “Collect reviews about this product from Reddit and YouTube and summarize the main complaints.” You tell the agent what data you need → it writes the code → collects the data → gives you the result. Basically, you’re giving your agent the ability to pull data from almost anywhere on the web. That opens up a whole different level of stuff you can automate.

YOUR AI AGENT CAN NOW COLLECT DATA FROM ALMOST ANY WEBSITE X, YouTube, Reddit, random websites, basically whatever you need for research, analytics or monitoring. Here are 3 GitHub tools that let your agent pull the data for you. 1. Agent-Reach Bundles tools for working with different platforms in one place. X, YouTube, Reddit, GitHub and more. 2. Patchright Enhanced Works through the browser with Playwright. You can listen to the requests a website makes and pull the data you need through a script. So if a website doesn’t have a proper API, you still have a way to get the data. 3. Scrapling A general purpose tool for pulling data from web pages. And the best part is that you can give all of this to your AI agent. For example: “Collect posts from 100 X accounts from the last month, find the most popular topics and analyze what’s working.” Or: “Collect reviews about this product from Reddit and YouTube and summarize the main complaints.” You tell the agent what data you need → it writes the code → collects the data → gives you the result. Basically, you’re giving your agent the ability to pull data from almost anywhere on the web. That opens up a whole different level of stuff you can automate.

TeddyCEO

189,086 views • 2 days ago

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?

Zach Wilson

20,412 views • 1 year ago

Wow AI is your one-stop shop for everything AI data training: 1️⃣ OTS Dataset 2️⃣ Custom Data Collection 3️⃣ Labeling 4️⃣ Crowdsourcing With Wow AI, you can be confident that your AI training data is of the highest quality, ethical, and compliant. Visit us 👉

Wow AI is your one-stop shop for everything AI data training: 1️⃣ OTS Dataset 2️⃣ Custom Data Collection 3️⃣ Labeling 4️⃣ Crowdsourcing With Wow AI, you can be confident that your AI training data is of the highest quality, ethical, and compliant. Visit us 👉

Wow AI

52,044 views • 2 years ago

Build your own Gen AI Agents with Salesforce’s new Agentforce platform—anyone can create, test, and scale custom Gen AI Agents using our unique Einstein1 low-code platform—deeply integrated with our Einstein1 platform. The future of AI Agent and App Dev is in your hands! With Agentforce, your custom AI agents: - Achieve greater accuracy by securely grounding in all your enterprise data with Data Cloud - Take action seamlessly by leveraging Flow automations - Connect effortlessly to your enterprise APIs through MuleSoft - Test with confidence in Sandboxes before deployment - Use in your apps, web, social, and Slack. Humans with AI drive customer success, together. Now everyone is an Einstein. More to come at Dreamforce 2024! 💙

Build your own Gen AI Agents with Salesforce’s new Agentforce platform—anyone can create, test, and scale custom Gen AI Agents using our unique Einstein1 low-code platform—deeply integrated with our Einstein1 platform. The future of AI Agent and App Dev is in your hands! With Agentforce, your custom AI agents: - Achieve greater accuracy by securely grounding in all your enterprise data with Data Cloud - Take action seamlessly by leveraging Flow automations - Connect effortlessly to your enterprise APIs through MuleSoft - Test with confidence in Sandboxes before deployment - Use in your apps, web, social, and Slack. Humans with AI drive customer success, together. Now everyone is an Einstein. More to come at Dreamforce 2024! 💙

Marc Benioff

71,010 views • 2 years ago

Open Interpreter is like ChatGPT Code Interpreter, but you can run it locally on your computer, and it is 100% open-source. Write and execute code, analyze data, control your computer, and more. Great work killian Here's how to install and use it: 👇

Open Interpreter is like ChatGPT Code Interpreter, but you can run it locally on your computer, and it is 100% open-source. Write and execute code, analyze data, control your computer, and more. Great work killian Here's how to install and use it: 👇

Matthew Berman

142,937 views • 2 years ago

See your data on Google Earth's real-world canvas. You can now import large-scale KML and GeoJSON datasets—up to billions of features—directly into Google Earth. Don’t just view your data; validate it against reality. Customize styling, create custom data layers, and uncover new insights to create visualizations that drive decisions. Take advantage of increased storage and file size quotas in our professional plans to securely overlay your proprietary intelligence atop our trusted imagery and data layers to create a single source of truth.

See your data on Google Earth's real-world canvas. You can now import large-scale KML and GeoJSON datasets—up to billions of features—directly into Google Earth. Don’t just view your data; validate it against reality. Customize styling, create custom data layers, and uncover new insights to create visualizations that drive decisions. Take advantage of increased storage and file size quotas in our professional plans to securely overlay your proprietary intelligence atop our trusted imagery and data layers to create a single source of truth.

Google Earth

25,901 views • 8 months ago