Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Introducing Swarm Web Extractor 🤖 Based on an objective, it autonomously searches the web, maps entire websites, and extracts data. Built with OpenAI’s new multi-agent framework Swarm, SerpApi, and . Check it out:

Eric Ciarla (hiring)

16,093 subscribers

91,788 views • 1 year ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

11 Comments

Eric Ciarla1 year ago

Check out the example here:

Eric Ciarla1 year ago

Inspired by @nickscamara_'s super cool tutorial!

Mendable1 year ago

@serp_api @firecrawl_dev This is awesome!

Eric Ciarla1 year ago

@serp_api @firecrawl_dev Appreciate it!

Firecrawl1 year ago

@serp_api Wow - super awesome!

Eric Ciarla1 year ago

@serp_api Thank you!

Rishi Raj Jain1 year ago

@serp_api @firecrawl_dev I just tried this and it's missing a vital step: - pip install git+

Matt Mazzeo1 year ago

@serp_api @firecrawl_dev Badass

Eric Ciarla1 year ago

@serp_api @firecrawl_dev 🫡

Udit Goenka1 year ago

@serp_api @firecrawl_dev Every time I try to install the open source, it asks for API? Am I missing something?

Eric Ciarla1 year ago

@serp_api @firecrawl_dev For this you are going to want api keys for openai, serper, and firecrawl

Related Videos

Announcing /extract beta 🔥 Extract structured data from single URLs, multiple URLs, or even entire websites! This new endpoint crawls entire websites for you and then extracts data based on a prompt. Check it out:

Announcing /extract beta 🔥 Extract structured data from single URLs, multiple URLs, or even entire websites! This new endpoint crawls entire websites for you and then extracts data based on a prompt. Check it out:

Firecrawl

71,968 views • 1 year ago

Introducing Claude 3.7 Web Crawler 🕸️ It finds pages, re-ranks them, then navigates entire websites with Anthropic's new model, returning data according to the entered objective. Powered by 3.7 Sonnet for powerful near-instant responses and Firecrawl

Introducing Claude 3.7 Web Crawler 🕸️ It finds pages, re-ranks them, then navigates entire websites with Anthropic's new model, returning data according to the entered objective. Powered by 3.7 Sonnet for powerful near-instant responses and Firecrawl

Caleb Peffer (Hiring!)

66,459 views • 1 year ago

What if AI agents could hunt for yield on the web? We built a small demo using the TinyFish Web Agent API The agent navigates websites, executes multi-step research workflows, completely autonomously. Exciting for the future of autonomous yield strategies. #TinyFishAccelerator #BuildInPublic

What if AI agents could hunt for yield on the web? We built a small demo using the TinyFish Web Agent API The agent navigates websites, executes multi-step research workflows, completely autonomously. Exciting for the future of autonomous yield strategies. #TinyFishAccelerator #BuildInPublic

YieldFi

10,901 views • 3 months ago

Introducing Spark 1 Pro and Spark 1 Mini 🔥 Two new models powering /agent, our state of the art agent that searches, navigates, and extracts web data from a prompt. Mini is 60% cheaper and Pro delivers higher accuracy, making this our most powerful extraction endpoint yet.

Introducing Spark 1 Pro and Spark 1 Mini 🔥 Two new models powering /agent, our state of the art agent that searches, navigates, and extracts web data from a prompt. Mini is 60% cheaper and Pro delivers higher accuracy, making this our most powerful extraction endpoint yet.

Firecrawl

30,271 views • 5 months ago

Introducing Prometheus, an experimental Forward Deployed Agent for web data. Describe the web data you need and it writes Firecrawl code to collect it. Run it yourself or let us host and automatically maintain it as pages change. Try it with Claude Fable 5 for free this week!

Introducing Prometheus, an experimental Forward Deployed Agent for web data. Describe the web data you need and it writes Firecrawl code to collect it. Run it yourself or let us host and automatically maintain it as pages change. Try it with Claude Fable 5 for free this week!

Firecrawl

74,945 views • 11 days ago

Oh wow…Firecrawl just dropped /agent, an API that autonomously navigates and discovers data from any website. You tell it what data you need, it figures out how to find it across complex sites. This is the future of web data.

Oh wow…Firecrawl just dropped /agent, an API that autonomously navigates and discovers data from any website. You tell it what data you need, it figures out how to find it across complex sites. This is the future of web data.

Alvaro Cintas

158,543 views • 6 months ago

I built an AI startup insight tool Agent using Firecrawl FIRE-1 and GPT-4o. FIRE-1 AI Agent navigates websites, clicks buttons, and extracts company data autonomously without writing a single line of custom code. 100% Opensource Code with step-by-step tutorial.

I built an AI startup insight tool Agent using Firecrawl FIRE-1 and GPT-4o. FIRE-1 AI Agent navigates websites, clicks buttons, and extracts company data autonomously without writing a single line of custom code. 100% Opensource Code with step-by-step tutorial.

Shubham Saboo

23,184 views • 1 year ago

I just found a web agent framework built specifically for LLMs. It turns any site into clean, agent-ready text. Any LLM can now act on the web. It’s called Notte, and it changes everything. Let me show you how:

I just found a web agent framework built specifically for LLMs. It turns any site into clean, agent-ready text. Any LLM can now act on the web. It’s called Notte, and it changes everything. Let me show you how:

Markandey Sharma

52,026 views • 1 year ago

🤖 Agents SDK—our new open-source SDK for orchestrating multi-agent workflows, improving upon Swarm. Configure agents with built-in tools, hand off tasks, add safety guardrails, and visualize execution traces for debugging and optimizing performance.

🤖 Agents SDK—our new open-source SDK for orchestrating multi-agent workflows, improving upon Swarm. Configure agents with built-in tools, hand off tasks, add safety guardrails, and visualize execution traces for debugging and optimizing performance.

OpenAI Developers

164,359 views • 1 year ago

Introducing web-agent, an open framework for building web agents 🔥 Build AI agents that search, scrape, and interact with the web - powered by the same architecture behind our /agent endpoint. 100% open source. Bring any model. Anthropic, OpenAI, or your own.

Introducing web-agent, an open framework for building web agents 🔥 Build AI agents that search, scrape, and interact with the web - powered by the same architecture behind our /agent endpoint. 100% open source. Bring any model. Anthropic, OpenAI, or your own.

Firecrawl

123,163 views • 2 months ago

pi-web-access is an extension that provides web search (and more) for Pi coding agent. The latest update introduces a new UX pattern: you can hit `Ctrl+S` to intercept agent searches mid-flight to inspect and curate exactly what makes it into the agent conversation. pi install npm:pi-web-access

pi-web-access is an extension that provides web search (and more) for Pi coding agent. The latest update introduces a new UX pattern: you can hit `Ctrl+S` to intercept agent searches mid-flight to inspect and curate exactly what makes it into the agent conversation. pi install npm:pi-web-access

Nico Bailon

29,352 views • 4 months ago

Operator is an AI agent by ChatGPT that browses the web and performs tasks. Currently in research preview with restrictions, it evolves with feedback. Check out the use cases below: 1. Crypto investment research based on tokens:

Operator is an AI agent by ChatGPT that browses the web and performs tasks. Currently in research preview with restrictions, it evolves with feedback. Check out the use cases below: 1. Crypto investment research based on tokens:

Shruti

205,754 views • 1 year ago

Multi-Agent workflows are the future of AI. OpenAI released new Agent APIs today, and Box built an Agent that combines documents from Box and web search tools to generate answers. Enterprise devs can grab sample code from our GitHub repo to customize with their data.

Multi-Agent workflows are the future of AI. OpenAI released new Agent APIs today, and Box built an Agent that combines documents from Box and web search tools to generate answers. Enterprise devs can grab sample code from our GitHub repo to customize with their data.

Aaron Levie

102,060 views • 1 year ago

Kimi 2.6 Agent Swarm is crazy. It deployed 12 parallel research agents, did 200+ web searches, and found me 30 current AI DevRel opportunities at Sarvam AI, Razorpay, Anthropic, NVIDIA, Groq, AWS, MongoDB, GitHub, OpenAI, and more + generated report was also very detailed.

Kimi 2.6 Agent Swarm is crazy. It deployed 12 parallel research agents, did 200+ web searches, and found me 30 current AI DevRel opportunities at Sarvam AI, Razorpay, Anthropic, NVIDIA, Groq, AWS, MongoDB, GitHub, OpenAI, and more + generated report was also very detailed.

AshutoshShrivastava

47,129 views • 1 month ago

opus 4.6 with new “swarm” mode vs. opus 4.6 without it. 2.5x faster + done better. swarms work! and multi-agent tmux view is *genius*. insane claude code update.

opus 4.6 with new “swarm” mode vs. opus 4.6 without it. 2.5x faster + done better. swarms work! and multi-agent tmux view is genius. insane claude code update.

Mckay Wrigley

288,693 views • 4 months ago

OpenClaw can now scrape websites without getting blocked. With the Scrapling plugin, your AI agent can crawl the web and collect data automatically.

OpenClaw can now scrape websites without getting blocked. With the Scrapling plugin, your AI agent can crawl the web and collect data automatically.

Julian Goldie SEO

23,619 views • 3 months ago

Webhound (Webhound) is an AI agent that builds datasets from the web. Instead of spending weeks gathering data, just describe what you need, and Webhound automatically finds, extracts, and organizes it into structured datasets you can export.

Webhound (Webhound) is an AI agent that builds datasets from the web. Instead of spending weeks gathering data, just describe what you need, and Webhound automatically finds, extracts, and organizes it into structured datasets you can export.

Y Combinator

23,264 views • 10 months ago

Spent the weekend exploring the new Coinbase Developer Platform🛡️ AgentKit 0.1.0 release - it is extremely powerful! I put together an open-source template that allows you to create a web-based, multi-chain agent capable of taking 20+ actions. Check out the demo!

Spent the weekend exploring the new Coinbase Developer Platform🛡️ AgentKit 0.1.0 release - it is extremely powerful! I put together an open-source template that allows you to create a web-based, multi-chain agent capable of taking 20+ actions. Check out the demo!

Jason Hedman

23,301 views • 1 year ago