Skip navigation
It is becoming f*cking dangerous: Claude Code turns into a web crawler for almost any website, and…
0:00 / 0:41

It is becoming f*cking dangerous: Claude Code turns into a web crawler for almost any website, and…

plusxbt

13K followers

Visit on X
148 likesDownloadOriginal post

10,941 views

It is becoming f*cking dangerous: Claude Code turns into a web crawler for almost any website, and you write zero lines of code 1. Install Microsoft's Playwright CLI (needs Node.js 20+): npm install -g @playwright/cli@latest 2. Run playwright-cli install --skills in your project folder, it adds a browser skill to .claude/skills 3. Open Claude Code in that folder, paste the start URLs and add: use playwright-cli 4. Name the fields you need: titles, prices, links, dates 5. Tell it to follow internal links, skip duplicates and respect robots.txt 6. Open a second terminal tab and run playwright-cli show to watch the browser live 7. Ask Claude to save the results to data.json, then turn the run into a script and test it once so you can rerun it Even Microsoft's Playwright MCP README says coding agents increasingly favor CLI with skills over MCP because it is more token-efficient Full guide in the article below

19 Comments

Comments from the original post on X (Twitter)

  • Mikel Lindsaar's profile picture

    @lindsaar

    Every store owner should point this at their own site first. Ask it for the price and stock on your product pages and see what comes back. Whatever it gets wrong is what agents are already quoting about you.

    1 like

  • Imran Hossen's profile picture

    @uiuximran

    good job

    1 like

  • shobhanai's profile picture

    @HeyShobhan

    Full Respect 🙌

    1 like

  • 小莓的爱车小故事's profile picture

    @TheDevilseye

    Bookmarked this

    1 like

  • 小太阳的汽车日记's profile picture

    @kh2freak12

    backlog pile on my screen outshines the release calendar right now

    1 like

  • Deep's profile picture

    @deeponailabs

    i'd limit the allowed sites, a crawler follows every link it finds

    1 like

  • Nick Sawinyh's profile picture

    @sawinyh

    Test the crawler on an empty result page too.

    1 like

  • Steven's profile picture

    @frogbetsteven

    Web crawlers for casino promos? I already scrape 12 property sites manually. The math says automate or die.

    1 like

  • Cortiq's profile picture

    @0xCortiq

    How to use it for Mac?

    1 like

  • 安叫兽|Bird🕊️ 🔶 BNB's profile picture

    @ajs6888

    最危险的是门槛没了,封禁和合规估计要先跟上了。

    1 like

  • Peter | designing & building with AI's profile picture

    @petersindex

    my CLAUDE.md has an evidence only, no guessing rule. on web work that rule needs something to check against, and a browser claude drives itself is that. snapshots going to disk also keep opus from rereading the whole DOM every turn

    1 like

  • Jonathan 🇺🇲's profile picture

    @thaonlyjonathan

    There's a bug in your system

    1 like

  • John's profile picture

    @frogbetjohn

    Web crawler that finds promo codes? Now that's +EV. What sites you hitting? - DVH

    1 like

  • Bercus's profile picture

    @BercusNA

    How does it handle sites with bot protection or a login? “Almost any website” is where these setups usually fall apart.

    1 like

  • 三分钟懂行's profile picture

    @Rennix7t

    这套流程里,最后保存成可重跑的脚本挺实用。同一批页面下次还要抓,就不用再从头描述一遍字段和规则。

    1 like

  • Josh Hoeg | Vibe Coding & AI's profile picture

    @joshtheaiguy

    zero lines of code right up until the site renames one div

    1 like

  • AiMarket24.pro's profile picture

    @AiMarket24

    Worth adding: respect robots.txt and rate limits, or "zero lines of code" turns into zero lines of code to get your IP blocked. Skills + Playwright is powerful precisely because it behaves like a real browser.

  • Fakrul Hasan Sarker's profile picture

    @fhsethen

    The reason CLI skills beat standard MCP setups is the schema tax. MCP clients re-transmit every tool definition on each turn, burning thousands of tokens before an agent even acts.

  • Vector's profile picture

    @Vectorxeth

    The token-efficiency point is the real story here. MCP dumps the whole accessibility tree into context every step, while CLI + skills lets Claude pull only what it needs. Step 7 is the underrated one too: once the run becomes a script, you stop paying tokens for every rerun