Headless Chrome is useful. It is also overused. For years, the default answer to “this page is hard to scrape” has been some version of: Use Puppeteer. Use Playwright. Add stealth. Wait for the page. Extract the DOM. Enter fullscreen mode Exit fullscreen mode That works often enough that it became muscle memory. But using a browser as the first step for every page is expensive, slow, operationally annoying, and frequently unnecessary. I’m building webclaw , a web extraction API, CLI, and MCP server for AI agents. One of the biggest architecture decisions was this: Do not make the browser the default path. Enter fullscreen mode Exit fullscreen mode The browser is an escalation path. Not the baseline. Why Browser-First Scraping Became The Default The web changed. Static HTML became React, Next.js, SPAs, hydration payloads, infinite scroll, client-side routing, consent banners, and heavily instrumented frontend apps. So scrapers adapted.…