An honest comparison of 8 hosted, OSS, and SaaS tools — pricing, MCP support, free tiers, and which one to pick for your custom GPT, Claude Project, or RAG pipeline. I have built fourteen custom GPTs in the last three months. Every one of them needed a web crawler to turn somebody's docs site into a clean knowledge file. The first knowledge file took me four hours, mostly fighting Playwright on my laptop at eleven pm. The last one took ninety seconds, mostly waiting for an Apify run to finish. In between, I tried twelve different web crawlers. Some are excellent. Some are punitively expensive. One is so good and so free I could not believe it was still online. This is the honest field report. Why this category matters in 2026 Custom GPTs, Claude Projects, and RAG pipelines all share the same bottleneck: the knowledge file . The model is rarely the problem. The prompt is rarely the problem.…