Menu

Post image 1
Post image 2
1 / 2
0

xcrawl-scraper v1.0.1 — Node.js SDK for Web Scraping

DEV Community·Charles·19 days ago
#W1VucYUr
Reading 0:00
15s threshold

Charles

I just released xcrawl-scraper v1.0.1 on npm!

Key Features

  • Scrape any webpage → clean Markdown, JSON, HTML, or text
  • Search the web → structured results with snippets
  • Crawl entire sites → built-in sitemap discovery
  • AI Extraction — describe what you want in plain English, get JSON back
  • Proxy control — choose exit region (US, JP, DE, GB) or sticky sessions

Quick Start

npm install xcrawl-scraper

Enter fullscreen mode Exit fullscreen mode

const { XCrawlScraper } = require('xcrawl-scraper');
const xcrawl = new XCrawlScraper({ apiKey: 'YOUR_API_KEY' });
const result = await xcrawl.scrapeMarkdown('https://example.com');
console.log(result.data.markdown);

Enter fullscreen mode Exit fullscreen mode

GitHub: https://github.com/yanxvdong123/xcrawl-scraper
npm: https://www.npmjs.com/package/xcrawl-scraper

Read More