Documentation
Everything you need to build with Scrapify
Guides, API reference, and working code examples to get your scraping pipeline running fast.
Quick Start
Get your first scraping job running in under 5 minutes.
- Creating an account
- Running your first scrape
- Viewing results in dashboard
- Exporting data
API Reference
Full REST API documentation for programmatic access.
- POST /api/scrape-async
- GET /api/job-status
- GET /api/scrape-history
- POST /api/webhooks/scrape-complete
AI Extraction
Configure LLM-powered structured extraction schemas.
- Built-in schemas (Products, Leads)
- Custom JSON schema definition
- Groq model configuration
- Output validation
Scheduled Jobs
Set up automated recurring scrapes with cron syntax.
- Cron expression format
- Managing schedules in dashboard
- Email notifications
- Failure handling
Data Export
Download and integrate your extracted data.
- JSON export format
- CSV schema mapping
- Markdown output
- Direct API download
Chat with Data
Query your scraped data using natural language RAG search.
- How semantic search works
- Chunking & embedding pipeline
- Query best practices
- Limitations & tips
API Reference
Base URL: https://your-app.vercel.app
Trigger a scrape job
curl -X POST https://your-app.vercel.app/api/scrape-async \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_TOKEN" \
-d '{
"url": "https://example.com/products",
"maxDepth": 2,
"maxLinks": 50,
"extractionSchema": "products"
}'Poll job status
curl https://your-app.vercel.app/api/job-status?jobId=JOB_ID \
-H "Authorization: Bearer YOUR_TOKEN"
# Response:
# { "status": "completed", "pagesCount": 48, "recordCount": 125 }| Method | Endpoint | Description |
|---|---|---|
| POST | /api/scrape-async | Queue a new scraping job |
| GET | /api/job-status | Poll job completion status |
| GET | /api/scrape-history | List all historical jobs |
| POST | /api/webhooks/scrape-complete | Receive completed job data (internal) |
| GET | /api/schedule | List active scheduled jobs |
| POST | /api/schedule | Create or update a scheduled job |