Skip to content
crawlkit

API Reference

Base URL

Section titled “Base URL”
http://localhost:4000

Override with --port:

Terminal window
crawlkit-api --port 8080

Authentication

Section titled “Authentication”

All endpoints require the X-API-Key header:

X-API-Key: <your-api-key>

Generate keys via POST /api/v1/keys. Default key ck_dev_default_key_for_testing is for local development only.

Endpoints

Section titled “Endpoints”

Health Check

Section titled “Health Check”
GET /health

Response:

{
  "status": "ok",
  "version": "0.4.0"
}

Start Crawl

Section titled “Start Crawl”
POST /api/v1/crawls

Request body:

{
  "start_url": "https://example.com",
  "max_pages": 50,
  "request_delay_ms": 500,
  "concurrency": 4
}

Response (202 Accepted):

{
  "crawl_id": "uuid",
  "status": "running",
  "message": "Crawl started successfully"
}

Get Crawl Status

Section titled “Get Crawl Status”
GET /api/v1/crawls/{crawl_id}

Get Crawl Statistics

Section titled “Get Crawl Statistics”
GET /api/v1/crawls/{crawl_id}/stats

List Crawls

Section titled “List Crawls”
GET /api/v1/crawls

Create API Key

Section titled “Create API Key”
POST /api/v1/keys

Request body:

{
  "name": "my-app",
  "requests_per_minute": 60
}

List API Keys

Section titled “List API Keys”
GET /api/v1/keys

Rate Limiting

Section titled “Rate Limiting”

Each API key has a configurable requests_per_minute limit. Exceeding the limit returns HTTP 429 Too Many Requests.