RankParse

URL Index

Get crawled URLs for a domain with status codes and MIME types.

GET /v1/url-index

Returns crawled URLs for a domain with status codes and MIME types. Costs 2 credits.

Parameters

ParameterTypeRequiredDescription
domainstringYesThe domain to query
limitintegerNoMax results (default 100, max 1000)
offsetintegerNoResults to skip for pagination

Request

curl "https://api.rankparse.com/v1/url-index?domain=example.com" \
  -H "X-API-Key: rp_your_key"

Response

{
  "data": [
    {
      "url": "https://example.com/",
      "status_code": 200,
      "mime": "text/html",
      "crawled_at": "2026-02-10"
    }
  ],
  "credits_used": 2
}

On this page