Use the headers parameter to send custom HTTP headers and cookies with your request. This is useful when the target website requires specific headers (like User-Agent, Referer, or Cookie) to return the expected content.
Use carefully with anti-bot protected sites - invalid or mismatched headers may cause blocks
If you override critical headers (e.g. Accept-Encoding), it may impact response parsing
When to Use
- Bypass geo or device-based content blocks by setting a
User-Agent
- Simulate logged-in sessions using
Cookie headers
- Add custom
Authorization headers to access gated content
- Force-language or locale-specific versions of a page (e.g.
Accept-Language: fr-FR)
The headers parameter is a JSON object with header names and values.
{
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)",
"Referer": "https://example.com/",
"Cookie": "session_id=abc123"
}
Example Request
curl --request POST \
--url 'https://api.hasdata.com/scrape/web' \
--header 'Content-Type: application/json' \
--header 'x-api-key: <your-api-key>' \
--data '{
"url": "https://example.com/dashboard",
"headers": {
"User-Agent": "Mozilla/5.0",
"Cookie": "auth_token=xyz789"
},
"outputFormat": ["html"]
}'
Notes
- All custom headers are passed as-is to the target page
- To send cookies, use the
Cookie header — multiple cookies should be in standard format: "key=value; key2=value2"