url | :string (required) | The URL to scrape. | Always required. |
formats | {:list, :any} | Output formats (e.g. ["markdown", "html"]). | Always — controls what data you get back. |
headers | :any | Custom HTTP headers as a map. | When you need cookies, user-agent, etc. |
include_tags | {:list, :string} | HTML tags to include. | When you only want specific tags. |
exclude_tags | {:list, :string} | HTML tags to exclude. | When filtering out specific tags. |
only_main_content | :boolean | Strip nav, footer, boilerplate. | When you want clean content only. |
timeout | :integer | Timeout in milliseconds. | When the default is too short. |
wait_for | :integer | Wait time in ms after page load. | When the page needs JS rendering time. |
mobile | :boolean | Emulate mobile device. | For mobile-specific content. |
parsers | {:list, :any} | Parser configs (e.g. PDF). | When scraping PDF files. |
actions | {:list, :any} | Pre-scrape browser actions. | When you need to interact before scraping. |
location | :keyword_list | Geolocation settings. | For geo-targeted content. |
skip_tls_verification | :boolean | Skip TLS verification. | For sites with invalid certs. |
remove_base64_images | :boolean | Remove base64 images. | For smaller output. |
block_ads | :boolean | Block ads. | When ads interfere with extraction. |
proxy | {:in, [:basic, :enhanced, :auto]} | Proxy type. | For anti-bot protection. |
max_age | :integer | Max cache age. | When cached data is acceptable. |
min_age | :integer | Min cache age. | When you only want cached data. |
store_in_cache | :boolean | Cache the result. | Set false for data protection. |
lockdown | :boolean | Serve from cache only. | For ZDR-like behavior. |
redact_pii | :boolean | Redact PII from output. | When output may contain personal info. |
zero_data_retention | :boolean | Enable zero data retention. | For ZDR compliance. |
profile | :keyword_list | Persistent browser profile. | When you need state to persist. |
audit_metadata | :keyword_list | SIEM metadata (keys: username). | Enterprise audit logging. |