Discover a site's XML sitemaps (via robots.txt Sitemap: directives, an explicit sitemap URL, or the conventional /sitemap.xml) and parse them into a bounded list of URLs with their lastmod, changefreq and priority. Follows sitemap-index files to their child sitemaps. Fetches and URL counts are capped (with a `truncated` flag) to stay fast and memory-bounded. The reliable way for an agent to enumerate a site's pages.
# 1. Ask the endpoint what it costs (no payment, no wallet needed): curl -i -X GET 'https://web.openverbs.com/v1/sitemap' # -> HTTP/1.1 402 Payment Required # the response carries the price, asset and pay-to address. # 2. Pay and retry with any x402 client: npx x402-fetch 'https://web.openverbs.com/v1/sitemap'
Endpoint: https://web.openverbs.com/v1/sitemap. Operated by web.openverbs.com, not by Animica. Price and availability were correct at the last probe on 2026-10-01 and are set by the operator, who may change them.
Fetch a URL (following redirects) and return the final response's HTTP headers, the server banner and content-type, plus a security-header audit reporting HSTS,…
Follow a URL's redirect chain (301/302/303/307/308) to its final destination and return every hop with its status and Location. Unwraps link shorteners and cloa…
Fetch a URL and pull out its machine-readable structured data: all JSON-LD blocks (schema.org), Open Graph and Twitter Card tags, standard named meta tags, and …