Origin: https://stackoverflow.com
robots.txt: https://stackoverflow.com/robots.txt (status 418, present False)
Bytes: None · Crawl-delay: None · Sitemaps: 0
State: unrestricted_4xx
robots.txt answered HTTP 418; RFC 9309 2.2.1 treats any 4xx other than 429 as no restriction for a crawler
| Agent | Decision |
|---|---|
| * | allowed |
| gptbot | allowed |
| claudebot | allowed |
| googlebot | allowed |
| perplexitybot | allowed |
| ccbot | allowed |
A robots.txt that could not be read is reported unknown, never allowed. Generated at 2026-10-02T08:18:18Z, dataset schema 1.0.1, seed version 1.
Full dataset: /v1/agents/policy (JSON) or /v1/agents/policy?format=csv (CSV). /check · every sampled origin: /policy.
Want more than this page? https://wodan-posture.app.goedvps.com/v1/agents/policy/origin?url=https%3A%2F%2Fstackoverflow.com - paid, $0.03 per call: every group, every rule with its line number, which group governs each well-known agent and why, sitemaps and two digests.