Origin: https://developer.mozilla.org
robots.txt: https://developer.mozilla.org/robots.txt (status 200, present True)
Bytes: 119 · Crawl-delay: None · Sitemaps: 1
State: present
| Agent | Decision |
|---|---|
| * | allowed |
| gptbot | allowed |
| claudebot | allowed |
| googlebot | allowed |
| perplexitybot | allowed |
| ccbot | allowed |
A robots.txt that could not be read is reported unknown, never allowed. Generated at 2026-10-02T08:18:18Z, dataset schema 1.0.1, seed version 1.
Full dataset: /v1/agents/policy (JSON) or /v1/agents/policy?format=csv (CSV). /check · every sampled origin: /policy.
Want more than this page? https://wodan-posture.app.goedvps.com/v1/agents/policy/origin?url=https%3A%2F%2Fdeveloper.mozilla.org - paid, $0.03 per call: every group, every rule with its line number, which group governs each well-known agent and why, sitemaps and two digests.