Free tools · SEO · no sign-up

robots.txt & sitemap check

Enter a domain; robots.txt syntax is checked and every sitemap is fetched: URL counts, lastmod coverage, size limits and broken XML.

How to read it

  • robots.txt: unknown directives are ignored; an empty User-agent line voids the group; a Sitemap line is the easiest way to let engines find the map. Google ignores Crawl-delay. 'Disallow: /' closes the whole site.
  • Limits: 50,000 URLs and 50 MB uncompressed per sitemap; split with an index when you exceed them.
  • lastmod: write the real change date only; Google ignores maps that say 'today' every day.
  • HTML returned: the sitemap address returns a page (often a 404 page that answers 200); the content type should be XML.
  • For an index, at most 20 child sitemaps are fetched; the URL total is computed from those children.

Other tools (15)

Have this run every week, automatically

The seops Index and IndexNow module schedules this check for every site you run and tells you when something changes. One site is free.