AI Bot Access Tester
Test which AI crawlers your robots.txt allows or blocks. Supports all major bots and RFC 9309 wildcards.
DISALLOWED
Matched agent: GPTBot
Matched rule: Disallow: /
How robots.txt AI bot control works
Websites use robots.txt to tell crawlers which pages they may or may not access. AI training crawlers like GPTBot, ClaudeBot, and Google-Extended respect these rules. A Disallow: /under a bot's User-agent blocks all access; an empty Disallow or Allow: / permits it. The most specific matching rule wins per RFC 9309.
Private & free — this tool runs entirely in your browser.
Recommended: Namecheap — Register a domain for your next project — from $1.98/yr.affiliate
Related Web & SEO tools
Meta Tag Generator
Build SEO title, description, and viewport meta tags.
Open Graph Generator
Generate Open Graph and Twitter Card meta tags.
Robots.txt Generator
Build a robots.txt file with per-bot rules. Block AI crawlers, set crawl delays, define sitemaps.
UTM Campaign URL Builder
Append UTM parameters to build trackable campaign URLs.
Hreflang Tag Generator
Generate hreflang link tags for multilingual SEO.
Sitemap XML Generator
Build an XML sitemap from a list of URLs.
SERP Snippet Preview
Preview how your page title and meta description appear in Google search.
Meta Tag Analyzer
Paste a page's HTML head to audit its title, meta, and social tags.