Robots Meta Tag Generator

Build HTML meta robots tags and X-Robots-Tag headers for search engines and social bots.

Target Bot: robotsDirectives: index, follow

Configure crawler directives:

index

Allow search engines to index this page

Follow links on this page

noarchive

Do not cache/archive this page

off

nosnippet

Do not show text snippet in results

off

noimageindex

Do not index images on this page

off

notranslate

Do not offer translation of this page

off

Target bot (name attribute)

HTML meta tag

HTTP header equivalent

Robots meta tags vs robots.txt

The <meta name="robots"> tag tells search engine crawlers how to handle a specific page. Unlike robots.txt (which controls crawling), meta robots tags control indexing and snippet behaviour — they are read after the page is fetched. Use noindex to remove a page from search results, and nofollow to prevent link equity from passing to external links. You can also target individual bots like googlebot or bingbot.

For whole-site crawl rules, use the Robots.txt Generator. For HTTP security headers, see the Security Header Generator.

Built and maintained by Meet Shah · Last updated 19 July 2026

What this tool is used for

Producing a noindex tag for a page that should not appear in results.
Adding nofollow or noarchive directives deliberately rather than by copy-paste.
Getting the header form for a non-HTML file such as a PDF.
Producing per-crawler directives for a specific bot.
Generating a tag to compare against what a page currently emits.

Frequently Asked Questions

What is the difference between noindex and nofollow?: noindex keeps the page out of the index; nofollow tells the crawler not to pass signals through the links on it. They are independent — a page can be indexed with nofollowed links, or excluded while still passing link equity onward.
Why does a noindex page have to stay crawlable?: Because the crawler has to fetch the page to see the tag. Blocking it in robots.txt means the instruction is never read, and a URL blocked from crawling can still be indexed from external links alone. To remove a page, allow the crawl and use noindex.
What do noarchive, nosnippet and noimageindex do?: noarchive suppresses the cached copy, nosnippet suppresses the description text in results, and noimageindex keeps the page's images out of image search. They are refinements of how an INDEXED page appears rather than whether it is indexed.
Can I target one crawler?: Yes — use the bot's token as the name attribute instead of robots: <meta name="googlebot" content="noindex">. A bot-specific tag overrides the generic one for that bot only, which is how a page is hidden from one engine and left in another.
Meta tag or X-Robots-Tag header?: The header does the same job and works on non-HTML resources — a PDF or an image cannot carry a meta tag. Use the meta tag for pages and the header for files, and never both with conflicting values, because the most restrictive one wins.

Common errors and gotchas

Combining noindex with a robots.txt disallow, where the crawler cannot fetch the page to see the noindex.
Applying noindex site-wide from a staging config and shipping it, which deindexes everything.
Expecting a noindex to remove a page instantly, when it takes a recrawl.
Using nofollow on internal links as a crawl-budget tool, which is not how it now behaves.
Setting the tag with JavaScript, where the crawler may never see it.