MARZ.tools

Robots.txt Generator

Build a valid robots.txt file with allow/disallow rules and sitemap links.

Loading tool…

About this tool

Generate a robots.txt file by specifying which user agents, paths, and directories to allow or disallow, plus your sitemap URL.

How to use

  1. Add rules for user agents, allowed paths, and disallowed paths.
  2. Add your sitemap URL.
  3. Copy the generated robots.txt content.

FAQ

Does robots.txt block pages from appearing in search results?

Not reliably — it only requests that crawlers not fetch a path. Pages can still be indexed if linked elsewhere. Use a noindex meta tag to actually prevent indexing.

Can I have different rules for different bots?

Yes, add separate rule blocks for specific user agents like Googlebot alongside a default wildcard block.

Related tools

SEO Title CheckerCheck your page title's length and preview how it looks in Google search results.Meta Description OptimizerCheck meta description length and preview it in Google search results.Slug GeneratorConvert any title into a clean, URL-safe slug.