Robots.txt Generator

Build a clean robots.txt file with crawl rules, optional sitemap details, and copy-ready output for your site.

Site URL

Sitemap URL

User-agent

Crawl policy

Include sitemap line

Crawl delay seconds

Output file

Upload as /robots.txt

Allow paths

Disallow paths

Enter one path per line. Paths that do not start with / are normalized automatically. Use block-all carefully because it tells crawlers not to index the site.

Policy

Custom

Allow Rules

Disallow Rules

Sitemap

Yes

Interpretation

Generate a robots.txt file to review the crawl policy.

Robots.txt Output

Generate rules to see the robots.txt output.

Ready.