Skip to content

How it works

  1. Browse

    Search and filter the published bot catalogue by name, category, country, or blocked prevalence.

  2. Select

    Add individual bots or the whole result page to your selection list.

  3. Generate

    Create robots.txt, Cloudflare WAF, or Apache .htaccess output, then copy it into your configuration.

Go to the generator Full help: outputs, caveats, and resources

Before you apply rules

  • robots.txt is voluntary — many bots ignore it.
  • Cloudflare WAF and .htaccess can enforce blocks; review carefully to avoid locking out legitimate traffic.
  • Always review generated output before deploying. See Terms and the Help guide.

Compare robots.txt, WAF, and .htaccess →