hello@anzum.com

robots.txt FileTo exclude all Robots From The Entire Site

If you don’t want to let your sites get indexed on search engines, you can disable access via robots.txt file.

Create a file named robots.txt and upload it to your root directory of domain.

User-agent: *
Disallow: /

To exclude all robots from certain directories robots.txt:
User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /junk/