robots.txt FileTo exclude all Robots From The Entire Site

by | Feb 12, 2015

If you don’t want to let your sites get indexed on search engines, you can disable access via robots.txt file.

Create a file named robots.txt and upload it to your root directory of domain.

User-agent: *
Disallow: /

To exclude all robots from certain directories robots.txt:
User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /junk/

About Marshall
Marshall is creative head at Elicus and works with the team to bring ideas to life. He is strongly focused on delivering a quality experience to clients and customers.