Test Site


I would like to test a website without search engines crawling it. I’m guessing I set up a “test” directory with a robots.txt set to not crawl. Is that accurate?

Do I just create a test folder and upload the site to that?


you could put it in any directory you want to, and yes place the robots.txt file in the directory

** if you need help with robots.txt –

Promotional Code: [color=#CC0000]2DOMAINSFREE[/color] (2 Free Domain Names for the life of your hosting!)


For development purposes you could make a subdomian (eg. test.domain.com)


User-agent: *
Disallow: /

Put it in your test.domain.com folder.

Maximum Cash Discount on any plan with MAXCASH


If I was really worried about robots crawling a site, I’d go with a really obscure subdomain, and maybe even password protect the site.

There’s no guarantee that an unscrupulous crawler will respect the robots.txt file