If google bot is your primary visitor and your website is inefficient enough to be using 250 cpu minutes from just the google bot we will put up a robots.txt. If you put up your own robots.txt to exclude your 16MB photo gallery that is resizing the images on the fly then we won’t create one for you, because we won’t notice you pushing the load on your machine high enough to make others complain.
So basically if you are always watching and optimizing your site and being a good neighbor, you are in the clear. But most people don’t know how to take care of their sites, so we do it for them to keep the machines stable. The trade-off being that a couple robot.txts are better then disabling the problem users or making them upgrade to a dedicated server or making them learn how to take care of their site if they don’t already know how.
If we did place a robots.txt on your site you are welcome to update it to better suit your site, since you obviously know what should be skipped better then we do.
If we did place a robots.txt on your site the policy is that you would be notified. Since it seems that some are claiming that a robots.txt was placed on their site without notification, I will issue an internal memo reiterating the policy.