I want to create a static mirror of a dynamically generated website, using wget. My site has about 2,000 pages, and I’m concerned about putting an undue load on the DH server when wget spiders its way through the site. I don’t want to annoy other shared users on my server.
Does anyone have any experience with this? Do you recommend using the -w (“wait”) option to slow down wget? What’s a reasonable wait value?