Problem using XML Sitemap Generator on VPS?


#1

Hello,

I just joined Dreamhost a few days ago and I’ve encountered my first serious problem.

I have a starter VPS account.

Standalone Sitemap Generator (PHP) v7.1, 2015-05-19 |
https://www.xml-sitemaps.com/

Been using the above script for close to 8 years and have never encountered a problem until now.

When attempting to create an XML sitemap for any of my domains Under my Dreamhost VPS account the script seeminly starts, then abruptly stops after 5 seconds (Creating sitemap.xml with a single entry, the homepage.)

Now I do have the CloudFlair option active on my domains so that maybe the issue, but I think it has something to do with default server bot/harvester blocking rules stopping this spider from crawling my site?

I’ve tried increasing various allotted resources (php.ini) to see if that was the issue and still it won’t work:

max_execution_time = 600
max_input_time = 600
max_input_vars = 1000
memory_limit = 512M

Can someone tell me how to fix this issue? I’ve emailed Dreamhost support several times with this query and haven’t received one single reply.

Any help would be greatly appreciated.


#2

Interesting tool. I tried it, and was able to download a sitemap without any problems.

Have you tried looking at your server logs to see if they tell you anything?


#3

That sounds plausible to me. CloudFlare attempts to block unauthorized bots from accessing your site, so I can certainly see how they might end up blocking this one as well. Try running the sitemap generator using the direct address for your domain to bypass CloudFlare, e.g. “resolve-to.example.com”?