After getting my Wiki problem solved, and it was really well I have no complaints about that at all to be clear, I started to get all kinds of 500 errors on our site.
Now, I have been told before that it is because our site was using too much resources for shared servers. I understand that, and so after talking about this for a while, we decide to move our sites to VPS.
I hadn’t expected however that our site is using well over 1GB of memory to run most of the time.
What’s more confusing is that if it was in the past, wouldn’t the site be down constantly? It wasn’t.
The only part of the site that caused a problem was the WP part of it. The Wiki and Forum and the Gallery would work every time without fail.
The one thing that bugs me, and my husband, is that we have only one week to sort this all out before we get hit with a massive bill for the server costs.
Now I know that DH staff can’t look at problems 24/7. I know that they have a lot of other people to look after. I do.
I’d like someone to tell us some way to fix this mess, sooner rather than later if they could.
Besides using cache on everything, cutting WP to the bone, and removing as many extra things as possible to drop the load, and it isn’t going down at all… What else can you do?
Sorry for writing this, we just haven’t got a clue what to do…
I apologize for the confusion that has come from this situation. That’s never our intention and we want all of our customers to understand what they are paying for and how adjust it and improve it.
Unfortunately, I am unable to give you the specifics of what is causing this for your site since I am unable to bring up your account. If you could provide me with the domain associated with the account or account number I’d be more then happy to investigate.
Also, upgrading to a VPS will handle traffic better and 1,000 hits a day a well within a VPS limit (even on its most basic) there may be another reason for the 500 error. Now it still might be the VPS require some configuration on our end but like I said I am unable to verify that.
You mention that only some parts of your sites would go down but the forums and other sections would not. That is because the url hitting the high memory limits are the ones being taken down.
There are many reasons as to why this can be happening.
I’ll be awaiting your response so that I can further investigate.
Again, I apologize for all the trouble.
I would like to apologize for all the confusion you have received from our techs.
We’ve been having quite a few issues with some server migrations we’ve been performing and unfortunately your servers are being affected.
It’s not due to the VPS not being able to handle the traffic your site is receiving but because of the servers having a high load due to the migrations.
This can easily get confused on our end and unfortunately that is what happened on our replies to you via your tickets.
I am going to watch over your ticket and account and escalate it and see if there is anything we do for you to stabilize the server you’re on.
We’ll continue to update you via email just so that we can release more information without having it out in the public here.
Again, I sincerely apologize for the confusion.
A blog doesn’t mean WordPress. I have a blog, for example, and it might look a little like WordPress to the untrained eye, but it’s really all static HTML files created on my local computer by a version of Jekyll called Octopress. That means that all the pages are generated once on my machine, then the results are uploaded to DH. The amount of CPU time it takes to serve a static HTML page is so low that DH doesn’t even measure it in your resource reporting logs.
The number you quote above might be referring to some comments I’ve made about one of my sites. It’s not a blog. 90% of those 250k hits/month, which doesn’t happen every month, are Ajax requests. These do require more CPU time than a static HTML page, but the amount is an order of magnitude less than a singe WordPress page. I’ve also worked very hard to cache enough from the database so that each unique visitor may result in perhaps 3-5 database requests for every 150 page views (which include said Ajax requests). All the information that user may need from the database is requested and cached early on in one connection, unlike WordPress.
Also, my heaviest page results in 20 total requests for images (some sprites) and JS and CSS (minified and combined where possible). Subsequent requests are often just a single request with no further assets needed. The static assets are all on another domain and behind CloudFlare. A single WordPress page can, and often does, result in 60-100 total requests for image-heavy sites or just plane unoptimised sites and are usually served from the same user. So that user could be serving 50+ static files at the same time as it’s requesting data from the database and rendering the page. Do that for two or three unique visitors at a time and you could easily run out of memory (the limit is ~100MB) and/or processes (the limit is 25) and/or concurrent connections (I think the limit is 20).
A wiki can be very resource intensive. Each page may require a database call and if users are editing, I could see how even a single wiki on shared hosting with 2k unique visitors per day could run out of resources.
The short response is: you can’t just install a few one-click apps and get 250k requests / month. It isn’t going to happen. To do that, you’ll need to install your own custom apps, or highly optimised versions of existing apps, and spend a significant amount of time endlessly tweaking and benchmarking as noted here.
For most people, it’s easier, faster, and perhaps more economical to just pay more for more resources.
Someone is requesting a wiki page (30MB), so now you have 70MB available for any other requests at the same moment.
Someone requests a gallery page at the same time (33MB), so now you have 37MB available for any other requests at the same moment.
Someone requests a WordPress page at the same time (40MB), or, if you are unlucky, logs into the Dashboard (65MB) at the same time.
Result, WordPress fails because there’s no memory left.
You do have each app running in it’s own user, right? You may be able to squeeze a bit more performance out of your shared hosting that way.
Also, based on what I wrote above, you should not just refer to your apps as a blog, wiki, and gallery. You should refer to them by the actual app name. Is your blog an Octopress blog? If so and it was going down, that would be perplexing. WordPress going down does not surprise me at all. Not one bit.
Wordpress, phpBB, MediaWiki, Coppermine are what we use. I understand that the more you have running the worse it is. What I cannot understand is how the entire thing worked well enough in shared to have the blog only with issues and now it takes well over 1 GB to support the same thing in VPS?
At this point I have to allow DH to do their thing, we haven’t heard back as yet, and the only other thing we can do is shift things around between shared and VPS and see what makes some kind of sense…
I’ve never used a VPS, but from what I understand, you have to cover the OS/Apache memory requirements as well, not just your apps unlike shared hosting where all you have to worry about is your apps’ total footprint.
If it worked ‘well enough’ on shared, then why did you move to VPS? You mention that you’ve been ‘told before’ that you were consuming too many resources, so clearly things weren’t working ‘well enough’.
If you’ve retained your old shared account, move the forum and gallery back on to it completely, set up caching that creates static HTML pages for the WP and Wiki, and move their assets into a sub-domain hosted on the shared account.
bobocat, the numbers you’re giving there are totally comparing apples and oranges. At least one of them (Coppermine) is talking about PHP memory limits, not system memory at all, and another one (phpBB) is referring to the memory requirements of cPanel, not phpBB.