503 & 524 errors - anyone have any info?


#1

Hi.

For the last 24 hours, my site keeps going down with 503 errors. Seems like, right before it stops resolving in the browser, I keep getting 524 - “Server closed the connection without sending any data” - errors. I can still ping the domain, though.

First thing I did was reboot the router and modem at my end. When that didn’t work, I rebooted the computer then rebooted the router and modem again. I’ve cleared the cache and so on. Still no dice.

I walk away for a bit, check the site on my phone, see it’s back up, and sit back down to get to work, only to have the issue return within 15 minutes.

Put in a ticket to support last night, but no response. Couldn’t find a way to update the original request with additional information, couldn’t, and submitted a second ticket like some kind of impatient kid.

DH Status has no updates about Fever, only mention of VPS maintenance this weekend. I’m not on a VPS as far as I know, so I’m a bit lost.

Anyone have any ideas what’s going on? It’s not like Dreamhost to leave me hanging like this. Usually, I get a response pretty quick.

Thanks in advance.

Brian


#2

The network’s probably toast. In which data center is your VPS located?


#3

Pretty sure I’m west coast, mate. Not sure. Hosted on Fever.

I suspected as much, but surprised to see nothing about it on DH Status & not hear back from support yet. My experience with DH had always been better than this. :frowning:

Thank you for the feedback, by the way.


#4

You can see what datacenter your servers are located here in your panel (“fever” is here in Los Angeles, CA):

https://panel.dreamhost.com/index.cgi?tree=status.dc&

You said you submitted tickets? What are your tickets’ tracking numbers (should start with a 5)? Let me know and I’ll check on them for you. Thanks!
[hr]
I think I found your tickets! Regarding gearboxmagazine.com? One of our techs just emailed you - the Apache service on your web server ‘fever’ was having some trouble, so he rebooted the Apache service and your sites are all loading without error now. If the problem happens again, please reply to his email with details and we’ll investigate further. Thanks so much for being patient, Brian!


#5

Thank you so much, Elle! Looks like I’m back I’m business! Appreciate your looking into things personally.

Brian
Happy DreamHost Customer


#6

No problem at all, Brian! Happy to help. :slight_smile: If it happens again, feel free to submit another ticket… or you can use our Live Chat service (just click the little ‘chat live’ icon in your panel), or even tweet us @dreamhostcare if you have a Twitter account. :slight_smile:

Cheers!


#7

Hey Elle.

Hate to be “that guy,” but unfortunately, the problem came right back after less than 10 minutes. Same exact symptoms. I’ve replied to the email from Randall M, but figured I’d mention it here, too.

Really appreciate your help all the same. I know we’ll get this resolved.

B


#8

Request a server move mate :wink:


#9

I can do that? If anyone at DH is listening right now, I think I’d like to do that, please. Replied to email: no response. Pinged supprt on Twitter: no response. Site is unusable at this point. Please help. Thank you.


#10

Yep! Explanation on why it can lead to a satisfying result is in this thread: https://discussion.dreamhost.com/thread-137572.html


#11

DR1665 - What account are you using to try and request this? I opened the account that uses the same email you have on the forums, and you haven’t asked us anything since August 2012. This may be the issue. If you’re using some random email, we may not be able to verify you’re you (social engineering is such a hassle…) - What domain is this for?

– Edited to add this.

I need more sleep again. Ahem. Found the account and the tickets and read through all of them. So the wp-login messages you saw, causing your 503, was part of that massive DDoS we had. The others, though, I noticed had a lot to do with images. It may be nothing at all, but it’s possible some idiot is hotlinking your images. You can block that with .htaccess - http://perishablepress.com/creating-the-ultimate-htaccess-anti-hotlinking-strategy/ is my favorite method - and that won’t have any impact on your site or Facebook (the block doesn’t affect how FB pulls in images). You may want to give that a shot.


#12

Hi there. Appreciate all the Dreamhosters who have touched this issue. I know you’re doing your best. :slight_smile: Unfortunately, the site is still all but useless for me even now. Today is pretty much day 5 of this.

Of course, now I’m a bit confused. I thought I had unlimited storage and bandwidth on this account. Sure, I know there has to be a limit somewhere, but I’m curious how hotlinking could be the problem. The error logs I saw in the latest response from Support suggest that there’s a 30 connection limit. How does that work? I can move an unlimited amount of bandwidth, but only via 30 people at a time? O_o

I’m also not inclined to buy the hotlinking theory. If there were that many people hotlinking the site, wouldn’t there be a gradual slowing down of site performance over time before the bottom would fall out? I haven’t noticed anything of the sort. And wouldn’t such linking show up in my Google Analytics? (This is where we reach the limits of my technical knowhow!) Besides, the images reported in the log excerpt I got are icons for plugins - not the sort of thing I suspect anyone would expect to draw a large amount of bandwidth, ya know? :slight_smile:

I’ve done some htaccess mods in the past, but don’t have time to do much more than copy>pasta the first entry on that page you linked above until the weekend. You can understand my reluctance to just fire up FileZilla and have at it. I tried some hasty copy>pasta earlier this year and let’s just say I’m glad I have a backup service in place.

So what’s the next step. Sxi suggested requesting a server move. Should we pursue that or is there some other definitive solution to this? I remember a few years back, when someone at DH pulled a Michael Bolton, misplaced a decimal point, and everyone with a credit card on file got the pinch for an entire year’s hosting, leading to a lot of down-for-non-payment sites the next day. I laughed at that. This is far more frustrating.

Again, sincerely appreciate the support on this one. Just looking to once and for all fix the problem. It’s taken me the better part of five days now to do 1.5 blog posts in Wordpress. I’m getting super-duper anxious. :stuck_out_tongue:

Thanks.


#13

Hey Brian!

First of all, truly thanks for your patience and cooperation with this so far. I know it’s terribly frustrating, and we really appreciate your patience with our investigation throughout the last few days. I had my support lead take another look at your account, after our Ipstenu mentioned this in another related forum post today.

After doing some digging using that post’s info, my lead disabled Pagespeed for you to test. We monitored your site for a few hours and it’s been lightning fast! Hopefully it stays that way. :slight_smile:

Please read my lead’s email, and see how it goes with Pagespeed disabled. If you run into issues again, reply to the email directly and we’ll do some more digging and see if we can find anything else. Thanks again for your patience, Brian!


#14

This might appear scathing, so I’ll just say up front that this isn’t a dig at Support - they’re on the front line and cop all the end-user flack for the generals who are screwing things up in the background. The support guys have the unenviable task of being the face of DreamHost that gets shouted at and ridiculed even when it’s not their fault that things aren’t working and they’re personally doing everything they can to help. There are some that are unquestionably inept, but they appear to be the minority.

The 503’s weren’t caused by a DDoS as has been inferred, and if PageSpeed (which I agree is absolutely terrible in it’s present beta form and should be turned off) was the only thing at fault then you would not have seen a reversion to poor performance after a 10 minute period of better results - it would have reverted to it’s poor state much quicker, probably instantaneously, and in any case would have appeared instantaneous from a user point of view.

I’d still be inclined to request a server move. It costs you nothing and the moves appear to be handled by people who actually look at what they’re doing rather than a just dumping stuff across the farms and only worrying about the consequences when someone complains.

Yeah! I remember Josh Jones (co-founder and CEO at the time) taking the rap for having “fat fingers” and making a typo that caused that accounting debacle. That was back in the day when if stuff went wrong people put their hand up and openly admitted they screwed up. This transparency, sadly, left DreamHost the day Josh relinquished the helm - and despite all the promises there has been no hint that it will return.

As much as I like DreamHost and will continue to use them for the unimportant sites I have here, if your site is going to be of any importance to you then another suggestion I’d make would be that you move the site to a different host. You will likely be met with the same lack-luster performance, support queues and overselling woes, but will at least be on a server where the system itself is sane, or at least somewhat predictable.


#15

A bit scathing, sXi, to be sure. I’m going to stick it out a while longer, but I can’t afford to have this kind of poor site performance lingering like this. As much as I hate to say it, changing hosts has crossed my mind lately. :frowning:

Just responded to another DH support team member. Today is 1 week the problem has persisted. As requested, I have created a temporary WP admin account and provided Support with logon credentials to see if they can’t replicate the issue. Latest issue tracking number is 5776533.

I need this resolved today so I can make up for lost time this weekend. Thank you.


#16

Bump. After nearly 2 weeks, it seemed like we finally got things resolved. I don’t know if it was related to a DDoS, a tired Apache box, or a couple leftover WP plugins, but the site seemed to work great for about a week. Then, this past Monday, it just went down hard as I was working on it. No 324. No 503. Just down.

Sent in a ticket. Site seemed fine by dinner time. Never did get anything in my inbox. Strange. Sadly, this afternoon, the 324s and 503s came right back again. When I went to submit yet another ticket (5795264), I saw a comment that my issue Monday was another Apache reboot.

C’mon guys. What is going on? It’s been 5 hours now. I hate to be one of those guys, but I’m 3+ years into this project, on the cusp of things really taking off, and NOW my site is going to play amateur nite? This dude has been abiding for nearly a month, now. If I have to consolidate domains and accounts and move to a VPS, fine. Somebody just tell me - definitively - what is going on and how we fix it for good?

I just pinged Media Temple about moving everything. I don’t want to do that. I like Dreamhost. I believe in Dreamhost. I just can’t believe this still isn’t resolved. If my site is doing something wrong, I need to know what so I can handle it, but everything I Google says 503s are host-side. If we have to reboot Apache this often, something is wrong.

Please. Thank you.


#17

Don’t fall for the VPS line unless your website is suffering from RAM issues.

Ipstenu hinted in another thread that there was some “upgrading” happening behind the scenes. I don’t know if she was referring to hardware upgrades or if they’re perhaps fixing the systemic problems and rolling out image updates across the servers. If it is the latter then that might solve your current woes - although an ETA is anyone’s guess. I’d ping AndrewF or Ipstenu - they’re the goto guys for under the hood info here. If AndrewF already has something cooked up then he might make changes to your server specifically to see if things improve.


#18

Thanks for the solid advice. No news of RAM issues. I’ll pass on the VPN.

Pinged Andrew. Fingers crossed. Next move is to MediaTemple, I guess.

(sigh)


#19

For the past 10 hours my site has all the sudden began to have these errors too…from 504 to 500 disconnect errors… they’re all on Dreamhosts end and this is not being discussed on their status… The site mostly has these errors when I’m logged into it. Just wanted to put out there that these issues are coming up for me all the sudden, and it’s only on one of my websites… I submitted a ticket. I wonder if they’ll actually do anything.


#20

They always do something. And I’ve never got the feeling they didn’t care. Working in a data center myself, it’s pretty much a given that you do what you can to fix the problem once and for all because, if you don’t, it’s going to come back and bite you when you’re in the middle of something else.

Ain’t nobody got time for that.