Traffic and high load?


#1

I have had problems running an image gallery ever since I started getting about thousand visitors a day, all generating a number of over 400,000 server requests per day (which is when my current webhotel asks me to take measures or they suspend my website). Traffic is ~30-50 gigaByte per day… Simply just images being browsed through.

I saw no mentions about the limit of server requests before signing up there, and I haven’t exceeded gigaByte traffic. It’s just that apparently it will harm their other customers if my website generates so much high load on the server.

Since the second time I was warned after looking over all possibilities of reducing the load (i.e disable hotlinking etc.) I have currenty shut down my image gallery to think of a solution. A friend recommended that I check out DreamHost.

I realize that anything near 500,000 server requests per day is quite a lot. Is it too much on DreamHost?


#2

There is no raw limit specified, and it’s very high here.
But your usage is WAY too huge.

Maybe some non-free services dedicated to this (photobuckets ?) are better.


Get [color=#CC0000]$97[/color] Off with promo code :[color=#CC0000]97USA[/color] :cool: -->choose your plan (promo code included)


#3

Yeah, that’s an enviably high load if you’re getting good pageviews rather than just hotlinks.

Have you thought about monetizing the site and using the proceeds to buy dedicated hosting?

Free unique IP and $67 off with promo code [color=#CC0000]FLENSFREEIP67[/color] or use [color=#CC0000]FLENS97[/color] for $97 off. Click here for more options


#4

Thansk for the response.

“Monetizing”. As in advertising? Maybe, but I doubt I could use google-ads, although I’m not sure. The content of the image gallery is humorous absurdity and some of it can seem quite obscene. I don’t know if that works with googleads. While many of the pictures are cute and harmless, some of the images are of nudity and some others are violent.

I am not too sure about the credibility of all the traffic and visitor statistics, but those approximate numbers are what I saw, and due to the situation with my current host I have not been able to experiment or investigate much further.

It annoys me a bit to think that for once people are coming to enjoy the content of my website, and they can’t.

Other alternatives for hosting images such as photobucket? I’m not too fond of it and we’re talking about over currently 3,000 images.

Well… is it possible that the way in which this image gallery is scripted is generating significantly more server load than is necessary? There’s nothing more to it than a visitor coming to the gallery, clicking next, random or previous, through as many of the thousands of images as they wish.


#5

it’s more likely just the raw number of concurrent connections that are the biggest problem, though your gallery software could probably be optimized to some degree, depending upon what you are using.

The shared server is just probably so busy serving your users’ requests, that other users on the server are not able to be properly served.

–rlparker


#6

500,000 dynamic pages per day is a lot. That’s about 5 dynamic pageviews per second, evenly spread out (and I doubt it’s evenly spread out). BTW, doing the math, that’s also 15 million pageviews a month.

Depending on the number of unique visitors represented, if you monetize properly, you could afford dedicated hosting.

One thing you might be able to do to reduce server load is to recode using AJAX or some other dynamic client-side technology and let your server just serve up the jpg content.

Free unique IP and $67 off with promo code [color=#CC0000]FLENSFREEIP67[/color] or use [color=#CC0000]FLENS97[/color] for $97 off. Click here for more options


#7

I don’t know for sure that this is relevant to your situation, but it may be of interest to others anyway.

A little while back, a bit frustrated with the slowness of my dh Gallery, and realising that I hardly used any of its facilities, I decided to automate a static html+image gallery.

This was only for my own and family’s use, so I haven’t gone to the nth degree with research, but basically, the approach I’ve settled on involved:

  • setting up llgal (http://home.gna.org/llgal/) on my dh site
  • configuring a dh cron job to run llgal once a day
  • setting up a cron job on my home machine to sync my laptop’s photos with a dir on my dh account

With those in place, I now have a very simple gallery. The upside: very, very fast, with little drain on the server; entirely automatic (any new photos just appear there with no intervention on my part). The downside: no facilities to speak of (no search, resize, etc); no real authentication (I do use apache basic authentication, but that’s limited).

My site is tiny and personal, so I’ve no idea how this would scale. Certainly for a lot of photo’s the first run of llgal (which uses convert) would take a while. But after that it’s incremental.


#8

You might want to consider optimizing your image gallery. Note that simply downloading an image from the server do not cause high server loads. Therefore by customizing your image gallery in such a way that it is static (by use of static page or cache), I am confident that it will run fine on DreamHost with the huge bandwidth and storage provided. Moreover, your site will load fast with this configuration. Feel free to reply if you are not sure on how to cache the image gallery.

||||||||||||||||||||||||||||||||||||||||||||||||||
Check Out DreamHost Promo Codes


#9

also do try to limit the number of bots that access your site.
They can account for much of the CPU usage.
What software are you using to make you gallery?
is it one of the BIG CMS? gallery2 copermine?


BUGabundo :o)
promo here
50$ discount with promo code “BUG50” on ALL plans
Free lifetime Domain with “BUGDOMAIN” promo code


#10

Remember that he’s said to be currently using 30-50GB a day in bandwidth. This is 0.9-1.5TB a month which is worrisomely close to the default limit of 2TB on a L1 plan.

Also as stated, this is with an interface that shows one picture at a time out of a pool of 3,000 and has a “next” and “random” button.

I think these requirements all combine to make it an interesting and perhaps impossible challenge to host this on shared hosting without tight custom coding. Noisyjazzman’s llgal idea is brilliant and, IMHO, the only hope. Dunno if it has a random component, though. True randomness will be hard to achieve, and fake randomness may fail for returning users in that in most implementations they’d get the same “random” sequence of pictures.

Free unique IP and $67 off with promo code [color=#CC0000]FLENSFREEIP67[/color] or use [color=#CC0000]FLENS97[/color] for $97 off. Click here for more options


#11

Thanks for the link to llgal! I’ve used, and very much like, it’s predecessor, igal for years. :slight_smile:

–rlparker


#12

Yes, I was lucky to stumble on llgal. It was a bit of a headache to get the deps to install on dh, but once running it’s been flawless.


#13

That sounds like a real good subject for a DH wiki article! :wink:

Is there any chance we can talk you into writing up what you did to get it working? If you do that, I’ll “wikify” it for you if you want. :slight_smile:

–rlparker


#14

I’m not sure that I remember, as I did it in one of those late-night blurs some weeks ago, without taking notes. But I’ll have a glance over it at the weekend when I’m (hopefully) less busy, and letcha know here.


#15

He he, I’ve had my share of those “blurry nights”, so I know exactly what you mean! :wink: If the weekend presents some time for that it would be great, and if not, that’s cool too! I’m sure others can research it for themselves if they need to. :slight_smile: Thanks!

–rlparker