Dreamhost PS Manager

software development
#1

I have written a Dreamhost API application that dynamically manages the size of the memory of a Dreamhost virtual private server. Having recently moved to a Dreamhost PS myself I initially started building this for my own use, however I am making it freely available to anyone else who may be interested under an open source license.

For more information and to download this application see: Dreamhost PS Manager.

The application resizes the memory of a Dreamhost PS based on actual usage and can optionally allow you to follow the resizing of your Dreamhost PS via Twitter.

If you decide to use or try this application, I’d be happy to receive some feedback as to your experience.

Otto - 7is7.com

#2

Hi,

I just found your application and will try to install it.

Nobody else has given any feedback yet ?

Thanks.

#3

Thanks for trying it out! Do let me know how it works out for you.

BTW. I have just released a new version (version 0.2.3) which contains some minor policy tweaks.

I have had some feedback by e-mail so far, they seemed to like it. Many more downloads, but of course I don’t know if they actually use it or not.

Otto - 7is7.com

#4

Great !

Thank you, it works, but I will test it better during the next days as I have a quite high traffic website during evening.

Would it be difficult to add an option to schedule to set my PS at max from 7pm to midnight everyday ?

Thanks a lot, I was really waiting for a tool as this one, Dreamhost should buy it from you to provide it to users !

#5

Thanks for using PsManager, I am glad you like it!

[quote]Would it be difficult to add an option to schedule to set my PS at max from 7pm to midnight everyday ?

[/quote]

It might be possible to add something to that effect.
With Maximum, do you actually mean 4000 MB (ie. the maximum available)?

BTW If you want me to e-mail you when there is a new version send me an e-mail (find my contact page at the bottom of the webpage about psmanager or any other page on my site).

Otto - 7is7.com

#6

Hello,

Yes, I would need to set up the PS at 4000 MB every evening from 7pm to midnight and I explain why with an example:

Yesterday night more than 30 people/minute were hitting one of my site but your script never set up my ps more than 3466 MB and i got 1500 errors because of that (i have a tracker on error page).

And i am lucky that I already set the factor on your script at 2 instead of 1, because i would have had thousands of errors.

Usually i get few errors when the ps is set up at 4000 MB, but never that much, so i would need to set the server at max every evening but not 24 hours because it’s not useful and the website doesn’t make enough money to pay a full PS full time.

In case you can’t help is there anyway to disable your script automatically during evening but still use it the rest of the time? Any cron command to do that ?

Thanks a lot for everything !

#7

Wow! Those are impressive numbers. You’re the only one I know with such high usage, so I am very interested in your case.

I actually have a change ready that would allow you to do what you want, I wanted to make some more changes before pushing this new version. But since it’s important to you I will push it with just this change asap.

I’ll post back here when it’s done.

Otto - 7is7.com

#8

The new version is available (version 0.2.4): PsManager

There is a new config file with it, that contains a bit of commented out code, supplied as an example.

ADVANCED - Example for time dependant minimum of 500MB.

#{

my ($sec,$min,$hour) = localtime();

if ($hour>=19 && $hour<23) {

$mem_min = 500;

$memory_factor = 1;

}

#}

You can copy this into your config file, and uncomment it, and change the mem_min from 500 to 4000. Also you don’t need to check that the hour is less or equal to 23 (it always is). For your requirements you’d get this:

ADVANCED - Example for time dependant minimum of 500MB.

{
my ($sec,$min,$hour) = localtime();
if ($hour>=19) {
$mem_min = 4000;
$memory_factor = 1;
}
}

Let me know how this goes.

Additional note:

The main cause of the problem you are encountering with the server errors is because you only get 450MB of swap memory. Which is incredibly low. Shared servers run on 4000MB of RAM and get 6000MB of swap memory, we get only 450MB. With only 450MB it’s hard to deal with traffic bursts.

I have entered a suggestion to increase swap memory, but it hasn’t appeared in the list yet. When it does it could use some support from other PS clients to get DH’s attention. I’ll probably post more about this sometime in the next few days.

Otto - 7is7.com

#9

Thank you so much !

So if i want to try and write this:

{
my ($sec,$min,$hour) = localtime();
if ($hour>=19 && $hour<23) {
$mem_min = 2000;
$memory_factor = 1.5;
}
}

It will set the PS to 2000MB minimum from 7pm to 11pm with a factor of 1.5 ?

And what happen if i setup this configuration and then the server needs more memory, it will increase even if time is between 7pm and 11pm? But never go below 2000MB ?

Also, about the time, i need to set the ps at 2000 MB minimum at 7pm French time, should i change something as servers are in America ?

Thanks a lot !

#10

The code you gave would do the following from 19:00 to 23:00 :

  • Take the actual memory usage and multiply that by 1.5
  • Check if that is lower than 2000 if it is set it to 2000.

So yes, If you are using say 1800 MB it will multiply it by 1.5 and set memory at 2700MB (higher than 2000MB). But if you are using 1200 MB it will multiply it by 1.5 which is 1800MB, but since this is lower than 2000 MB it will set it at 2000 MB.

My servers are on california time, I suppose yours are too, which means that you should convert local French time to local California time. However France and the USA start and end Daylight Savings time at different dates. So this it getting a bit more complicated.

It’s probably better to simply change the timezone the script runs in to Central European Time (which is what France uses):

{
$ENV{TZ} = “CET”;
my ($sec,$min,$hour) = localtime();
… etc…

Otto - 7is7.com

#11

It didn’t work tonight with the code i showed you, even though i set the time to California time PDT from 10 to 14 (7pm to 11pm French time).

I am now trying your new tool version without time customization to check if it works as the previous one and will report later.

Thanks for all.

ps: please tell me to vote for your dreamhost suggestion about swap memory when available on vote

#12

That’s strange, maybe a typo somewhere? I’ve of course tested it on my account and am currently still running it with this setting:

{
my ($sec,$min,$hour) = localtime();
if ($hour>=10 && $hour<=22) {
$mem_min = 250;
$memory_factor = 1;
}
}

So I am pretty sure it should work. Be sure to put this at the end of the config file.

[quote]ps: please tell me to vote for your dreamhost suggestion about swap memory when available on vote

[/quote]

It’s still not listed in the suggestions. I wonder how long it will take?

Edit:
I just found out that dreamhost is already working on it!!!
http://discussion.dreamhost.com/showflat.pl?Board=curious&Number=119616

Otto - 7is7.com

#13

Hi again,

I used again the previous version of your tool and it worked, but when i try the new version again it doesn’t change my PS at all even though i use it with the default parameters (without ADVANCED settings).

Any idea about what could happen ?

Thanks !

#14

No idea. It could be anything, maybe the permissions aren’t set right? It’s hard to say without an error message. So let’s try to find out what’s going wrong.

  1. Try to see what gets written in cronjob’s log file.

You entered a job in crontab like this:

          • $HOME/bin/psmanager.pl > /tmp/psmanager_cron.log 2>&1

Now you can do the following on the command line (change the filename to whatever you decided to use as filename):

tail -f /tmp/psmanager_cron.log

if something goes wrong an error message will appear in about 1 minute.

  1. See if the process is running at all:

Supposing you are running psmanager in ~/bin there’s a log file in
~/bin/psmanager.log
do a:
tail ~/bin/psmanager.log
to see if the last values get entered.
A new line should be added every minute.

Otto - 7is7.com

#15

Hi,

No error there: psmanager_cron.log

And psmanager.log is created, one line added every minute, but PS resources still do not change.

Please tell me if you need more information.

Sorry to disturb, I can keep using the first version until someone else meets the same problem as me.

Thank you again.

#16

There are plenty of things we could look at still.

Usually if the resource doesn’t get changed, it’s because it doesn’t need to be changed. However a copy of the last lines of psmanager.log (for the new version) might help to see what is going on.

As for psmanager_cron.log it gets cleared every minute, it’s easy to miss any message, that’s why “tail -f” is required to see it. You can also change the cronjob by using “>>” instead of “>”, then the log file doesn’t get cleared each time:

          • $HOME/bin/psmanager.pl >> /tmp/psmanager_cron.log 2>&1

I am pretty sure, we should be able to get this working.

Otto - 7is7.com

#17

Hi again,

I’m using tail -f to check for errors in psmanager_cron.log and i don’t see any.

I tried the new version again and it’s now working when I don’t use the Advanced function, but if I take off # sign in front of these lines it doesn’t work anymore:

{

my ($sec,$min,$hour) = localtime();

if ($hour>=10 && $hour<14) {

$mem_min = 2000;

$memory_factor = 2;

}

}

(when i tried it again time was not between 10 and 14 US time, so I don’t understand how these lines have any effect ?!)

Thanks.

#18

OK. So we got some progress…

[quote]but if I take off # sign in front of these lines it doesn’t work anymore:

[/quote]

Do you get an error message then? There should be one if it causes the script to stop working.

Something else you can try:
Try adding the Advanced parts little by little to see when it stops working for you:

First step:

{
my ($sec,$min,$hour) = localtime();
}

Second step:

{
my ($sec,$min,$hour) = localtime();
if ($hour>=10 && $hour<14) {
}
}

Last step add the other two missing lines.

Otto - 7is7.com

#19

Incredible !

I installed it completely again tonight and it worked with the advanced part !

Let me test it tomorrow again and I will tell you if it’s really ok now.

Thank you so much for all !

#20

Great news!
Thanks for letting me know.

Mind you, there is one small ‘bug’, that if your current memory usage is not more than 10% lower than a new minimum, it won’t increase it, because it only makes changes of more than 10%. I think this is a relatively minor issue, but I’ll try to fix it sometime, however this will take a bit more time because I am going on vacation.

Otto - 7is7.com