How to Fix your Broken Rails App on DreamHost
I'm composing and posting this brief guide because when I uploaded my working Rails app to DreamHost last week, I found that it wouldn't run. It ran (and still runs) fine on my development boxes (Linux, MacOS X, Windows) but it failed to start on DreamHost. I followed the DreamHost wiki's Ruby on Rails page when I transferred my app to DH, but the instructions on that page were not sufficient to get my app running. There are a number of posts about this problem in the DH wiki and forums, as well as other posts on ruby-lang.org. It is probably best to have read at least the DH wiki's RoR page before trying to use this guide.
DreamHost uses Apache as their webserver. The (somewhat limited) reading I've done on the subject suggests that getting Rails and fastcgi to play nice with Apache is a difficult and confusing task. Most authors of howtos about getting a Rails app set up for production deployment say that using Lighttpd or nginx or WEBrick is a better idea, if one has a choice. That's not very helpful for those of us who haven't the choice.
First of all, if everything is working properly, the script dispatch.fcgi (in <railsapp>/public) will kick off the entire app when a connection is received by Apache at the appropriate URL. It expects to receive some parameters from Apache, so if you run it on the commandline, it will run and then die, and it will put some error messages in its crash log. Since it should always receive data from Apache, the error messages are confusing and they look like part of Rails is breaking: part of the (f)cgi hierarchy attempts to perform 'split' on a string which is nil, because the required input was not provided to dispatch.fcgi
However, there are many reasons why dispatch.fcgi may be invoked without the necessary input, even when it is invoked by Apache. First, therefore, it is necessary to determine whether dispatch.fcgi is capable of starting the Rails app in the first place. I recommend running the script from the command line. If there are unmet dependencies, or if for any other reason your Rails app doesn't manage to run, you will see descriptive error messages in the terminal where you ran dispatch.fcgi. Figure out what these errors mean, if there are some, and fix them.
In my case, I discovered that certain gems which my application requires were not installed on DH. I followed the DH wiki's instructions in order to compile and install my own private version of Ruby, and install my own gems as well. If your Rails app has similar unmet dependencies, you may have to do the same thing. I recommend this post by Vivek which refers to two separate pages within the DH wiki, rather than trying to puzzle out how to setup your own ruby and gems from the wiki alone, which can be somewhat confusing. I also recommend that you review the entire procedure before you start, so that you'll understand why you are making whatever changes you make.
If you do have to install your own ruby and/or gems, you will have to adjust various paths. (This is outlined in the article I referenced above, as well as in the DH wiki.) You should make the adjustments in .bash_profile or in a file sourced by .bash_profile. You may also have to explicitly adjust the gems path in environment.rb:
[b]ENV['GEM_PATH'] = '/home/[i]foo[/i]/.gems:/usr/lib/ruby/gems/1.8'[/b]
although in my particular case this turned out not to be necessary. If you want your private version of ruby to be used to execute your application you will have to adjust the shebang at the top of dispatch.fcgi (and you may wish to keep dispatch.cgi and dispatch.rb up-to-date with that change). In my case, I found that #!/usr/bin/env ruby worked just fine. Others have reported that the full path to the appropriate ruby executable was required. This depends on your execution path—your private bin directories should come first:
thereby ensuring that unqualified executables will be sought first in your private locations, and only if they are not found there will they be executed from the defaults installed on DreamHost servers.
Once your dependencies are met, dispatch.fcgi should execute and die without making any output in the terminal. I have seen other posts saying that it generates a 500 error and prints that in the terminal: I have never seen this happen, but if it did I would interpret that to mean that the script was at least able to initialize the application. I would expect the predictable error in fastcgi.crash.log containing the errors related to calling 'split' on a nil. Another way to ensure that your application's dependencies are met is to run ruby script/console from the toplevel directory of your Rails app. If it loads and shows no errors then the app is capable of running.
Should your app be able to run in console mode but not execute when accessed from the web, the remaining errors are related to Apache and its ability (or inability) to launch the app on the basis of its own requirements. It needs to be able to write in the log directory, and it also needs to be able to write in the various subdirs of the app's tmp directory. The last fix I performed, which finally allowed my app to run when accessed over the web, was to chmod all those directories to mode 766 (all perms for owner, and read+write for all others). Apache doesn't need to cd to those directories, so execute permission is not required.
I say these errors are related to Apache because they don't seem to be an issue when using Mongrel to serve the app. Given that Apache runs my app as the user which owns it (set up that way in the DH panel), and that Mongrel does the same because I run it as myself, I infer that the permissions requirements on <railsapp>/log and <railsapp>/tmp/* are Apache's requirements. If anyone knows differently, please contribute by sharing your knowledge!
Unfortunately, as is often the case in systems administration and troubleshooting, it is hard to boil all this down to a simple step by step process. Nevertheless, I will try to provide such a procedure:
- Check to make sure that your Rails app is capable of running by using <railsapp>/script/console to make sure that it can load and operate on your DreamHost server account. You will need to have SSH access set up so you can log in to your DH server in order to do this.
- If your app can run when invoked with the console script, make sure that dispatch.fcgi uses the same ruby interpreter and loads the same gems as the console script. Check the shebang line in dispatch.fcgi against the output of which ruby. If you're using #!/usr/bin/env ruby, try executing env ruby -v in order to see which version of ruby is executing—yours should be different from the default on DH (otherwise why have your own?) so it should be obvious which one is being invoked. If need be, run ruby -d dispatch.fcgi and watch the paths of the gems which get loaded.
- If your app makes noise on STDOUT while it is running, this may cause it to break when run as a CGI program. Remove or comment out your 'puts' statements.
- If your app seems to be using the right interpreter and loading the right gems but still won't run through the webserver, check your permissions on <railsapp>/log and <railsapp>/tmp/*
- If your app starts, produces correct output to your web browser, and then dies while you're trying to use it, then it is possible that you are running afoul of the various process-limitations DH imposes upon applications on the shared hosts. They impose a 200MB RAM limitation for the total of all processes being run by any given user. If you are running more than one Rails app, you can circumvent this limitation by creating a separate user for each app. If you are only running one app and it is routinely growing past that 200MB boundary, you will need to look into alternatives to DH shared hosting: possibly DH's emerging virtual private hosting service, or a dedicated hosting provider. DH recommends Hosting.com.
- Depending on your app, you may be able to mitigate the reaping of your dispatch.fcgi processes by using Todd Huss's technique, as outlined in these two articles.
I hope this collection of various solution strategies for non-working Rails apps on DreamHost will be useful to other users who find themselves scratching their heads (or ripping out their hair) because their operational Rails apps won't run on DreamHost. If I have made errors I welcome correction. I also encourage other users who have had similar problems to add their observations and solution methods.