Any number of problems, from the sublimly trivial to the "deep and dark" can cause this error. Have you looked at your server error logs for hints? It is usually something very simple to fix, especially with a mature product like Joomla.
I don't even know how to respond to that...er, yep..it is the webserver that is experiencing the error, though that is a far cry from saying the "webhost causes this problem". The reality is that the code caused the problem for the webserver
My guess would be that probably a lot of people could give you some suggestions if they had the barest of information.
First of all, I suggest you take a look at how to ask smart questions and then get back to us with a little more info so we can help you:
1) What version of Joomla?
2) Dreamhost "One-Click", or did you do it yourself?
3) PHP4? PHP5? PHP as CGI? (possibly answered by #2 above)
4) Does Joomla run at all? Can you load the "administrator" back-end at all, or does it crash when you try to do a certain thing?
5) How are your permissions set on the Joomla Directories (again, possibly answered in #2 above)
6) and , my favorite, what is the output of phpinfo? (just search these forums for further on how to generate that if it is new to you.)
7) The url of your site?
I strongly suspect that, with a little more information, those here can get you running in no time. Joomla is pretty solid, and I have lots of Joomla's installed around various Dreamhost servers.
Do a little specific research and post again; we can probably help.