Now it is making sense! You are connecting directly to the Dreamhost server rather than using your domain name, and the user you are using does not appear to be authorized shell access.
I have mentioned this several times; you must have your user enabled for shell access to connect via ssh (or telnet). If your user is an FTP only user, that will cause you connection to be refused if you try to connect via ssh, ssh2, or telnet even if you use the correct username and password for your (ftp) user.
Goto the Control Panel, Manage users screen and look at the user name you are using to login. Under the "TYPE" column, make sure that user is indicated as being a "shell" user, not an "ftp" user.
If the user is not indicated as being a "shell" user, click the "edit" link to the right of the "Disk Storage" column for the user name, and select the check box for "shell" in the next screen, and save the settings.
After a short wait, the user's status will be changed and you will be able to login via ssh.