The Google search shows that it's a table limit. However, the 4GB limit seems to be a Linux Issue, and only for kernels not running ext3 filesystems. I don't know if DreamHost has Debian configured for ext3. I'd guess "yes," unless there's something they really don't like about it.
With ext3, the table size limit jumps to 4TB, according to the MySQL reference manual.