OK, I got it working to the point that it serves as XHTML to a browser that prefers XHTML over HTML (Webkit), but if HTML comes first in the accept string (Firefox default, Opera), my Content-Type header gets overwritten. It doesn't seem to be a mod_rails issue as it happens with mongrel too. Probably a Rails 2 thing and not Dreamhost specific.
I also found this:
but the proposed solution means not serving text/html as a fallback at all, so it serves the XML view to browsers that don't do XHTML.