Personally, I find that W3 Total Cache is the secret to “fast” WordPress sites. Essentially, it creates static html versions of every page so your website doesn’t have to manually create them for every visit. That doesn’t help you when you’re trying to post, but it helps your website visitors. You’ll want to set the cache time pretty long. I use a week (604800 seconds). And you’ll want to prime the cache so the site’s generally always ready for visitors. I do that with my own plugin, though W3 Total Cache has a way to do it itself. To do that, you’ll also need to schedule cron from the DreamHost control panel so it gets called frequently–mine is every 5 minutes. Depends on your requirements. My cache primer runs for maximum 10 seconds every 5 minutes so it builds the cache with fairly low server impact. I think the W3 Total Cache option has you set a page fetch limit instead of a time limit.
Between VPS and good caching, I see decent performance.
CloudFlare is another way to pick up some speed. Essentially that provides a static copy off the DreamHost servers at CloudFlare. It takes control of the whole domain’s DNS though, so I haven’t been brave enough to try it.
DreamHost offers WordPress hosting now too, so if you just have the 1 site, it might be reasonable to just do that. That way they take care of the caching and CDN issues and you don’t have to worry about them. They claim fast performance, though I have not tried them. I have too many sites to make it a cost-reasonable option.