Right now I have both setup (roundcube.dreamhost.com and my own installation) and they both seem about the same speed.
The load on mariner says: load average: 1.44, 1.95, 1.88
I try to keep the inbox relatively low on mail so it's not bad. It's just the subfolders that have up to 500+ messages.
I've tried Entourage and Outlook Express as mail clients and they both seem normal (ie not slow) once they are initially synced.