There are a great many things that impact the speed with which your site loads, and most of them are not server related.
Where's your site (url)? Pings? Traceroutes? What is your server load? What software are using (you mentioned "administering" your site)?
Some times when users report slow speeds, other users are able to get quite good results from the same site, other times the site software itself is responsible for the delays, and in other cases the server itself may be under a heavy load..
Things like inefficient MySQL queries, inefficient site design, tables that need indexing, and other users' activity on your server can be the culprits.
There may be several things you can do to speed things up, and there may be some "tweaks" DreamHost can make to help also - it's just really hard to tell what the situation may be with no more information than you have given.
Providing a URL to your site would be a good place to start, so that others here could visit the site, check what is happening, view code, etc.