There are too many things that can affect performance of a site, ranging from applications running on your computer to a slowdown at your DSL provider. It could just be something in one of the many systems that transfer data between DH and your machine. It could also be that someone is doing something heavy on your server, it could be that the database is getting pounded by someone else.
That's just some of many of the variables that can affect performance. Hanging out in the dreamhost forums is showing 1.0 - 3.0 second page refresh times right now.