Would it make you feel better if I told you it loads quickly for me? Probably not.
When it's running slow, log into your server via SSH and do an 'uptime' command to see the server load. It should be below 5. You may also try browsing through your server logs and look for errors.
There's also the off chance that one of your external sources from quantserve or geotrust are holding things up.
Another option is to try a static html page, or even just the logo to see how quickly that loads. Quicly loading static files would indicate that the slowdown is with the database.