I have not had that problem, but I can confirm that DH does limit the number of concurrent connection from a given referrer.
Have you looked at your logs to see if someone is DOSsing you by hammering your site that hard and that repeatedly?
Were you linked by some powerhouse website that is pointing everyone to you?
Is some other site, hotlinking one of your images?
What is the url of you site?