I do host a video sharing website-type on Dreamhost. So my experience could eventually help.
Dreamhost is a fairly good host in general. Now, if you receive a high amount of visit (from digg.com for example), your website will display your video less fast than on a "normal day", because there is a lot of request on the server and on a particular video. But that will happen with most of the shared-hosting company.
One of the (many) advantage to host with Dreamhost if you have a video website is that they allocate a large amount of broadband.
I also highly recommend you to encode your video in flv (http://en.wikipedia.org/wiki/Flv)
If you want a more elaborated answer I need to know: What is the format of the video you are going to encode in FLV ? Are you coding your website?
Hope this help.
Get $55 discount: use this code DHDISCOUNT55 when you sign up with Dreamhost for one year.