I am in Australia and I host my sites on DreamHost and have not experienced any problems related to the physical location of the server.
The visitors to my websites tend to be fairly evenly distributed between Europe and the USA, so it made sense for me to host in the USA instead of here in Australia. If the majority of my visitors were from Australia, I probably would have hosted my sites locally, due to the reduction in latency (the ping from here to the west coast of the USA is around 190ms on a good day
As for Google search results; I don't think Google uses IP addresses to pinpoint geographical location for search engine placement, I think they tend to rely more on the domain suffix. For example, if you want your site to rank well on the German Google, then I think it is important to use a .de domain and having the site in the German language wouldn't hurt either
Save [color=#CC0000][b]$50[/b][/color] on DreamHost hosting using promo code [color=#CC0000][b]SAVEMONEY[/b][/color] ( Click for promo code details )