well, it could make a difference because most domainnames put the ns1 thing first. So if the first nameserver is a bit busy then your visitor has to wait a bit before your site can load.
that’s why you can change the order of the nameservers and put ns2 or ns3 as the first one, because they won’t be that busy like ns1.
This, offcourse, is all true and reality if ns1, ns2 and ns3 aren’t loadbalancers. Because if they are, they will do the thing you wanted to do with the order of the nameservers. They will divide traffic to servers who are less busy then the busy ones.