No you don’t need to keep shared hosting, you can have VPS only without shared hosting. However you do get to keep shared MySQL hosting (important if you plan on using WordPress).
You don’t loose features like one-click installs, but as far as I know you would loose the 1 free domain per year and any other such benefits that you might have received when you signed up.
It’s pretty quick, as I recall it was within a day or so. But yes you would be moved to another physical server.