You most likely used the one click installer to install wordpress on your domain. Yes, that makes your domain live on the internet right away. The one click combines a few manual steps like FTPing your files, and setting some configuration options.
FTP will allow you to view and change those files, most commonly with wordpress user will want to upload 3rd party theme’s and plugin’s for wordpress.
Don’t want users to be able to access your site? use the htaccess/webdav tab in the panel to add password protection to your domain. You can remove it later, when you are ready to “go live” (FYI, Don’t check the WEBDAV box on that page.)