Ok, you've got two options. You can delete the files with FTP, or you can install the forum in an directory - ie. you'd set the install place to be youdomain.com/forum
You might find the k-base helpful for more into about FTP.
First to use FTP (File Transfer Protocol) you'll need an FTP client. If you're in windows I like filezilla I think the K-base reccomends cutFTP. Next you'll need to put in the details to connect to your domain. just make a new connection, or use the quick connect you'll find going across the screen near the top. for ftp address just fill in yourdomain.com the user name and password will be the user you set up with the domain, not the one you use to log into the panel. (if you need to double check, log into the panel > Domains > Manage > click edit across from the domain in question. the usre selected to run CGI as is the one you need to log in with)
So, fill in the user/pass and click connect. if you've got everythign filled in right, you'll get a listing of folders that looks something like logs, maildir, yourDomain.com. double click on youDomain.com to open that folder. Now select and delete all of the files here. that will clear out oscommerce. (do what I said above to delete the database)
If you get an error about the directory not being empty ( I don't think you will using filezilla) you'll need to find the setting in your FTP client to show hidden files - once that's ticked try deleting them again.
Search the forum for more posts about how to set up an FTP client - There's bunches.
Hope this helps...