Yes it is possible. What happens to all your mail really depends on how you are using POP3, and how you configure your new mail client versus your old mail client.
If your client is currently taking all of your mail off of the server, not leaving a copy behind, then IMAP won’t have any messages to begin with. If your client is leaving a copy on the server, then you will still have mail there for IMAP to see. Potentially there will be mail from a long time back you didn’t even know you still had around (again depending on your config of your pop client)
I use Thuderbird for my mail for here for the most part. What I did to test some of this is I left my imap connection to the mailbox installed, and setup a pop setup. I can copy mail between them. So if your mailbox is empty, you may be able to put important mail back on the server if you want it there.
You may also just keep the important mail in local folders within your mail application.
find / offer local services free