Joomla is a CMS.
The difference between CMS and blogs seem to be how much you can do with it, yet you hear people refer to some of them as either one.
If something will only let you make posts to a page, add a few links, etc... it's probably a blog.
If lets you do just about anything, as well as incorporate stuff like forums and other software into the design, it's probably a CMS.
If it's fat, bloated and does more than you need, it's probably a CMS, that you could replace with a lighter piece of blog software.
I just call them all CMS, since no matter how little they do, or how little your site is, whatever you install is what's managing your content.
Personally, I like Wordpress. Between it and all of the themes/styles/plugins out there, it does anything I'd require. It's also the most user-friendly out of the ones that I've tried, plus I like that it spits out valid XHTML.
Check out opensourcecms.com -- they have a bunch of demos installed, so you can try them out before going through the trouble of installing one.
Another advantage to a big name like Wordpress is that with all of the people using it, you probably won't have to look far for help when you need it.
Save $96 at Dreamhost with the 96DOLLARSOFF promo code.