As this is a WordPress site, all of your content is actually stored in your database. What’s in your public directory is WordPress core, themes, plugins, and any images or other files you’ve uploaded through the media gallery. Those files are located in your /wp-content/uploads/ directory. It’s not a bad idea to backup that directory.
Since you’re already on Dreamhost, you might want to consider Ipstenu’s DreamObjects Backup plugin, which will automatically backup your WordPress database and wp-content directory. See here: https://wordpress.org/plugins/dreamobjects/ You need to have a DreamObjects account, but if the only thing you are storing there is backups, the cost is pennies a month, and really worth it for the peace of mind it brings. I can’t recommend that plugin highly enough.