Give them your password. You're welcome All kidding aside, this is the easiest way, and a suitable alternate to this is to set up a special FTP user before you create the site so you can share that password. In my case, every domain gets its own FTP user, but I mostly do that for housekeeping. In your case, you can give the keys to one domain to someone without them having the keys to your other domains.
But if you want a different user to have access, try these hoops: