Problem installing Image::EXIF from cpan

I’m trying to install the perl module Image::EXIF using cpan. I get the following error during the make:

I’ve installed libexif to ~/lib, but obviously here /usr/bin/ld can’t find it. I thought this could be got round by setting LD_LIBRARY_PATH to ~/lib, but that doesn’t seem to do anything.

Any ideas?