You're absolutely right that it has to do with directories. In particular, it has to do with what the browser considers your "current directory" to be based on the URL, which will naturally have a big influence on how relative links work.
The "current directory" for http://www.silkrooster.com/links.php/ is http://www.silkrooster.com/links.php/
The "current directory" for http://www.silkrooster.com/links.php is http://www.silkrooster.com/
Your images, scripts, etc. are in folders underneath http://www.silkrooster.com/ but not also in folders underneath http://www.silkrooster.com/links.php/ which is why your site breaks with the slash but works just fine without it.
In terms of a solution, two ideas come to mind. First, you could simply not use the slash, and try to eliminate any references that might be out there in AdSense or wherever to the slashed version of your site's address. Second, you could put on every one of your pages; the tag forces the browser to treat whatever it specifies as the "current directory" regardless of what the URL says.