Failure Report / Failed Referrer Report?

When looking at the web server stats for my website I see the two reports mentioned in the title of this question…Failure Report and Failed Referrer Report.

My question is why did those failures happen?

Yesterday’s report had one URL listed as having 66 failed requests, yet when I click on that URL it works properly.

Why are those failures happening?

It is some thing the UA (user agent) is doing that fails when they request your files or have your page as a referrer in the UA string. This is usually not a mistake at your end, nor your server. This is usually something that helps protect your web site files from malicious behavior.

The failure can be caused by many different things; a bad request, a method not allowed, something in the UA string that is not allowed, a request from an IP address that is being blocked.

Well…I sure would like to be able get more details from the specific failures being reported.

In most cases a specific URL is being referred to by another page on my website. That referral will work time after time…and then fail.

I’d like to know why a URL works most of the time but fails some of the time. Maybe I could do something to fix it…

Download your raw Access logs and Error Logs:

username > logs > account name > http

Then use a text editor to read them. With the older logs you may need to change the file name to have a “.txt” file extension.

The Error logs will just have the file requests that failed for one reason or another. There you can get the IP address of the UA. Copy that IP address and search the Access log for the actual event to see what happened. Look for response status codes in the 400s or 500s for the errors.

Here’s a digest of what the codes represent: