Facebook can't scrape my site anymore - help?!


#1

I’ve been running a blog for years now, and sharing blog posts (with the correct thumbnail, title and description) on Facebook always worked fine. Last week however, when sharing new blog posts, all the meta data was obviously missing - Facebook didn’t even pick up the title of the page. When debugging the new blog posts, I get:

[quote]Response Code : 503

Missing Required Property: The og:url property is required, but not present.
Missing Required Property: The og:type property is required, but not present.
Missing Required Property: The og:title property is required, but not present.[/quote]
https://developers.facebook.com/tools/debug/og/object?q=http%3A%2F%2Fwww.discodemons.net%2F2012%2F02%2F16%2Fradiohead-hannes-fischer-edit%2F

However, when debugging older posts, everything’s there, Facebook finds all the required meta data:
https://developers.facebook.com/tools/debug/og/object?q=http%3A%2F%2Fwww.discodemons.net%2F2012%2F01%2F31%2Fjustice-daft-punk-helix%2F

When opening my site in the browser and viewing the source code, all the required meta data is there, for both new and old blog posts. Obviously, my server is showing a different version of my site (or no version at all) to the Facebook crawler than it shows in the browser.

This problem came literally over night, I didn’t change anything in my WordPress or server configuration. Any help much appreciated, I’m pretty much at my wit’s end here with this!


#2

[quote=“discodemons, post:1, topic:57074”]When opening my site in the browser and viewing the source code, all the required meta data is there, for both new and old blog posts. Obviously, my server is showing a different version of my site (or no version at all) to the Facebook crawler than it shows in the browser.

This problem came literally over night, I didn’t change anything in my WordPress or server configuration. Any help much appreciated, I’m pretty much at my wit’s end here with this!
[/quote]

Pay attention to the error messages and use a search engine.

Response Code: 503
Bad Response Code: URL returned a bad HTTP response code.

"10.5.4 503 Service Unavailable

The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.

  Note: The existence of the 503 status code does not imply that a
  server must use it when becoming overloaded. Some servers may wish
  to simply refuse the connection."

There is no mention of when or how often they atempted to access the URL, I suggest you check your access.log and error.log files in case this particular error is getting logged.


#3

I am having the same error on my site today:

I checked some other sites i have all on the same server and got the same issue. There is an open support ticket in facebook as well (that i did not open) addressing the issue:

http://developers.facebook.com/bugs/279743805428206

It has something to do with the communication between facebooks servers and dreamhosts, not user error on site owners end, which is why the sites are getting 503 on previously fine pages.

What really stinks is I use facebook comments on my site and thoes are also affected. I put in a support request with dreamhost noonish today but have yet to hear back. I really hope this gets resolved soon.


#4

Everything works fine again now for me, even though I didn’t change anything the past few days. Very weird…


#5

We accidentally deployed an overly protective mod_security rule. Should be fixed for most users now.


#6

Hi Andrew F,
I am a web server admin and I have this problem to my website. Is it possible to tell me detail of solution about mod_security please?

Thank you