Thursday, September 8, 2011

Google +1 vs robots.txt

The presence of Google +1 button on a page will override robots.txt restrictions for that page. Google +1 button should only be placed with pages with content intended to be public, Google crawler will ignore robots.txt blocks for that page and fetch it.

No comments: