The presence of Google +1 button on a page will override robots.txt restrictions for that page. Google +1 button should only be placed with pages with content intended to be public, Google crawler will ignore robots.txt blocks for that page and fetch it.
No comments:
Post a Comment