Sitecore content tree indexed by Search Engines

Have you noticed that your Sitecore content tree may be indexed by search engines?

Try searching for “/sitecore/content/” in your favourite search engine. When I entered this into Bing the third link that I was shown was:

Clicking on the link takes you to a Page Error message on the RFU site as there’s no layout associated with the content item. The url shown in the browser address bar is:

Presumably having content indexed by search engines under two separate url’s on a site isn’t going to be too good for your SEO rankings unless appropriate action is taken.

One way that I found of successfully dealing with this problem was to add the following to the sites robots.txt file:

User-agent: *
Disallow: /Sitecore/

Can anyone offer a good explanation of what is allowing the structure of the Sitecore CMS’s content tree to be indexed by the search engines?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s