Sitecore content tree indexed by Search Engines


Have you noticed that your Sitecore content tree may be indexed by search engines?

Try searching for “/sitecore/content/” in your favourite search engine. When I entered this into Bing the third link that I was shown was:

www.rfu.com

http://www.rfu.com/sitecore/content.aspx

Clicking on the link takes you to a Page Error message on the RFU site as there’s no layout associated with the content item. The url shown in the browser address bar is:

http://www.rfu.com/error?item=%2fsitecore%2fcontent&layout=%7b00000000-0000-0000-0000-000000000000%7d&device=Default

Presumably having content indexed by search engines under two separate url’s on a site isn’t going to be too good for your SEO rankings unless appropriate action is taken.

One way that I found of successfully dealing with this problem was to add the following to the sites robots.txt file:

User-agent: *
Disallow: /Sitecore/

Can anyone offer a good explanation of what is allowing the structure of the Sitecore CMS’s content tree to be indexed by the search engines?

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s