How to setting Robot.txt on your Blog

              The Robots Exclusion Standard, or commonly known as the Robots Exclusion Protocol or robots.txt protocol is a convention to prevent web crawlers cooperate with other web robots to access all or part of a website that when viewed in the open.
Robots are often used search engines to categorize and archive web sites, or web master to correct the source code. From some of the articles I read, penggunanan robots.txt is very useful to boost SEO blog you know.
Suppose there is a page that does not want us to show to the public and search engines so we could utilize the functionality of this robots.txt to block the page. Blogger already support this functionality and this shows that the blogger is getting easier for us to SEO optimization blog.
If not understood well, you can register your blog on google websmaster here. Then the clay on the crawl errors, the function will be performed by robots.txt blocking this.

How To Enable And Setting Robots.txt:

1. Log in to blogger with your blogger account.
2. Click settings> Preferences search / Search preferences.

3. On the tab select robots.txt crawlers and indexing.

4. And enter the following command:

User-agent: Mediapartners-Google

Disallow:

User-agent: *

Disallow: /search

Allow: /

Sitemap: http://yourblogurl.blogspot.com/feeds/posts/default?orderby=UPDATED

Replace the url http://yourblogurl.blogspot.com with your blog url.

Description:

*. The first line group is owned by Google user agent information indicates that this blog is a partner of Google, it is easy for those of you who want to use the services in order to google adsense ads that will be displayed are relevant to your blog content, it also tells us that our blog can be accessed via search engines ,

*. The second row is a group for all machine information search engine crawlers. There is written disallow: / search Yag signifies all the search engine robots to index page should not label. which is useful for middle-duplication article highly disliked by google.

*. The third row is a group that functions facilitate sitemap search engines index your blog

*. The sign "/" is a sign of your home page so do not just put this sign on the column because the resulting disallow search engines can not index your home.

5. Click Save changes.

Setting Robot Header Tags

1. Still in Search settings menu preferences / Preferences searches

2. The lowermost section there is a menu Custom robots header tags. Click the edit.

3. Select yes

4. The settings box will appear, match the setting as shown below

5. Save changes

One more note to be careful to use this function because if any-one even your blog being ignored by search engines.

Tweet