Add Custom Robots .txt file and header tags to Blogger

Tweet

You know that it is a SEO training Section and in this post i tell you about Custom Robots. Robots.txt system is a best way to tell search engines whether they are allow to index a page in the search results or not. The bots are automated and before they access to your site, they check the robots.txt file to make sure whether they are allowed to crawl this page or not. Sometime, people do not want to stop searching engine from crawling their whole website. Therefore, today i will tell you a perfect method about Robots.txt file and Robot Header tags that how to create and add these features your blog. You have to only follow my instructions to make your blog perfect to Search Engines.

1. What is Custom robots.txt file and how to create and add it to your Blog:

Custom robots.txt: This option provides you the ability to edit your whole robots.txt file. You can just type your content whether you want the content to be crawled by spiders or not. However, You can always undo your actions and can always revert back to normal. 

How to Add custom robots.txt to Blogger

i). Go to Blogger Dashboard> Settings> Search Preferences

ii). Now Click on Edit "Custom robots.txt"

iii). Click on "Yes" to Edit this function

iv). Now Add the below set of Code
                                                                 Code View

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://tnfiles.blogspot.com/feeds/posts/default?orderby=UPDATED

Now replace http://tnfiles.blogspot.com with your blog address and save it, you done it.

Note: Be careful about editing robots.txt. Any mistake can bring penalty for your blog indexing and crawling.

2. What is Custom Robots Header tags and how to create and add it to your Blog:

Custom robots header tags: This option is a bit completed. It does not provide you the ability to write your codes instead it provides few options with the check boxes. So, if have no idea about head tags then stay away from it.

How to Add custom Robots Header tags to Blogger

i). Go to Blogger Dashboard> Settings> Search Preferences

ii). Now Click on Edit "Custom robots header tags"

iii). And follow the below image and put the mark like the blow image