Robots.txt is a content record which contains few lines of basic code. It is saved money on the site or blog’s server which train the web crawlers to how to file and creep your blog in the list items. That implies you can confine any website page on your blog from web crawlers with the goal that it can’t get listed in web search tools like your blog names page, your demo page or whatever other pages that are not as critical to get filed. Keep in mind forget that pursuit crawlers check the robots.txt document before creeping any page.
Each blog hosted on blogger has its default robots.txt file which is something look like this:
User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://example.blogspot.com/feeds/posts/default?orderby=UPDATED
Explanation
-
- User-agent: Mediapartners-Google
-
- User-agent: *
Disallow: /search
https://www.blogspotpoint.com/search/label/SEO
Must Read: How To Add Twiiter Widgets In Blogger Blog
Disallow Particular Post
Now suppose if we want to exclude a particular post from indexing then we can add below lines in the code.
Disallow: /yyyy/mm/post-url.html
Disallow: /2013/03/post-url.html
Disallow Particular Page
Disallow: /p/page-url.html
-
- Sitemap: http://example.blogspot.com/feeds/posts/default?orderby=UPDATED
Must Check: Complete SEO + HTML5 Meta Tags Pack 2017 For Blogger Template
Sitemap: http://example.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500
Sitemap: http://example.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500
Sitemap: http://example.blogspot.com/atom.xml?redirect=false&start-index=500&max-results=1000
How To Add Custom Robots.Txt to Blogger
- Go to your blogger blog.
- Navigate to Settings >> Search Preferences ›› Crawlers and indexing ›› Custom robots.txt ›› Edit ›› Yes
- Now paste your robots.txt file code in the box.
- Click on Save Changes button.
- You are done!
How to Check Your Robots.txt File?
https://www.blogspotpoint.com/robots.txt
Once you visit the robots.txt file URL you will see the entire code which you are using in your custom robots.txt file. See below image.
This was the present finish instructional exercise on the best way to include custom robots.txt record in blogger. I truly attempt with my heart to make this instructional exercise as basic and useful as would be prudent. Yet at the same time in the event that you have and uncertainty or inquiry at that point don’t hesitate to ask me. Try not to put any code in your custom robots.txt settings without thinking about it. Essentially ask to me to determine your questions. I’ll let you know everything in detail. Much appreciated folks to peruse this instructional exercise. On the off chance that you like it at that point please underpins me to spread my words by sharing this post on your online networking profiles. Glad Blogging!