How to Fix Robots Txt File In Blogger and WordPress


Hey, hope you are fine in this Quarantine. If you have a website and the website is hosted on blogger or WordPress, you got the Robot txt error on the website through the mail. If you got his kind of mail, you need to fix it. First, you can cause critical SEO that can also negatively impact your ranking and traffic.


As you can see on the ScreenShot where you can see Fix Coverage issues, this section of Google Console finds some errors Robot txt, crawler issue,404 error, and 505 error today; we will be discussing the robots txt blogger error. In this post, you will learn what a robot txt file is, the importance of a robot txt file, why robot txt shows, and why you need it.

If you want to fix this, read this article till the end, and I will tell you How to fix the robot txt file in blogger and WordPress step by step with the use of screenshots, so Let’s start.

 What is Robot.txt

Robot.txt is a file in the form of words and symbols. That resides in the root directory of your website and gives search engine crawlers instructions as to which page they are indexing; crawl while during the indexing and crawlings.

When it’s visiting your website, the first thing they do is check out the website content. Also, check your labels of websites and check and read the URL for indexing the particular websites.

Importance of blogger robots txt file.

Robot.txt file is important because it helps your website with indexing, and crawling and also helps to SEO and ranking of the article.Robot.txt helps your website find your page publicly, and a particular website can be crawled and added to its index.

 Suppose you have a big website where crawling and indexing can be very resource-intensive. Crawlers from various search engines will try to crawl and index the whole website, which can be serious crate problems.

Robots Txt File In Blogger

Why does robot.txt show in Google Console

It shows the Robots.txt file in Google console coverage when we do not update the robot’s text file on hosting site websites like Blogger and WordPress.In the case of a blogger, we used labels while posting the article while we added labels on the post. If we added the same labels on the single post, you get this type of problem.

 We used the tags and category in the case of word press; if we used the same tags, you could get this type of problem.When we use the same labels, tags, and category on a single article, when the crawler is indexing them and get the three labels to have a single article, you get the duplicate content error. So it’s very important to fix it because it will impact your SEO.


 Two Important things you should know about Robot.txt

  1.  The first thing is that adding rules to the robot.txt is the only directive. It means that SEO Search Engine obeys and follows the rules according to a robot txt file.
  2. The second thing is if you added the if you block any page on the robot.txt, that would be generated and removed or not appear on the web.

How robots.txt works

Robot.txt file is the form of a simple keywords structure. In which some predefined keywords and value combinations you can use 

 Allow, Disallow, Crawler Delay, Sitemap, and User-agent.

 Allow:- You can use the allow on your robot.txt file it gives access to a specific folder on our website, and even the parent directory is disallowed.

 For example, If you disallow access to the photos directory but allow you to be located, your blog subfolders are located in subfolders.

 User-agent: *

Disallow: /search

Disallow: /category/

Disallow: /tag/

Allow: /

 User-Agent:* It helps the specific crawler take the directive’s specific account. You can use * it specify the crawler name, as you can see below.

 User-Agent:* this allows crawlers. 

User-Agent: this is for the Google bot Only.

 Disallow:- it directive that instructs the user agent, not any URL

 Sitemap:- It’s the direct major search engine, including Google. It’s very helpful to specify the find the location of the XML file.

 The search will be able to find the cause of the XML sitemap. It will be to the bottom of the robot.txt file.


If want to add site for blogger so you click on below link:-

Sitemap for blogger

 How to Fix Blogger Robots txt

Step:-1 First of login to your blogger, which has the Robot.Txt error, and clicks on your blogger’s Setting.

 As you can see on the screenshot and follow me by step by step

  Setting > Crawlers and indexing

Step:-2 Now you can see the bottom of Crawlers and indexing has Custom robot.txt now enable the  1 Enable custom robots.txt and now Go to google and Denrate the custom eobots.txt file. Click ≻ Blogger site map


Now generating code is a copy on the arrow 2 section and save the section.

 Stwp:-3 Now enable the Custom robots header tag and at active all the buttons as according to screen show as you see on the picture.

 In Home page tags

all     enable

no dp   enable  



In Archive and search page tags

Noindex enable

nodp enable

 Inpost and tags

all enable

nodp enable


Now click on the save and close the blogger.


Step:- 4 Now open the Google console on your browser and click on the coverage on the google console.

See the screenshot I marked by the arrow; your screen also shows like that index through blocked by the robots.txt, so click on this error.

  After clicking on this screenshot, your screen will be shown like seen on the bottom screenshot. Now click on start validation, and your error will be remove after some days from the coverage. It will be fix within 15 days after the fix; you will be getting Gmail from the Google Console.


Robots Txt File In WordPress

In WordPress, we need to install one plugin, YOST SEO installs the Yoast SEO.After installing it, open the Yoast SEO, and then your laptop screen will be sho like as below screenshot.  ↡↡↡

 Now you can see three options are shown on the screen Category, Tag, and format; there are some changes in the three options as you see in the pictures.

 In category ➡️ NO

In Tag ➡️ NO

In Format ➡️ Disable

 After active, these options save the sitting, and these fills are fixed after some days.

Also Read:-

How to add a responsive table for blogger

How to add Table of Contents in Blogger/Blogspot?

How to Add Reaction Bar Buttons in Blogger and WordPress

How to Add Contact Form in blogger

Free Privacy Policy For Website

 Note: If you don’t save all these sitting, you must save all these settings. If you don’t do this can impact your read carefully this Robots Txt File In Blogger and WordPress. 

Thanks for the reading; please drop the comments if not understand anything; we’ll help within 8hr and share the comment in the comment section about Robots Txt File In Blogger and WordPress.

Exit mobile version