Last updated on February 17th, 2022
1. What are robot’s meta tags?
Meta Robots tag also called Robot Meta Directives gives SEO Professionals the authority to control how individual pages should be crawled, indexed, and served to users in search results.
These Robots Meta Directives are of two types:
1. Meta Robots tag
This meta robots tag is a part of the HTML Page Code and should be placed within <head> tags. This Meta Robots tag can be used to control indexing behavior at the page level.
<meta name="robots" content="noimageindex, nofollow, nosnippet">
2. x-robots tag:
This x-robots tag can be used when you need to control the indexation of a page as a whole and to control very specific elements of a webpage. It is included as a part of HTTP header.
x-robots-tag: noindex, nofollow, nosnippet
Every parameter that can be used in a meta robots tag can also be specified in an x–robots–tag.
There is a list of parameters that crawlers agree and follow when they are included in meta robots directives.
List of Indexation-controlling parameters:
· Index: It is a default meta tag that instructs search engines to index a webpage and there is no need of specifying it.
· Noinxex: Instructs crawlers not to index a page.
· Follow: It is a default meta tag and it instructs crawlers to follow all the links and pass link juice and equity to the linked pages
· NoFollow: Instruct crawlers not to follow the links and pass the link equity on this page.
· Noimageindex: Instructs search engine bots not to index all the images on a page.
· None: Similar to the use of both noindex and nofollow attributes at once.
· Noarchive: To avoid displaying of cached version of a page on SERPs.
· Nocache: Similar to how noarchive functions, but this attribute is used specifically for browsers like Firefox and Microsoft Edge.
· nosnippet: Instructs crawlers not to display a snippet of a page i.e meta description on a SERPs.
· Unavailable_after: Instructs crawlers not to index a page after a particular date.