Robot Meta tag is used to give instructions to the web spiders. It tells the search engine how to treat the content of a page. It is a piece of code which is incorporated in the “head” section of a webpage.
Some of the main robots Meta tag values or parameters are as follows:
- FOLLOW: This tag instructs the crawler to follow the links on the page.
- NOFOLLOW: This tag instructs the crawler not to follow the links on the page.
- INDEX: This tag instructs the crawler to index the page.
- NOINDEX: This tag is used to instruct the search engine crawler not to index the page.