< back to knowledge

Robots, NoIndex, NoFollow SEO Best Practices Guide

by Andreas Voniatis Data Scientist

Noindex Tags

Noindex tags disallow search engines from showing a page in their results. Pages with the noindex tag will be read by search engines, but they won’t be indexed. Reasons why one might want to use this meta tag include:

  • advising robots not to index a very large database
  • web pages that are very transitory
  • pages that one wishes to keep slightly more private, printer and mobile-friendly versions of pages

To stopsearch engine web crawlers from including your site page in their search engine index, place the following meta tag into the <head> section of your page:

<meta name="robots" content="noindex">

The alternative method is a HTTP response header:

HTTP/1.1 200 OK
(…)
X-Robots-Tag: noindex
(…)

For the noindex directive to take effect, the URL must not be blocked by a robots.txt file, and it must be otherwise accessible to a crawler. If the page is blocked by a robots.txt or is otherwise inaccessbile to crawlers, then Googlebot and other search engines will never see the noindex directive. Additionally, the page could still appear in search results especially if other pages link to it.

Nofollow tags

The “nofollow” tag provides a way for webmasters to tell search engines “Don’t follow links on this page” or “Don’t follow this specific link”.

Nofollow tags can help you avoid problems such as search engines perceiving sites to be selling influence or somehow involved in schemes deemed as unacceptable SEO practices.

<meta name="robots" content="nofollow">

 

Get Artios Alerts
Contact us
<

    Get Artios SEO Alerts

    For the latest knowledge on how search engines impact your business