Purpose of the Meta Robots Tag
The Meta Robots tag allows you to control whether or not you want a particular page to be indexed by the search engines. By default, search engine spiders (crawlers) will try to index every web page of your website and follow links from one page to the next. The best use of the Meta Robots tag is when you want to keep the search engine spiders OUT and not let them index a particular web page.

Guidelines to Writing a Meta Robots Tag
Make certain you double and triple check your code when using this tag. Improperly formatted Meta Robot tags, or their incorrect usage, could keep search engines from indexing your web pages or even your entire website.

If you use robots.txt file to block search engine spiders, you do not need to use the Meta Robots tag. If you want the search engines to index your website, you really don't need to use the tag either, because the spiders will index by default, as that is there purpose, to locate and index pages as a result of following the links from web pages already in their search engine index.

Meta Robots tags can be used on just pages that you do not want spidered or indexed by the search engines.

Always place the Meta Robots tag in the head of your html document.

Example Formatting of a Correct HTML Meta Robots Tag
1. <meta name="robots" content="index,follow">
2. <meta name="robots" content="noindex,follow">
3. <meta name="robots" content="index,nofollow">
4. <meta name="robots" content="noindex,nofollow">

Translation:

1. Index this page and follow all the links to other pages.
2. DO NOT Index this page but follow all the links to other pages.
3. Index this page but DO NOT follow the links to other pages.
4. DO NOT Index this page AND DO NOT follow all the links to other pages.

Note: Never put more than one Meta Robot tag on the same web page or more than one directive in the content area of the tag.