Skip to Main Content

Robots Meta Tag: Proper Use

Posted on 3.05.2007
The next issue of Website Magazine will focus on all thing "search engine optimization" (so subscribe now if you have not already) but we thought we'd start out the day peaking at the some of the content we'll be discussing within the contents. One of the most discussed, least understood aspects of SEO (search engine optimization) is how to properly use the robots meta tag. Low and behold, today at the official Google Webmaster Central Blog, Google's Vanessa Fox discusses how Googlebot handles these important tags.

Here's a quick summary, read the post in detail at the Webmaster Central Blog:

Multiple Content Values
- Google recommends that you place all content values in one meta tag, i.e. <meta name="ROBOTS" content="NOINDEX, NOFOLLOW" /> ; If presented on two lines Google will simply aggregate the tags. If content values conflict (use noindex and index in different tags), Google will use the most restrictive - the noindex.

Unnecessary content values
It's almost comical that they have to mention this, but Google by default will follow pages without being told. That means there is no reason to tag pages with the index or follow content values.

Directing a robots meta tag specifically at Googlebot
It's pretty simple to notify the Googlebot of what to do - just change the meta name from
Casing and spacing
Rarely discussed in the industry, Googlebot understands any combination of lowercase and uppercase in meta tags. If you have multiple content values, i.e.

Leave Your Comment

Login to Comment

Become a Member

Not already a part of our community?
Sign up to participate in the discussion. It's free and quick.

Sign Up


Leave a comment
    Load more comments
    New code