By: Deepti Jain

In November 2006, Google, Yahoo and Microsoft agreed to support the Sitemap protocol. The Sitemap protocol is a standard that can help you to submit all of your web pages to search engines.

Last week, Ask.com joined the team so that now all four major search engines support the Sitemap protocol. In addition, search engines can now automatically detect your Sitemaps file.

What are sitemaps and what is the official Sitemaps protocol?

A Sitemap is an XML file that can be made available on a website and acts as a marker for search engines to crawl certain pages. It is an easy way for webmasters to make their sites more search engine friendly. It does this by conveniently allowing webmasters to list all of their URLs along with optional metadata, such as the last time the page changed, to improve how search engines crawl and index their websites.

That information helps search engines to more intelligently crawl your site. The Sitemaps protocol is a standard that makes it easier to create a sitemap that can be parsed by all search engines.

How do search engines will find the Sitemaps file?

The big four search engines Google, Yahoo, Live/MSN and Ask will now automatically index your Sitemaps file if you add a link to your Sitemaps file in your robots.txt file.

All the major search engines regularly check the robots.txt of a website. Just add the following line to your robots.txt file:

Sitemap: http:/www.yoursite.com/sitemap.xml

Of course, you can also actively inform Google and Yahoo about your Sitemap file. This is not possible with Live/MSN and Ask.

Using the Sitemap protocol does not guarantee that your web pages are included in search engines, but provides hints for web crawlers to do a better job of crawling your site.

Sitemap protocol