12th September 2006

When Was The Last Time You Saw The Googlebot?

By: Jason Lee Miller | Source: webpronews.com

The Google crew cleared up a bit about how the Googlebot caches webpages. The date at the cached page reflects the last time the page was modified.

When the Googlebot drops by a webpage that has not been changed since the last time it was there, it recognizes that the fact and doesn’t bother to download the contents and chew up your bandwidth.

Vanessa Fox explains:

When you look at Google’s cache of a page (for instance, by using the cache: operator or clicking the Cached link under a URL in the search results), you can see the date that Googlebot retrieved that page. Previously, the date we listed for the page’s cache was the date that we last successfully fetched the content of the page. This meant that even if we visited a page very recently, the cache date might be quite a bit older if the page hadn’t changed since the previous visit. This made it difficult for webmasters to use the cache date we display to determine Googlebot’s most recent visit. Consider the following example:

1. Googlebot crawls a page on April 12, 2006.

2. Our cached version of that page notes that "This is G o o g l e’s cache of http://www.example.com/ as retrieved on April 12, 2006 20:02:06 GMT."

3. Periodically, Googlebot checks to see if that page has changed, and each time, receives a Not-Modified response. For instance, on August 27, 2006, Googlebot checks the page, receives a Not-Modified response, and therefore, doesn’t download the contents of the page.

4. On August 28, 2006, our cached version of the page still shows the April 12, 2006 date — the date we last downloaded the page’s contents, even though Googlebot last visited the day before.

Spread the word: readit

This entry was posted on Tuesday, September 12th, 2006 at 1:18 am and is filed under SEO/Search Engine News. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

Leave a Reply

Spread the Word
delicious
digg
technorati
reddit
magnolia
stumbleupon
yahoo
google
  • Subscribe

  • Add to Google
  • Add to My Yahoo!
  • Subscribe with Bloglines
  • Subscribe in NewsGator Online
  • Add to Technorati Favorites!
  • Feedburner Reader
  • Get free E-Book on blogging

  • Online Marketing
  • RSS


eXTReMe Tracker