As the rollout of Slurp 3.0 begins, sites using IP-based recognition of crawlers might see a drop in Yahoo activity, the company said.

Yahoo has launched the latest version of its search engine, and notified Web site administrators of some infrastructure changes.

The rollout of Slurp 3.0 officially began Monday and is expected to take several weeks. Among the changes caused by the upgrade would be crawling from a different and much smaller set of IP addresses, but still from the crawl.yahoo.net domain.

As a result, sites using IP-based recognition of crawlers might see a drop in Yahoo activity. “We strongly recommend that you move to reverse DNS-based identification of Yahoo Slurp if you’re using any other method to avoid this problem,” Yahoo developers said in the portal’s search blog. “The current set of IPs will disappear from your Web logs in the next several weeks.”

Yahoo said the crawlers also would publish a new user-agent, Yahoo Slurp/3.0. Existing robots.txt directives for “Slurp” or “Yahoo Slurp” would still work, but the new crawler would not recognize directives specific to “Slurp/2.0.” However, Yahoo said usage of the latter user-agent is “very rare on the Web, so you won’t likely be affected.”

Yahoo has set up a help page for more details. The changes are expected to affect the main Yahoo Web Search crawlers.

Research from Web analysis firm Hitwise show that users of Yahoo search are often younger and spend less money online than people who prefer Google. Web searchers who use the latter are most likely to have spent more than $500 online, according to Hitwise.

Yahoo is battling a $40-plus billion takeover bid by Microsoft. Yahoo has recently taken actions that make it more difficult for Microsoft (NSDQ: MSFT) to succeed. The software maker, however, has said it is prepared to launch a hostile takeover.

Spread the word: related/readit

Source: informationweek