By : Nidhi Gupta

Once upon a time, on the World Wide Web, all URLs were fixed strings — static in form. The idea of URL parameters then came along, allowing for database driven sites and session ids in URLs to create personalized experiences for users. At that time, the Web was alive with rich data and experiences. Then came the crawlers, which made it easier for users to navigate through the Web; however, they inevitably battled with dynamic URL parameters and every webmaster had to choose between a dynamic site and search traffic.

Today comes a new wave for search engines with the first-ever Beta launch of ‘Dynamic URL Rewriting’ in Site Explorer. The new feature provides the ability for site owners to alert Yahoo! of the dynamic parameters in URLs that they’d like Yahoo! to ignore, which we’ll then automatically rewrite accordingly. Try this out for all the cases where you’d want to use parameters in your URLs that don’t affect the content of your page, but that have other important uses.

How to get there?

• Login to Site Explorer from Yahoo! Search.
• Add to My Sites and then authenticate any sites that you own or manage.
• For any sites that you have authenticated, you’ll see a ‘Dynamic URLs’ tab.
• On this tab you can enter parameters you want us to either remove from URLs or always crawl with a specific value.
• Once you enter the parameter, we’ll show you the # of URLs we estimate will be affected.
• After you confirm the action, we’ll modify our crawler such that every time we see a URL from your site with that parameter, we’ll automatically rewrite it within our system as per your instruction.

So you might wonder what the feature really gives you. Utilizing the ‘Dynamic URL Rewriting’ feature enables:

• A more efficient crawl of your site, with fewer duplicate URLs being crawled.
• Better and deeper site coverage, as Yahoo! will be able to use its crawler capacity to find and index more new content on your site.
• More unique content discovered, as Yahoo! will handle more dynamic parameters in your URLs (if you remove the content-neutral dynamic parameters).
• Fewer chances of crawler traps, or web page sets that can cause an infinite number of requests or a poorly constructed crawler to crash.
• Cleaner and easier-to-read URLs displayed in the search results.
• Better site ranking due to reduced fragmentation of links and anchor text to your site’s pages.

Spread the word: related/bookmark it/readit