By: Jason Lee Miller

Crashing right after launch is, apparently, a startup rite of passage. If, however, you’ve touted your new search engine as a Google killer, you might want to make sure crashes can’t happen. Google never goes down, and quite simply, can’t be killed with overloaded servers.

After Powerset’s sudden sale to Microsoft, the blogosphere needed a new contender. A former Google search architect and her Stanford professor husband, along with other former Googlers operating under the protective wing of the anti-noncompete laws of California (a law, ironically, Google likes to leverage when it can), thought for sure they could provide that new challenger.

And then all went blank at Cuil (cool), which was touted to have thrice the index of Google, scanning 121 billion web pages. Servers today couldn’t keep up with demand, illustrating what Powerset foresaw as their biggest hurdle: scalability. Microsoft provided that, along with enough cash to see it through. Even if you could get a query to return something today, though, reviews of the results have been mixed.

The results are supposed to be an alternative to Google’s ranking system, which is often criticized for being more of a popularity contest (among a myriad other criticisms) in the search results. Hence all the Wikipedia and YouTube returns.

Cuil is said to operate differently from Google’s distributed server, load-balancing concept—which incidently handles about a trillion URLs several times daily and manages to stay online—and has its servers divided according to category. If one searches for a sports-related query, for example, there are designated sports servers to handle that. One issue, as we’re seeing today: If a spike in sports queries knocks the sports servers offline, other non-specialized servers specializing in, say, cooking, will handle the results instead.

Spread the word: related/readit

Read the rest of this entry »