Search

Yahoo Slurps Somewhere Else

0 views

The migration of the Slurp is complete, says Yahoo. Over the past few weeks, the search engine has been transitioning its crawler, dubbed (disgustingly) "Slurp," to a new address at crawl.yahoo.net. Adjust your server logs as necessary and join the curmudgeons who are unimpressed.

Too little (or too much, by some complaints), too late, it would seem.

The


Why do we need to allow an army of Yahoo spiders to redundantly abuse our servers?
Is it a conceptual problem that Yahoo can't share pages already downloaded?

When I posed that question to one of their engineers I was given a lame excuse that the various crawlers had different needs….

Funny, Google managed to make some of their crawlers share CACHE, so we know it can be done.

Negativity often rings louder and truer than other things, but there is at least one voice in that forum who thinks Yahoo's update is "a small evolutionary improvement above" Google.

Even if in our hearts, we know that's not true. :)
 

Suggest a Correction

Found an error or have a suggestion? Let us know and we'll review it.

Share this article

Comments (0)

Please sign in to leave a comment.

No comments yet. Be the first to comment!