Skip to Main Content

Slurp Gets a New Name

Posted on 6.07.2007
The Yahoo! Search blog announced a few weeks ago they would be migrating their Web crawler (Slurp) over to a new domain. That transition is now complete and all YahoO! machines are (and will be) coming from crawl.yahoo.net.

This change does not cover Yahoo!'s other crawlers including those from Yahoo! China or others which cater to specific verticals such as Yahoo! Shopping or Yahoo! Travel which have their own user-agent.

Yahoo! is encouraging sites to setup reverse DNS-based authentication of its crawler to ensure no rogue bots masquerade as Slurp. They provide a good checklist on how to make that happen.
WebsiteMagazineMiniLogo

Leave Your Comment

Login to Comment

Become a Member

Not already a part of our community?
Sign up to participate in the discussion. It's free and quick.

Sign Up

 

Leave a comment
    Load more comments
    New code
  • Your Guide to 2017 Sales Tax Changes

    2017 Sales Tax Changes