New MSN Crawler and Index Update
Last week on Microsoft's Webmaster Center Blog, the Live Search team announced that it will be testing an update to MSNBot (the company's official site crawler), which may show up as a new crawler name in your referrer logs. The new crawler user agent string will appear as:
If you take a look at your raw server logs, you will still see the current version msnbot/1.1 (+http://search.msn.com/msnbot.htm) as Live Search’s primary user agent.
Many webmasters are seeing results from different data centers, and Live has been rotating different indexes in and out as recently as yesterday. This could mean that the crawlers represent two completely different algorithms. Whether this ultimately will improve relevancy at Live.com or any of the Microsoft Search properties is anyone's guess. It is probably safe to say that the damage has been done, so don't expect your MSN traffic stats to shoot up any time soon. Remember, this is all about users, and Microsoft has a long way to go to steal back its share of global searches from Google and Yahoo.