Crawl Errors: Major Upgrade for Google Webmaster Tools
Several rather significant enhancements are being rolled out in Google Webmaster Tools this week. Google announced that it now detects and reports several new types of errors, is showing which errors are most important, and is letting webmasters dive into the details.
Google will be splitting the types of errors reported in GWT into two types - site errors (those which affect an entire site, e.g. DNS resolution failures) and URL errors (those where Googlebot tried to crawl a URL but failed, e.g. a 404 error). Google now tracks the "failure rates" for each type of site-wide error and will try to send alerts when the errors become frequent enough to warrant attention.
Perhaps the best part of this round of enhancements is that Google is helping webmaster prioritize the errors. Webmasters could previously see up to 100,000 errors of each type but there was no way to sort, search or mark the progress made in correcting those errors. Now, Google is showing what they think are the 1000 most important errors up front, enabling webmasters to indicate to Google when those errors have been fixed, and even view details about them - and that's where things get really good. Clicking on an individual error URL will show a detail pane with information including the last time Google attempted to crawl the URL, when the problem was first noticed and a brief explanation of the error.