Official Google Webmaster Central Blog

Smartphone crawl errors in Webmaster Tools

Posted: 04 Dec 2013 02:50 AM PST

Webmaster level: all

Some smartphone-optimized websites are misconfigured in that they don't show searchers the information they were seeking. For example, smartphone users are shown an error page or get redirected to an irrelevant page, but desktop users are shown the content they wanted. Some of these problems, detected by Googlebot as crawl errors, significantly hurt your website's user experience and are the basis of some of our recently-announced ranking changes for smartphone search results.

Starting today, you can use the expanded Crawl Errors feature in Webmaster Tools to help identify pages on your sites that show these types of problems. We're introducing a new Smartphone errors tab where we share pages we've identified with errors only found with Googlebot for smartphones.

Some of the errors we share include:

Server errors: A server error is when Googlebot got an HTTP error status code when it crawled the page.
Not found errors and soft 404s: A page can show a "not found" message to Googlebot, either by returning an HTTP 404 status code or when the page is detected as a soft error page.
Faulty redirects: A faulty redirect is a smartphone-specific error that occurs when a desktop page redirects smartphone users to a page that is not relevant to their query. A typical example is when all pages on the desktop site redirect smartphone users to the homepage of the smartphone-optimized site.
Blocked URLs: A blocked URL is when the site's robots.txt explicitly disallows crawling by Googlebot for smartphones. Typically, such smartphone-specific robots.txt disallow directives are erroneous. You should investigate your server configuration if you see blocked URLs reported in Webmaster Tools.

Fixing any issues shown in Webmaster Tools can make your site better for users and help our algorithms better index your content. You can learn more about how to build smartphone websites and how to fix common errors. As always, please ask in our forums if you have any questions.

Posted by Pierre Far, Webmaster Trends Analyst