BT
x Your opinion matters! Please fill in the InfoQ Survey about your reading habits!

Google Is Now Indexing JavaScript and CSS Content: Pitfalls to Avoid

by Sergio De Simone on May 30, 2014 |

Google has announced that Googlebot, its web crawler, now executes and indexes some content in JavaScript. This change could impact negatively search results, unless a few basic rules are taken into account.

In a post on Google's Webmaster Central Blog, developers Erik Hendriks and Michael Xu, and Webmaster Trends Analyst Kazushi Nagayama, advise web developers that some changes that have taken place in the way Google handles JavaScript content when indexing web pages could impact negatively search results and provide a few hints as to how to prevent that.

According to Hendriks, Xu, and Nagayama, "in the past few months, Google's indexing system has been rendering a substantial number of web pages more like an average user’s browser" would when JavaScript has been turned on. This differs from the way Google traditionally indexed pages, i.e., by "only looking at the raw textual content that we’d get in the HTTP response body and didn't really interpret what a typical browser running JavaScript would see."

Sometimes, they write, JavaScript rendering will not have the expected outcome, "which may negatively impact search results for a site". To prevent this from occurring, they are offering some hints at potential problems and possible ways around them:

  • Sites blocking JavaScript or CSS files from access will not allow Google's indexing system to see a site like an average user. The recommendation is to allow for JavaScript and CSS files access in robots.txt. This is especially relevant for mobile sites, since JavaScript and CSS files would allow Google algorithms to understand that a page is optimized for mobile.
  • Web servers should also be able to handle the volume of crawl requests for resources, otherwise rendering results may be affected.
  • JavaScript code which is too complex or arcane could also prevent rendering the page fully and accurately.
  • Sometimes, JavaScript is used to remove content from a page rather than adding; this will make the removed content inaccessible to Google indexing engine.

Finally, suggest Hendriks, Xu and Nagayama, it's always a good idea to have a webpage degrade gracefully. This will make its content accessible to search engines that can't execute JavaScript yet.

Google is working on a tool that should be available soon to help webmasters better understand how Googlebot renders JavaScript and CSS content.

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Access to more data by Icaro Dourado

Finally! This subject has been in academic study for years and now Google becomes to enter the so-called deep web

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

1 Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2014 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT