Google Is Now Indexing JavaScript and CSS Content: Pitfalls to Avoid

| by Sergio De Simone Follow 7 Followers on May 30, 2014. Estimated reading time: 1 minute |

A note to our readers: You asked so we have developed a set of features that allow you to reduce the noise: you can get email and web notifications for topics you are interested in. Learn more about our new features.

Google has announced that Googlebot, its web crawler, now executes and indexes some content in JavaScript. This change could impact negatively search results, unless a few basic rules are taken into account.

In a post on Google's Webmaster Central Blog, developers Erik Hendriks and Michael Xu, and Webmaster Trends Analyst Kazushi Nagayama, advise web developers that some changes that have taken place in the way Google handles JavaScript content when indexing web pages could impact negatively search results and provide a few hints as to how to prevent that.

According to Hendriks, Xu, and Nagayama, "in the past few months, Google's indexing system has been rendering a substantial number of web pages more like an average user’s browser" would when JavaScript has been turned on. This differs from the way Google traditionally indexed pages, i.e., by "only looking at the raw textual content that we’d get in the HTTP response body and didn't really interpret what a typical browser running JavaScript would see."

Sometimes, they write, JavaScript rendering will not have the expected outcome, "which may negatively impact search results for a site". To prevent this from occurring, they are offering some hints at potential problems and possible ways around them:

  • Sites blocking JavaScript or CSS files from access will not allow Google's indexing system to see a site like an average user. The recommendation is to allow for JavaScript and CSS files access in robots.txt. This is especially relevant for mobile sites, since JavaScript and CSS files would allow Google algorithms to understand that a page is optimized for mobile.
  • Web servers should also be able to handle the volume of crawl requests for resources, otherwise rendering results may be affected.
  • JavaScript code which is too complex or arcane could also prevent rendering the page fully and accurately.
  • Sometimes, JavaScript is used to remove content from a page rather than adding; this will make the removed content inaccessible to Google indexing engine.

Finally, suggest Hendriks, Xu and Nagayama, it's always a good idea to have a webpage degrade gracefully. This will make its content accessible to search engines that can't execute JavaScript yet.

Google is working on a tool that should be available soon to help webmasters better understand how Googlebot renders JavaScript and CSS content.

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Access to more data by Icaro Dourado

Finally! This subject has been in academic study for years and now Google becomes to enter the so-called deep web

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

1 Discuss

Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you