BT

InfoQ Homepage News Google Is Now Indexing JavaScript and CSS Content: Pitfalls to Avoid

Google Is Now Indexing JavaScript and CSS Content: Pitfalls to Avoid

This item in japanese

Lire ce contenu en français

This item in chinese

Bookmarks

Google has announced that Googlebot, its web crawler, now executes and indexes some content in JavaScript. This change could impact negatively search results, unless a few basic rules are taken into account.

In a post on Google's Webmaster Central Blog, developers Erik Hendriks and Michael Xu, and Webmaster Trends Analyst Kazushi Nagayama, advise web developers that some changes that have taken place in the way Google handles JavaScript content when indexing web pages could impact negatively search results and provide a few hints as to how to prevent that.

According to Hendriks, Xu, and Nagayama, "in the past few months, Google's indexing system has been rendering a substantial number of web pages more like an average user’s browser" would when JavaScript has been turned on. This differs from the way Google traditionally indexed pages, i.e., by "only looking at the raw textual content that we’d get in the HTTP response body and didn't really interpret what a typical browser running JavaScript would see."

Sometimes, they write, JavaScript rendering will not have the expected outcome, "which may negatively impact search results for a site". To prevent this from occurring, they are offering some hints at potential problems and possible ways around them:

  • Sites blocking JavaScript or CSS files from access will not allow Google's indexing system to see a site like an average user. The recommendation is to allow for JavaScript and CSS files access in robots.txt. This is especially relevant for mobile sites, since JavaScript and CSS files would allow Google algorithms to understand that a page is optimized for mobile.
  • Web servers should also be able to handle the volume of crawl requests for resources, otherwise rendering results may be affected.
  • JavaScript code which is too complex or arcane could also prevent rendering the page fully and accurately.
  • Sometimes, JavaScript is used to remove content from a page rather than adding; this will make the removed content inaccessible to Google indexing engine.

Finally, suggest Hendriks, Xu and Nagayama, it's always a good idea to have a webpage degrade gracefully. This will make its content accessible to search engines that can't execute JavaScript yet.

Google is working on a tool that should be available soon to help webmasters better understand how Googlebot renders JavaScript and CSS content.

Rate this Article

Adoption
Style

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

  • Access to more data

    by Icaro Dourado /

    Your message is awaiting moderation. Thank you for participating in the discussion.

    Finally! This subject has been in academic study for years and now Google becomes to enter the so-called deep web

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

BT

Is your profile up-to-date? Please take a moment to review and update.

Note: If updating/changing your email, a validation request will be sent

Company name:
Company role:
Company size:
Country/Zone:
State/Province/Region:
You will be sent an email to validate the new email address. This pop-up will close itself in a few moments.