It took Google’s Web crawlers 15 years to come to terms with JavaScript

Image Credit: Neon Tommy/Flickr

JavaScript was created in 1995. Google’s search engine debuted in 1998. Yet it took 15 years for the two to fully intertwine.

Up until a few months ago, Google’s search engine crawlers couldn’t widely and accurately render one of the Web’s most fundamental programming languages. Nearly two decades old, JavaScript powers much of the Web’s interactivity. For years, #Google’s search engine crawlers couldn’t accurately render JavaScript, and thus, missed quite a bit of content.

The reasoning behind this delayed marriage is simple: When Google’s search engine was created, JavaScript was still immature, and Flash powered much of the Web’s interactivity in isolation. As Flash gradually died off, Google stuck to its guns: HTML, and later, CSS.

Today Google shared in a blog post that over “the past few months, our indexing system has been rendering a substantial number of web pages more like an average user’s browser with JavaScript turned on.”
Google is now finally able to interpret the Web much like a modern browser can. For Google, doing so is imperative to its survival. JavaScript is often used today to display content — text, images, and files that Google must understand in order to grow as an ad firm.

This is not a sudden development for Google. Back in 2012, Google “webspam” team head Matt Cutts urged developers to not hide their JavaScript from their crawlers because Google was “getting better” at crawling it.

Like in marriage, Google isn’t a flawless partner; the company’s crawlers aren’t perfect, Google shares, despite all the progress announced today. --VB