JavaScript Indexing Delays Are Still an Issue for Google

With Googlebot running on the most recent version of Chrome and JavaScript content being indexed faster than ever, it’s apparent that Google is getting better at indexing JavaScript.

It feels like SEO professionals are finally reaching the happy ending to a long and complicated movie.

Unfortunately, this movie has a twist:

JavaScript indexing delays are still a massive issue.

JavaScript Indexing Delays

It’s understood that indexing JavaScript content is delayed in Google, as it only happens when Google has enough resources for it.

But how long are those delays?

Here’s what we discovered.

The good news is that we can see that Google indexes JavaScript very quickly now.

As much as 60% of the JavaScript content is indexed within the first 24 hours after indexing HTML.

But there is also bad news.

As much as 32% of the tested pages have unindexed JavaScript content after one month, due to a variety of reasons.

Take a look at our data below:

How Long Should You Wait for Google to Index Your Javascript Content?

It depends on the website.

Here are a few examples.

It’s apparent than Google can quickly index the JavaScript content of the NYPost and Reuters (with a delay of less than 30 minutes, which is really fantastic!).

However, Google struggles with other news websites, such as the DailyStar and NBCNews.

There’s no evidence to suggest that Google prioritizes news websites for indexing JavaScript over other types of websites.

Where Did the Data Come From?

In order to check Javascript delays, Onely created TGIF (The Google Indexing Forecast).

The workflow behind it is pretty straightforward.

• A sample of 50+ popular websites was created and their sitemaps were tracked daily for newly added pages. Example websites in our sample included Ikea, T-Mobile, EventBrite, Reuters, and TrustPilot.
• All newly added pages were tracked to see if they were indexed and if their JavaScript content was indexed as well.

The methodology is depicted in this flowchart:

Why Should You Care?

Imagine you have a news website and the main content is injected by JavaScript.

If Google can index your news after, let’s say, 30 hours (which isn’t unusual!), that’s no longer news and you simply will lose a lot of your readers.

Who wants to hear about yesterday’s sun, if it’s raining today?

Things may get even worse when you use JavaScript for generating links to similar stories.

Indexing delays can cause Google to take a lot more time in discovering newly added pages on your news website.

Now imagine you have an ecommerce store and the competition is stiff. Meaning, your many competitors are publishing the same product with the same description.

We’ve noticed that ecommerce stores with product pages that are indexed first, tend to rank higher.

If it takes ages for Google to index your JS-dependent product description, your competitors will be taking the top positions for prominent queries.

The winner takes all.

It’s obvious that Google wants to diversify the content.

For transactional queries, you will often see not only offers but also reviews, opinions, release notes, etc.

There’s no room for competing product offers, except for paid ads.

It isn’t only about ecommerce and news websites.

There are many other types of websites that should care about Google indexing their JavaScript content quickly, such as event websites, job listings, and classifieds.

In general, any website that produces a lot of new content and changes it frequently.

Google Doesn’t Index All the JavaScript Content

Still not convinced that this is an issue worth paying attention to?

We also checked a random sample of URLs from popular ecommerce and news websites.

On average, JavaScript content is not indexed in Google in 25% of these websites’ pages.

This is one of the results of the two waves of indexing.

The second wave is not guaranteed.

Indexing JavaScript can fail due to many reasons, or may not happen at all.

What Can You Do?

The first thing you can do is to identify what content is generated by JavaScript.

You can use a tool like WWJD (What Would JavaScript Do?) to facilitate that (disclosure: this is my company’s tool).

You can check on a sample of URLs to verify if Google can index JavaScript content in a timely manner by using the “site:” command in Google.

Just take a sample of URLs created, let’s say, two weeks ago and type in Google: site:{URL} “{fragment}”

URL is the URL of a page and fragment is the fragment generated by JS. Commonly, it’s “About the product” or “Similar items”.

If Google cannot index your JavaScript in a timely manner, or even worse, if it cannot index JS content at all, you should try to discover the source of the problem, as this could be an indication of a larger JavaScript SEO issue that could lead to organic traffic drops.

Wrapping Up

If you are using JavaScript for generating important content, you have to implement it wisely and keep it under control.

Not doing so is like driving over 200 km/h (over 124 mph) and suddenly taking your hands off the steering wheel.

Whenever you notice Google ranking drops in regards to your website, you should always verify if it’s connected to your JavaScript generated content.

More Resources:

Image Credits

Featured & In-Post Images: Onely