How to Make Google Index a React Single Page Application Properly

When I was debating with myself a few weeks back whether or not I should build my death calendar project with server side rendering, the latest data I could find online said that this was essentially unnecessary since the Googlebot (the crawling mechanism that Google uses to index your site) was able to execute JavaScript since a few years back.

Well, it turns out this is not the entire story.

Fetch As Google

Lucky enough, Google offers a tool to allow you to see how Googlebot interprets your page. It’s available in under Google Webmaster Tools, under the title “Fetch as Google” once you’ve verified your internet property.

Running this test on my death date prediction tool it was clear that something was not working as intended.

So what’s the issue?

Googlebot uses Chrome 41

Turns out, Googlebot uses a version of Chrome that is by now several years old. The only way to get it to work is to polyfill whatever parts that are missing from that version of Chrome.

I debated for a while whether I should switch to server side rendering, but decided against it since that would make the entire setup significantly more complicated. Instead, I chose to go the route of including polyfills for older browser to accommodate Chrome 41. The downside of this is of course that the bundle size increases, but it felt like the most pragmatic approach.

After adding polyfills to my webpack setup and revisiting the “Fetch as Google”-tool, the site was being rendered properly again.

Howto: webpack, babel-polyfill, react and .babelrc

There are a few different changes needing to be made. I’ve outlined them below.

1. Add node_modules to your what webpack transpilates with babel

This will probably significantly increase your build time (almost doubled mine), but it seeems like some of the dependant libraries that I used required polyfilling as well. Your mileage may vary as to whether or not Googlebot will be able to crawl your site without this step.


// webpack.config.js
{
  test: /\.js$/,
  include: [
    path.resolve(__dirname, 'src'),
    path.resolve(__dirname, 'node_modules'),
  ],
  loader: 'babel-loader',
}

 

2. Add babel-polyfill as a dependency

yarn add babel-polyfill

3. Update your .babelrc file to include useBuiltIns and target older browsers


// .babelrc
[
  "env",
  {
    "targets": {
      "browsers": [
        ">1%"
      ]
    },
    "useBuiltIns": true
  }
],

4. Deploy new version and confirm with “Fetch as Google” that it’s working as intended.

Done.

Hope you Googlers find this useful, as information on this topic / for my setup was pretty sparse AFAICS.

Leave a Reply

Your email address will not be published. Required fields are marked *