Ok so I got this working like this:

ReactDOM.render(
  <ApolloProvider client={client}>
    <Routes history={browserHistory} />
  </ApolloProvider>, document.getElementById('root')
);

if (module.hot) {
  module.hot.accept()
}

Another issue came up — after hot reloading a page where I had a form for mutations, I get an error when trying to run that mutation.

Uncaught (in promise) TypeError: _this.props.addApp is not a function