Compiled config vs runtime config

[mars]

💚 The custom environment variables feature.

---

Update February 2017: We devised a solution for runtime config on Heroku. create-react-app-buildpack now supports runtime environment variables by way of injecting values into the javascript bundle at runtime. This is still an open question for other runtimes and deployment techniques.

---

This feature leverages Webpack's DefinePlugin which is explained as:

…global constants which can be configured at compile time.

The issue is not all configuration should be compiled into an application.

Things that are stable between environments (dev, staging, production) of an app make sense to compile-in:

• version number
• commit sha/number
• browser support flags

Things that change between environments would be better provided dynamically from the current runtime:

• URLs of an APIs (may change for each environment)
• secret tokens (may change for each user or request)

Ideally a bundle could be tested in CI, then promoted through environments to production, but compiling all of these values into the bundle means that every promotion requires rebuild.

I've used Node to serve single-page apps in the past with a base HTML template that gets environment variables set via inline <script> at runtime; even used a server-side Redux reducer to capture process.env values into an initial state.

Now, I'm trying to find a good solution to the conflation of runtime configuration values being compiled into the bundle.

create-react-app is clearly focused on producing a bundle that can be used anywhere.

Is this project open to solutions/hooks for injecting runtime variables, or is this wandering into npm run eject territory?

[gaearon]

An ad hoc solution would be to put some placeholder into HTML and then replace it with a script tag initializing a global before serving the app. This is however outside of CRA concerns IMO.

[glenngillen]

I got directed to this thread via Twitter, so need to qualify my input with I've limited experience re best practice for React & single page apps. My interest is around 12factor/Heroku specific aspects.

So looking at the problem as it pertains to Heroku implementation of 12factor, the issue is really that the notion of a slug and release end up conflated for single page apps because they environment is baked in at build/compile time rather than runtime. I understand the appeal in doing that, but I wonder what the options are for trying to avoid it.

An ad hoc solution would be to put some placeholder into HTML and then replace it with a script tag initializing a global before serving the app

Could that be encapsulated into some higher order and reusable concept like process.env on the server side? If we can converge on a single object to return the values as a best practice, then it seems the problem reduces down to "where does that object get the values from?". To which I'd suggest the following as an order of preference, which could be pluggable/configured by the developer:

• Fetched from a another service
  • Pros: Can actually get runtime specific values at runtime. Changes are reflected immediately. No need to rebuild for each environment. No need to rebuild for each value change
  • Cons: Another thing to run/use. If you're running it yourself now potentially need to run a server somewhere to lookup & return these values.
• All values stored in nested object hierarchy, runtime is able to determine which one to use
  • Pros: No need to rebuild for each environment.
  • Cons: Need to rebuild for each value change. Values for all environments are exposed in source to client, is that a potential security issue?
• Values interpolated in at build time:
  • Pros: Client only exposes what is relevant for a given environment
  • Cons: Need to rebuild for each environment. Need to whenever a value changes.

[tibdex]

The Twelve-Factor App guide mentioned by @mars has a section about "build, release, run" cycle. It explains that the build stage is only about creating an executable that could run anywhere. It is the release stage that should be responsible for combining this executable with the the deployment environment configuration.

create-react-app is really close to being able to generate builds deployable anywhere!

Indeed, if merged, this PR #1489 will relieve apps not using the HTML5 pushState history API to specify the URL on which they will be deployed.

Then, the only remaining task to make create-react-app builds compliant with the Twelve-Factor App recommendations is to let the deployment environment specify some part of the configuration. In fact, create-react-app documents how to use environment variables but they only work at compile time.

One of the solutions to add support for runtime environment variables that is really simple and lightweight would be to add an env.js file inside the public directory and load it in a script tag in index.html just before the main.js bundle.

env.js could have the following content:

// This file will not end up inside the main application JavaScript bundle.
// Instead, it will simply be copied inside the build folder.
// The generated "index.html" will require it just before this main bundle.
// You can thus use it to define some environment variables that will
// be made available synchronously in all your JS modules under "src". 
//
// Warning: this file will not be transpiled by Babel and cannot contain
// any syntax that is not yet supported by your targeted browsers.

window.env = {
  // This option can be retrieved in "src/index.js" with "window.env.API_URL".
  API_URL: 'http://localhost:9090'
};

The body of index.html would be:

<body>
    <div id="root"></div>
    <!--
      Load "env.js" script before the main JavaScript bundle.
      You can use it to define "runtime" environment variables.
      Please open this file for more details.
    -->
    <script src="%PUBLIC_URL%/env.js"></script>
  </body>

After running npm run build, you would find your env.js file untounched under build/env.js. You would then be able to edit this file manually directly on the deployment machine or through a small bash script to change API_URL to https://api.my-app.com for instance.

Should I submit a PR documenting this technique or even adding this env.js file to the public folder of the boilerplate ?

[jayhuang75]

@tibdex I like this solution, but did you test via the npm run test with the jest. seems will have some error and it can not get in the test level. any advice?

[mars]

Writing configuration values into a file is antithetical to 12-factor. Instead, always use runtime environment variables for configuration.

create-react-app [CRA] does not provide a production runtime. It only builds a static bundle. Runtime environment is not something solvable by CRA, unless it becomes prescriptive about deployment which is unlikely given its wide range of applications.

Solving this problem is about finding a way to set those variable values in the React app at runtime instead of buildtime. I know of two ways to accomplish this:

1. Render runtime environment values into global variables in a <script> element in index.html with a Node/Express server. Then, use those globals within your React app. (This would need to be resolved for local development too.)
2. Deploy to Heroku using create-react-app-buildpack which supports runtime environment variables by way of injecting values into the javascript bundle at runtime.

This is a challenging topic 🤓 Bravo for trying to solve it 👏👏👏

[glenngillen]

I've been working on a somewhat related effort to manage the configuration/credentials at runtime problem. And I'd be interested in helping come up with a standard approach of doing it for SPAs like this. The basic approach is:

• Expose a single key in build phase that can be included in the build in some fashion
• Have a client in the runtime that can use that key to query an API (in this case https://api.voltos.io), and receive back whatever config the runtime needs
• Expose that additional config in a way that can be re-used by the client/rest of the SPA

If we can reach agreement on what the API response should look like, how to define what the env var API host/url should be, and how/where it should be exposed I'd be happy to do the heavy lifting and try to write a shim. I can foresee some API-like wrapper that lets people store their secrets direct on S3 and the API just exposes them in the right format.

What do y'all think?

[mars]

@glenngillen from my perspective, configuration should be in-place when the javascript app loads in the browser. Requiring remote fetch of config values from an external service:

• taxes performance (how does the app render before those config values are present?)
• increases complexity (how do you account for the async config dependency in any given javascript app?)
• decreases reliability (what happen if the config service is inaccessible?)

The fundamental problem is that SPAs can be deployed many many different ways: static web sites, GitHub sites, instant deploy services (Heroku, Now, Netlify, etc), embedded in blogs (Wordpress, Drupal), composed with web app servers (Node, Rails, Django, etc), and the runtime is where this problem must inevitably be solved.

[glenngillen]

@mars agreed, but... as you've mentioned "runtime" in this context includes things like static web/hosting directly off s3 and GitHub pages. Short of fetching config on demand I see no alternative for these apps other than pushing the config back into the build/compile phase.

[jayhuang75]

@tibdex Finally I get the unit test code fixed. Basically we have to mock the window.env in the jest mock folder
@mars Thanks for the encourage! :) and our use case, its build once deploy everywhere with the docker image to the PAAS. Once the docker image deploy to the PAAS, we can just change the environment variable to point the different API EndPoint.
Disclaimer: we just export the public accessible API endpoint. If you want to add your secret key, etc. please DO NOT use the solution.

[tibdex]

@jayhuang75, I think a better place to provide the configuration values to the tests might be src/setupTests.js.

@mars, I agree that ideally the config should not be stored in a file. The env.js file could however be generated by the server on which the static bundle is deployed. The server would then be able to use it's own OS environment variables to generate this file. I believe it's a good and simple way to solve runtime config if you have control over the server ;)

[mars]

@tibdex your technique of adding a <script src> include to index.html and separately writing an env.js file during deployment seems sound for the purely static use-case. Preserving the local dev & testing experience is going to be the challenge.

Perhaps using a module-based approach like I did for Heroku runtime env would allow you to capture logic for dev/test vs production runtime and graceful fallback/error-messaging when the env is not setup correctly?

[ivosabev]

I am having the same problem. I need to change a the index.html based on a dynamic value from the database. This is easy to do when serving a prebuilt CRA via NodeJS with a simple string search and replace, but becomes impossible in dev mode without gutting the CRA inside out. The more I think about it I do not see an easy way for doing this dynamically in both environments. Any plans to support this or any ideas how to implement it with the least CRA changes possible?

[rmoorman]

@tibdex you mean something along the lines of this (regarding the server on which the static bundle is deployed)? Seems to be working allright :-)

const express = require('express');
const path = require('path');
const serialize = require('serialize-javascript');

const env = {
  'API_URL': process.env.API_URL,
  'DYNO': process.env.DYNO || 'Not running on a dyno',
}

const app = express();

app.use(express.static(path.join(__dirname, 'build')));

app.get('/env.js', function (req, res) {
  res.set('Content-Type', 'application/javascript');
  res.send('var env = ' + serialize(env));
});

app.get('/*', function (req, res) {
  res.sendFile(path.join(__dirname, 'build', 'index.html'));
});

app.listen(process.env.PORT);