Dejan Lukić

Config Hell

Originally written for Deno.

Introduction to Config Hell

When you spin up a Node repo of any flavor, your root directory is full of config files immediately. For example, in the latest version of Next.js, you get next.config.js, eslintrc.json, package.json, and tsconfig.json.

Add styling, and you'll have postcss.config.js and tailwind.config.js. Need middleware? Add middleware.ts to it. Want error monitoring? Add sentry.server, sentry.client, and sentry.edge config.js. Plus, your env files, Git files, and your Docker files.

Every project needs config files, and every tool needs config files. Each is different, and sometimes they contradict each other. Welcome to config hell.

How Did We Get Here?

Config files are neither new nor JS-specific. Before you had UIs, you had config files on Unix systems. They are everywhere, on databases and server-side stuff.

Having a separate config file with parameters became the de facto pattern to use software (compared to passing CLI flags or using environment variables or whatever else) because:

But programming languages mostly get away without them. None in Python, None in Ruby. C and C++ have a set-and-forget CMake.

And even if they do, they're mostly the same thing forever–you don't have to update CMake every week, and it doesn't interfere with anything else.

The Package Updating Havoc

Config files in Node have proliferated. There are a ton of them. They sometimes populate the whole root directory hosting tens of different config files.

Updating config files is another common nightmare. Frequently changing syntax can bring breaking changes to your project and, with it, uncertainty and unnecessary spent time dealing with short-term fixes which might be deprecated within the next update.

The frequent and rapid growth and changes of libraries can lead to inadequate versioning practices. While proper documentation and testing are crucial, they may not address the versioning issue. As a result, even code examples from a month-old library may become obsolete or faulty, requiring you to spend countless hours trying to make them work again.

In the meantime, you could learn something more practical, like moving away from Node to Deno.

The Dependency Confliction Hell

Conflicting dependencies, caused by different libraries and frameworks requiring different versions of the same dependency, which may only sometimes work together harmoniously, are direct cause by bad dependency tree management.

Conflicting dependencies can create a lot of management overhead, as you must manually ensure that the config files override each other precisely to ensure that the codebase stays working as intended.

You must write these things and know about all the crap that goes into them. In and out. That can take anywhere from a few minutes to whole sprints, wasting precious time and money.

Why do we have them? The modular way Node works is unconstrained – you need a build step, so you need a build config; there's no built-in linter, so you need a linter, so you need a linter config; no tests, so you need a test suite, so you need a test suite config, and the list goes forever.

Every tool, framework, or plugin conceivably ships with a config file. So is npm's config hell a linear function correlated to purely the number of npm/JS frameworks/tools that exist? Perhaps.

Are there any notable or prominent tools or frameworks that don't have a config file? In another parallel universe.

The Holy Config Convention

In 2020, the Node community discussed this issue as the creeping scourge of config files, addressing the problems of current config file handling and structure.

As stated by the community, "We put config files in the project root because we put config files in the project root. There is no reason other than a lack of an alternative convention.".

Agreeing on a convention for implementing a shared subdirectory, and updating tools and frameworks to adopt it, would significantly reduce the problem.

The idea of the community is that if the convention is agreed upon, it would be flexible enough to reach beyond the JS ecosystem.

The users pointed out many issues. Would they move package.json? It would be problematic for Node, where the package.json "type" field is set.

Linters' config would face similar issues due to being composed up the hierarchy, and they usually oversee the parent directory.

In late 2020, Ian Sutherland, one of the Node's engineers, closed the issue, stating that they're probably not "able to get consensus on a new config directory". Following that, getting all tools to allow the customization of the config file's location seems more realistic.

The Boilerplate Reduction - Best Practices

In 2018, Ryan Dahl, the father of Node, announced Deno - a hope for partially solving Node's dependency tree explosion problem. Ryan designed Deno to counter his mistakes when he originally designed Node.

A configuration file is not required now and will not be necessary for the future. Deno still works fine with the default options and no configuration file.

While configuration files may not always be necessary when using Deno, they can be a valuable tool for customizing the environment and other features.

Using too much boilerplate code in a project can accumulate cruft, making the codebase more complicated to read and understand.

Boilerplate code is often repetitive and can distract from the essential parts of the code. As a result, you may spend more time navigating and understanding the boilerplate than focusing on the code's core logic.

Stepping into a code base with many configuration files can be overwhelming to understand how the different files interact and affect the application's overall behavior.

Here are some guides on how to overcome that issue:

How to ”Hide” the Config Hell

You're displeased by the look of tens of different configuration files, all ending differently, some starting with a dot, some ending in json, and so on. While opting out of their usage is not a solution, you can follow some tricks to hide them.

Adding the path to package.json

Adding a path to the package.json file can help hide configuration hell by centralizing configuration settings in a single location. Here's how you can do it:

  1. Create a new folder for your configuration files, and add a config folder to your project's root directory. You can name this folder whatever you'd like.

  2. Create a default.json file in the config folder, and add your default configuration settings to this file.

  3. In the package.json file, add a config field and specify the path to the configuration folder like this:

    {
  "name": "my-project",
  "version": "1.0.0",
  "config": {
    "path": "./config"
  }
}

  4. Install the config package by running the following command in your project's root directory.

    $ npm install config

  5. You can now use the config package to access your configuration settings in your code. For example, to access the value of a setting called foo in your default.json file, you would do the following:

    const config = require('config');
const foo = config.get('foo');

Using IDE plugins

You can hide files from your IDE's sidebar, usually within the IDE's settings. Here are the settings for common IDEs.

Visual Studio Code

  1. Open Visual Studio Code

  2. Press CTRL (cmd) + SHIFT + P

  3. Search for Preferences: Open User Settings (JSON)

  4. In the end, add the following.

    "files.exclude": {
	"node_modules/": true,
	"**/.*": true,
	// Any other config files
}

JetBrains IDEA

For JetBrains IDEA, follow the File nesting rules outlined by their documentation.

Final Thoughts

The long-term solution is for tools to allow end-users to specify where they want their config files to be loaded and not try to get everyone to agree to store config files in a specific directory.

As a tool author, a good suggestion is to give users options to set paths to where to put the config file.

Deno does away with config files. You can use one, but absolutely not required, and best if you don't. Why? Opinionated. There is a Deno way to do Deno stuff, so all config is already factored in.

Deno's goal is to make programming fun and productive. But, unfortunately, setting up config is not fun and slows down momentum. Hence, the built-in toolchain.

The main tools that require config are packaged with Deno. Entire toolchain out of the box:

Conclusion

Config files are inevitable. That's OK. But you have to be discerning about when cruft is accumulating. The challenges of managing configuration files in Node, including the proliferation of files, conflicting dependencies, and frequent updates, are the tip of the iceberg of every Node developer's journey.

The inability to agree on and implement a convention leaves us to create alternative solutions. For example, tools should allow end-users to specify where they want their config files to be loaded without causing other packages to break.

References

https://github.com/nodejs/tooling/issues/79 https://medium.com/netscape/npm-dependency-errors-then-youre-doing-it-wrong-635160a89150 https://twitter.com/wesbos/status/1620793926306205701?s=46&t=EKpXnu4K6pqpaOGW0iZU3A

https://www.reddit.com/r/webdev/comments/ds4qij/javascript_configurations_have_become_a_hell/

#nodejs #thoughts #tips