You know that the require('express') line in your server.js gets the popular web framework loaded up. You're guessing that it gets loaded from node_modules as installed by npm. You are aware that you can include your own source files with require('./routes').
However, you are at a bit unease about the specifics of how require() finds its targets. And what if both you and your dependency depend on a library but different versions of it, which one gets loaded? You think you'd be writing better code if you understood modules better.
Modules organize program code
Modules are used to organize and create structure in program code. Modules allow you to define boundaries around logically related functionality and keep it separate from rest of the program code.
Every real world program, that is larger than a simple example application, will end up containing multiple modules.
Logically there are three categories of modules in Node.js
your own source files
core node modules
installed dependencies in node_modules
Load your own source files using relative path
You can include your own source files by passing a relative path to require(). The path is relative to the file calling require.
const routes = require('./routes');
Loads routes from the same directory.
const fb = require('../auth/fb');
Loads fb from parent directory's auth directory.
Supported types: .js, .json and .node
When loading your module
require('./routes')
your source file being loaded is named routes.js, but you pass it to require() without the extension.
Node tries to read the file using three different extensions: .js, .json and .node. This would result in trying:
./routes.js
./routes.json
./routes.node
If it finds a .js file, it gets interpreted as JavaScript source. The .json file will be parsed as JSON and the .node file will be loaded as compiled binary addon.
Core modules bundled in node binary
The argument to require() can be one of the reserved module names called core modules. They are modules that are bundled into the node binary. Modules such as fs, path and http are core modules.
Core module names have preference over any other name. Calling require('fs') will always load the core filesystem module even if you would have same name dependency in your project.
Installed dependencies in node_modules
Calling require() with a name that is not a core module causes node to start looking for matching name inside a node_modules directory.
The search is repetitive and starts from the immediate parent of the file calling require(). It continues to parent of that directory, and parent of that, until it finds a match or the root of the filesystem is reached.
When matching directory is found, node looks in for
package.json, and tries to read its "main" property
index.js, .json or .node
For example if you have
const _ = require('underscore');
somewhere in your program source. You have the correct dependency in package.json and you have called npm install. Then at the top level of the project have node_modules/underscore directory and there is a package.json file:
...
},
"main": "underscore.js",
...
that tells node to load node_modules/underscore/underscore.js.
If the search doesn't find a match, as the last resort node looks for your home directory and the global node installation directory. However, it is good design to keep your modules local to your project so you have complete control over them and don't depend on what is globally installed on the system.
Modules are cached
It is common to require the same module from multiple locations in your program code. Node does caching so only the first time new module is loaded will the code for that module be parsed and executed. The second time the same module is loaded the cached result is returned.
The caching is based on the actual resolved file location of the module, not just what is passed to require() as an argument. This has an important consequence. The same require() call may result in different module being loaded depending on the calling file location.
Dependencies have their own node_modules
The dependencies you state in package.json are installed under node_modules directory. Those dependencies may in turn depend on other modules. These are called transitive dependencies.
Npm takes care of installing transitive dependencies. For each module it checks if it depends on any module that is not already present in parent level node_modules directory. For any module having missing transitive dependencies, it acts if calling npm install inside that module's directory with one exception.
The exception is that it only installs dependencies not present in any parent node_modules directories. It only installs necessary missing dependencies.
Installing transitive dependencies may lead to deep level of nested node_module directories. For example installing express-4.12.4 results in directory
where express depends on type-is that in turn depends on mime-types that depends on mime-db.
Module depends on different version of library
Npm installs transitive dependencies so that it leverages on modules already present in higher up node_modules directories. This means you do have to worry about making all dependencies work together. Using newer module in your project level package.json may cause some dependent library to go off its rocker when it does not get the version it expects.
Conclusion
That's all there really is to loading modules. There are three types of modules: your own source files, core node modules and installed dependencies. Modules are always cached and npm tries to leverage modules already present higher up in the tree.
