README
spodr
Introduction
spodr is a utility to concurrently manage a work area that contains software projects that are usually:
- using git
- NodeJS-based
- dependent upon each other
These points are however optional. spodr respects non-NodeJS git checkouts or plain directories in the work area and applies tasks as appropriate.
For NodeJS projects, spodr will take over common tasks like linking projects with each other and keeping dependencies up-to-date. For any git checkout, spodr will help with tasks like pulling and pushing commits.
Note that spodr is somewhat opinionated. This is most apparent with the handling of git branches. spodr has the desire to always check out a branch named dev
, if it exists, unless you're currently on a branch that is neither dev
nor master
.
Getting Started
Preparing the Work Area
To begin working with spodr, enter an empty folder that will become your work area and clone some of your existing projects.
You can also just take any existing folder that contains your git checkouts. spodr does not maintain any metadata that would designate your work area in any specific way.
spodr also comes with 2 importers that allow you to easily clone all projects from a GitHub organization or a GitLab group. These might require you to provide an API access token. Follow the instructions in the console output.
$ spodr init --github stacktracejs
By default, spodr will try to run as many processes as possible. This can cause problems because multiple
npm
processes have the tendency to saturate any system (which, in turn, triggers further race-condition based bugs innpm
andyarn
) and they are prone to failure when running in parallel. Thus, it may be advisable to limit the number of concurrent processes using the--jobs
argument.
When using your own GitLab server, you have to specify that with --gitlab-host
or through the GITLAB_HOST
environment variable.
spodr will check out the default branch as configured server-side. If you want to ensure that you get the dev
branch, run spodr update
.
Dependency Installation
First and foremost, the spodr dependency installation mechnism is for development only. Your production deployments still rely on npm or yarn.
spodr can automatically download and install all dependencies of all packages referenced in the entire work area, without relying on external package managers.
When doing so, spodr will always download the highest possible matching version declared as a dependency for each module in the entire tree. In that, it drastically differs from how npm and yarn treat a dependency tree, where there is a desire to deduplicate the tree as much as possible and utilize packages, lower on the tree, that have matching semver ranges. spodr doesn't care if there is a matching package, if the declared semver range would allow for a newer version of the dependency. It will then use the newer version at the deeper branch.
spodr still massively benefits from deduplication, because it treats the entire work area as a single dependency tree.
Additionally, spodr will maintain a package cache local to each work area. This is the cache from where every dependency is linked into the projects. The projects that you would previously link globally are, to maintain their connections, now linked through that cache and don't conflict with modules in other work areas.
Once a package is cached, it is never copied. Every package is linked into the respective node_modules
folders of each project as required.
Root Pinning
$ spodr install --pin-roots
When providing --pin-roots
, spodr ensures that the root project (the one that you have in your work area) is used throughout the entire dependency tree, regardless of any requested semver range. This replaces the previous update --link --linkdep
operation, but is far more reliable, as consecutive npm install
runs could break node_modules
by replacing packages in linked projects.
When you don't provide --pin-roots
, your root projects are still linked into every location in the dependency tree, where their version matches the requested semver range. This can lead to instances of your root projects being downloaded into the package cache, with versions differing from those in your work area. This commonly happens when you don't have all of your "own" packages in the work area.
Usually, you want to provide --pin-roots
whenever you install dependencies. This might become the default in the final implementation and will have to be disabled with --no-pin-roots
.
Updating the cache
$ spodr install --update
If you want to ensure you have all the latest versions of all dependencies, you can use --update
. spodr will then ask the registry for every package again to see if newer versions are available and use those.
You can also just delete parts or the entire package cache at any time and rebuild it from scratch.
Note that others features may imply --update
, as spodr has a very strong desire to ensure that all packages are always used at the most recent version possible. If you want to prevent a certain package version from being used, utilize version locking.
Version Locking
When spodr generates the dependency tree, you can instruct it to replace certain versions of packages with different versions, thus, lock the dependency into a given version.
When you lock a version, it will be locked throughout the entire global dependency tree, regardless of the location it exists in.
Example
"locks": {
"chai-as-promised": {
"*": "5.3.0"
},
"eslint": {
"^5.8.0": "5.8.0"
},
"uglify-js": {
"^3.0.0": "3.4.8"
}
}
This .spodrlock.json
would cause all versions of chai-as-promised
to be locked down to version 5.3.0. The versions of eslint
and uglify-js
would be replaced as well, if they'd match the given version ranges. Multiple version ranges could be defined for a module.
Peering
A module can request to find a peer dependency in the dependency tree. This means that, while it doesn't directly want to depend on a module, it wants to be able to find a module of the given name and version in the tree. Usually, this is produced by a depending module declaring the dependency itself. This mechanism is controlled through the peerDependencies
in the package.json
.
This mechanism is important to resolve issues in other dependency managers. In spodr, you'd always want every single package to declare every single dependency it has. However, that is not being done, because people usually don't use spodr.
When npm or yarn see a peer dependency being declared, they warn you if your package higher up in the tree doesn't depend on the requested package. If the package is depended upon, the package is installed high up in the tree and will be found through module resolution.
When spodr sees a peer dependency being declared, it links the best possible version directly into the node_modules
of the requesting package.
Additionally, peering can be controlled through the .spodrlock.json
. This is required when modules blindly assume a specific dependency tree structure and just require()
a module by name, even through they neither directly or peer depend on it. This works in other package managers, because they register packages always as high up as possible in the isolated node_modules
folder of every single module. spodr doesn't do that for performance reasons. So you have to declare a peering manually to ensure a given module is available as a dependency of another module.
Example
"peering": {
"eslint": {
"*": {
"^eslint-plugin-.*