moon's able to achieve high performance and blazing speeds by implementing a cache that's powered by
our own unique smart hashing layer. All cache is stored in
.moon/cache, relative from the
workspace root (be sure to git ignore this folder).
Incremental builds are possible through a concept known as hashing, where in multiple sources are aggregated to generate a unique hash. In the context of moon, each time a target is ran we generate a hash, and if this hash already exists we abort early (cache hit), otherwise we continue the run (cache miss).
The tiniest change may trigger a different hash, for example, changing a line of code (when an input), or updating a package version, so don't worry if you see a lot of hashes.
Our smart hashing currently takes the following sources into account:
- Command (
command) being ran and its arguments (
- Input sources (
- Output targets (
- Environment variables (
- Dependencies between projects (
dependsOn) and tasks (
- For Node.js tasks:
- Node.js version.
package.jsondependencies (including development and peer).
tsconfig.jsoncompiler options (when applicable).
Archiving & hydration
On top of our hashing layer, we have another concept known as archiving, where in we create a
tarball archive of a task's outputs and store it in
.moon/cache/outputs. These are akin to build
When we encounter a cache hit on a hash, we trigger a mechanism known as hydration, where we efficiently unpack an existing tarball archive into a task's outputs. This can be understood as a timeline, where every point in time will have its own hash + archive that moon can play back.
The following diagram outlines our cache folder structure and why each piece exists.
# Stores hash manifests of every ran task. Exists purely for debugging purposes.
# Contents includes all sources used to generate the hash.
# Stores `tar.gz` archives of a task's outputs based on its generated hash.
# State information about anything and everything within moon. Toolchain,
# dependencies, projects, running targets, etc.
# Files at the root pertain to the entire workspace.
# Files for a project are nested within a folder by the project name.
# Information about the project, its tasks, and its configs.
# Can be used at runtime by tasks that require this information.
# Contents of the child process, including the exit code and
# unique hash that is referenced above.
# Outputs of last run target.