Skip to main content

Cache

moon's able to achieve high performance and blazing speeds by implementing a cache that's powered by our own unique smart hashing layer. All cache is stored in .moon/cache, relative from the workspace root (be sure to git ignore this folder).

Hashing

Incremental builds are possible through a concept known as hashing, where in multiple sources are aggregated to generate a unique hash. In the context of moon, each time a target is ran we generate a hash, and if this hash already exists we abort early (cache hit), otherwise we continue the run (cache miss).

The tiniest change may trigger a different hash, for example, changing a line of code (when an input), or updating a package version, so don't worry if you see a lot of hashes.

Our smart hashing currently takes the following sources into account:

  • Command (command) being ran and its arguments (args).
  • Input sources (inputs).
  • Output targets (outputs).
  • Environment variables (env).
  • Dependencies between projects (dependsOn) and tasks (deps).
  • For Node.js tasks:
    • Node.js version.
    • package.json dependencies (including development and peer).
    • tsconfig.json compiler options (when applicable).

Archiving & hydration

On top of our hashing layer, we have another concept known as archiving, where in we create a tarball archive of a task's outputs and store it in .moon/cache/outputs. These are akin to build artifacts.

When we encounter a cache hit on a hash, we trigger a mechanism known as hydration, where we efficiently unpack an existing tarball archive into a task's outputs. This can be understood as a timeline, where every point in time will have its own hash + archive that moon can play back.

File structure

The following diagram outlines our cache folder structure and why each piece exists.

.moon/cache/
# Stores hash manifests of every ran task. Exists purely for debugging purposes.
hashes/
# Contents includes all sources used to generate the hash.
<hash>.json

# Stores `tar.gz` archives of a task's outputs based on its generated hash.
outputs/
<hash>.tar.gz

# State information about anything and everything within moon. Toolchain,
# dependencies, projects, running targets, etc.
states/
# Files at the root pertain to the entire workspace.
<state>.json

# Files for a project are nested within a folder by the project name.
<project>/
# Information about the project, its tasks, and its configs.
# Can be used at runtime by tasks that require this information.
runfile.json

<task>/
# Contents of the child process, including the exit code and
# unique hash that is referenced above.
lastRun.json

# Outputs of last run target.
stderr.log
stdout.log