Cache

moon's able to achieve high performance and blazing speeds by implementing a cache that's powered by our own unique smart hashing layer. All cache is stored in .moon/cache, relative from the workspace root (be sure to git ignore this folder).

Hashing

Incremental builds are possible through a concept known as hashing, where in multiple sources are aggregated to generate a unique hash. In the context of moon, each time a target is ran we generate a hash, and if this hash already exists we abort early (cache hit), otherwise we continue the run (cache miss).

The tiniest change may trigger a different hash, for example, changing a line of code (when an input), or updating a package version, so don't worry if you see a lot of hashes.

Our smart hashing currently takes the following sources into account:

• Command (command) being ran and its arguments (args).
• Input sources (inputs).
• Output targets (outputs).
• Environment variables (env).
• Dependencies between projects (dependsOn) and tasks (deps).
• For Node.js tasks:
  • Node.js version.
  • package.json dependencies (including development and peer).
  • tsconfig.json compiler options (when applicable).

Archiving & hydration

On top of our hashing layer, we have another concept known as archiving, where in we create a tarball archive of a task's outputs and store it in .moon/cache/outputs. These are akin to build artifacts.

When we encounter a cache hit on a hash, we trigger a mechanism known as hydration, where we efficiently unpack an existing tarball archive into a task's outputs. This can be understood as a timeline, where every point in time will have its own hash + archive that moon can play back.

File structure

The following diagram outlines our cache folder structure and why each piece exists.

.moon/cache/
    # Stores hash manifests of every ran task. Exists purely for debugging purposes.
    hashes/
        # Contents includes all sources used to generate the hash.
        <hash>.json

    # Stores `tar.gz` archives of a task's outputs based on its generated hash.
    outputs/
        <hash>.tar.gz

    # State information about anything and everything within moon. Toolchain,
    # dependencies, projects, running targets, etc.
    states/
        # Files at the root pertain to the entire workspace.
        <state>.json

        # Files for a project are nested within a folder by the project name.
        <project>/
            # Information about the project, its tasks, and its configs.
            # Can be used at runtime by tasks that require this information.
            runfile.json

            <task>/
                # Contents of the child process, including the exit code and
                # unique hash that is referenced above.
                lastRun.json

                # Outputs of last run target.
                stderr.log
                stdout.log