moon v0.20 - Toolchain, caching, and hydration improvements

With this release, we've focused heavily on future proofing our toolchain and how it integrates with moon. We've also landed a handful of quality of life improvements.

Breaking changes

To start, we have a few breaking changes this release to be aware of!

Moved toolchain settings

The .moon/workspace.yml config file was getting rather bloated and complicated, as it contained the projects list, settings for each toolchain language, and settings for each supported service (runner, generator, etc). Furthermore, this file will keep getting larger with each new language and service we support.

To future proof moon, and to land as many breaking changes before an official v1, we've decided to move toolchain specific settings into a new file, .moon/toolchain.yml. This new file will house all language and dependency manager specific settings.

To migrate, move the node and typescript settings from .moon/workspace.yml to .moon/toolchain.yml.

.moon/toolchain.yml

node:
  # ...

typescript:
  # ...

Moved project-level overrides

Continuing with the changes above, we've also moved the workspace.node and workspace.typescript from moon.yml into a new parent field, toolchain. We think this makes more sense.

moon.yml

# Before
workspace:
  node:
    version: '...'

# After
toolchain:
  node:
    version: '...'

Future of the toolchain

We're really proud of our toolchain, as it avoids an array problems that developers deal with on a day to day basis, primarily around running tasks using the wrong version of Node.js or their chosen package manager. While we're in the process of supporting additional languages, starting with Deno, we had an idea... Since this is basically a better "version manager for tools", why not extract this out into something else?

And that's what we plan to do! Before we do so, we've had to make some architectural changes, many of which have landed in this release. During this process, we were able to implement an even better toolchain, polish many of the edge cases, and improve performance! Stay tuned for more information on this new tool!

Increased output hydration by 8-10x

In moon, hydration is the concept of unpacking an existing hashed artifact into a task's outputs during a cache hit. In our previous implementation, we would delete all existing outputs before unpacking the archive to ensure a clean slate and to avoid stale files. While this worked, it wasn't the most performant, taking about 280ms for 1,000 files (which is still reasonably fast!).

In our new implementation, we now utilize a smart file tree diffing algorithm that will only unpack files with different content, and will automatically remove stale files in the process. This has resulted in a 10x performance increase, taking about 30ms! Multiply this by many projects, the results should be very apparent.

We hope you enjoy this improvement, as it's the first of many to come!

Cache any and all targets

By default, all targets that produce outputs are cached and archived. For targets that do not produce outputs, like testing, linting, and typechecking, these are not cached. This will result in slower CI times as they are continuously ran, even if nothing has changed. However with our new remote caching layer, we can skip these from running entirely!

To achieve this, we've added a new setting to .moon/workspace.yml called runner.archivableTargets. This setting accepts a list of targets that should be archived, regardless of whether or not they produce outputs.

For example, if we want to cache and archive testing, linting, and typechecking, we can define the following:

.moon/workspace.yml

runner:
  archivableTargets:
    - ':lint'
    - ':test'
    - ':typecheck'

Path handling in generator templates

Our code generation is powered by Tera, a templating engine with a ton of built-in utilities. However, it does not provide utilities for file paths, and as such, we now provide path_join and path_relative filters.

{{ some_path | path_relative(from = workspace_root) }}

We also now inject variables for the working directory, destination, and workspace root. This will help with custom paths, especially within frontmatter! View the codegen docs for more information.

Other changes

View the official release for a full list of changes.

• Added vcs.remoteCandidates to .moon/workspace.yml to customize the remotes for Git to query against.
• Added support for moduleSuffixes and moduleDetection in TypeScript tsconfig.json compiler options.
• YAML files will now respect the closest .editorconfig file.
• Refactored terminal output for bette readability.

What's next?

Expect the following in the v0.21 release!

• An interactive dependency and project graph visualizer.
• A more performant project graph.
• More quality of life improvements for affected files.