Skip to main content

moon v0.19 - Remote caching beta, affected files, and graph optimization

· 3 min read
Miles Johnson
Miles Johnson
Founder, developer

With this release, we've focused heavily on our remote caching architecture. Since this is a massive task that requires much work behind the scenes, this overall release is rather light.

Remote caching beta​

A major request for moon is to support remote caching. This is a critical feature that allows artifacts to be shared across machines and environments, by storing them in a cloud storage provider, like AWS S3.

We've been working on this feature for months, primarily behind the scenes building the necessary services and tools, and we've finally reached a point where we could integrate it with moon directly! However, since this feature is very complex, it's not yet available to the public, but while you wait, you can sign up for beta access! Jump over to the official remote caching documentation for more information!

We'll be working on remote caching through every release, continually polishing the implementation, supporting new features, enabling new regions, calculating pricing, so on and so forth. So stay tuned!

Affected files filtering​

We've spent a good portion of this release working on affected detection for projects and tasks, primarily through --affected, and have uncovered and fixed a handful of edge cases. With that being said, we asked ourselves, "How could we improve the developer experience even more"? What about passing the affected files list to the running task?

And that's exactly what we've done through the new task option affectedFiles! This option works in unison with the --affected option, enabling some awesome functionality. Given the following config:

moon.yml
tasks:
  lint:
    command: 'eslint'
    options:
      affectedFiles: true

When we run this task without --affected, it will append an argument for the current directory (.). This will lint the entire project.

# moon run project:lint
eslint .

However, when we run this task with --affected, and have have touched files in the working tree, it will now append an argument for each file that matches the task's inputs. This will now only lint the provided files, instead of the entire project.

# moon run project:lint --affected
eslint ./file.ts ./another/file.ts

This functionality is perfect pre-commit hooks, and as such, we've added a guide for utilizing this pattern with Git hooks!

Dependency graph optimizations​

Our dependency graph determines which targets to run, what tools to install, projects to sync, and in which order. It's a very complex directed acyclic graph, and has only gotten more complicated over the year, especially with the introduction of project-level tool overrides.

We decided to rewrite the dependency graph from the ground up to mitigate many of these issues, optimize nodes and edges, reduce the number of data cloning, and to increase the overall performance. Everything will still work exactly the same (all of our existing tests passed with no issue)!

On top of this, we've also migrated the internals of moon to a new hashing algorithm that should see a 5-10% increase in performance!

Other changes​

View the official release for a full list of changes.

  • Package managers in the toolchain are now installed in isolation, instead of through Node.js.
  • JSON files will now respect the closest .editorconfig file.
  • Webhook payloads now include information about the running CI/CD environment.
  • Generator can now merge JSON/YAML template files.

What's next?​

Expect the following in the v0.20 release!

  • Continuation of remote caching work!
  • Performance improvements for output hydration.
  • Toolchain improvements.