In this release, we've taken the next step in modernizing our action pipeline, by rewriting the dependency graph.
Hello action graph, goodbye dependency graph
For the past few months, we've been working on a rewrite of our action pipeline, which consists of the project graph, dependency graph, task executor, process pipeline, and more. It's a slow process, with many different pieces that must land in sequence, but we're almost done. The next step in this process is the introduction of the new action graph, which replaces the previous dependency graph.
For the most part, the graphs work in a similar fashion, but since we rewrote it from the ground up, we were able to resolve any discrepancies and performance issues. The biggest changes between the new and old graphs are:
- All actions now depend on the
SyncWorkspaceaction, instead of this action running arbitrarily.
- Cleaned up dependency chains between actions, greatly reducing the number of nodes in the graph.
RunTask, including interactive and persistent variants.
- And lastly, we ditched our batched task approach for a ready queue. Continue reading for more information!
A new performant thread pool
In the old dependency graph, when we'd execute a task, we'd order the graph topologically and then group actions into batches (or buckets) based on their dependency chains. Batches would then be executed in order within the thread pool. This approach worked well, but had one major flaw: it wasn't as performant as could be. For example, if our thread pool size was 12, and a batch only had 2 tasks in it, what were the other 10 threads doing? Absolutely nothing. They were sitting idly, waiting for a task.
And now with the new action graph, we take full advantage of all threads in the pool. Instead of the batched approach above, we now use a topological task-ready queue, where a thread without work (or is waiting for work) can poll the graph for a new task to run. A task is considered ready to run if it either has no dependencies, or all of its dependencies (in the chain) have been ran.
For large graphs, this should result in a significant performance improvement!
Automatic dependency linking (breaking)
In v1.17, we changed the scope from "peer" to "build" to reduce friction.
Because of these graph changes, we do have a minor "breaking change". Tasks that depend (via
on other tasks from arbitrary projects (the parent project doesn't implicitly or explicitly depend
on the other project), not including the root-level project, will now automatically mark that other
project as a "peer" dependency (if not already configured with
dependsOn). For example, "b"
becomes a peer dependency for "a".
Now internally becomes:
- id: 'b'
If you'd prefer this dependency to not be a peer, you can explicitly configure it with a different scope. For Node.js projects, the "build" scope can be used as a no-op replacement.
- id: 'b'
scope: 'build' # production, development
We're marking this as a breaking change as this could subtly introduce cycles in the project graph
that weren't present before, and for Node.js projects, this may inject
this change was necessary to ensure accurate dependency chains in the graph.
moonrepo/setup-toolchain GitHub action
We've begun a process to deprecate the moonrepo/setup-moon-action and moonrepo/setup-proto GitHub actions, and instead, combine and replace them with a new moonrepo/setup-toolchain action. Why a new action instead of fixing the others?
The biggest problem was that both previous actions shared about 90% of the same code, but were slightly different in how they installed the binaries and cached the toolchain. It was was also confusing for consumers to understand and know which action to use (because they shouldn't be used together).
To remedy this, we're prototyping the new moonrepo/setup-toolchain action, which has been working quite well. It aims to solve the following:
protoglobally so that installed tools can also be executed globally.
- Conditionally installs
moonglobally if the repository is using moon (attempts to detect a
- Caches the toolchain (
~/.proto) so subsequent runs are faster.
.moon/toolchain.ymlfiles to generate a unique cache key.
- Cleans the toolchain before caching to remove unused or stale tools.
- Can auto-install tools when used.
- uses: actions/checkout@v4
- - uses: moonrepo/setup-moon-action@v1
+ - uses: moonrepo/setup-toolchain@v0
Now supported in Railway
If you're a big fan of Railway (like we are), and you're deploying a Node.js backed application, then you'll be happy to hear that Railway now officially and natively supports moon! We spent some time over the past month integrating moon support into their Nixpacks architecture.
To make use of this, set the
NIXPACKS_MOON_APP_NAME environment variable to the name of your moon
project that you want to be deployed. This will then automatically run
moon run <app>:build and
moon run <app>:start respectively. To customize the task names, you can set the
NIXPACKS_MOON_START_TASK environment variables.
This is currently only supported for Node.js projects, but will be expanded to other languages in the future!
View the official release for a full list of changes.
- Added a
- Added a
- Added the ability to skip non-
RunTaskactions using environment variables.
- Deprecated the