Continuous integration (CI)
All companies and projects rely on continuous integration (CI) to ensure high quality code and to
avoid regressions. Because this is such a critical piece of every developer's workflow, we wanted to
support it as a first-class feature within moon, and we do just that with the
moon ci
command.
How it works
The ci
command does all the heavy lifting necessary for effectively running jobs. It achieves this
by automatically running the following steps:
- Determines touched files by comparing the current HEAD against a base.
- Determines all targets that need to run based on touched files.
- Additionally runs affected targets dependencies and dependents.
- Generates an action and dependency graph.
- Installs the toolchain, Node.js, and npm dependencies.
- Runs all actions within the graph using a thread pool.
- Displays stats about all passing, failed, and invalid actions.
Configuring tasks
By default, all tasks run in CI, as you should always be building, linting, typechecking, testing,
so on and so forth. However, this isn't always true, so this can be disabled on a per-task basis
through the runInCI
or local
options.
tasks:
dev:
command: 'webpack server'
options:
runInCI: false
# Or
local: true
This option must be set to false for tasks that spawn a long-running or never-ending process, like
HTTP or development servers. To help mitigate this, tasks named dev
, start
, or serve
are false
by default. This can be easily controlled with the local
setting.
Integrating
Although moon has an integrated toolchain, we still require Node.js and dependencies to be installed beforehand, as moon is currently shipped as an npm package. This is unfortunate and we're looking into other distribution channels.
With that being said, the following examples can be referenced for setting up moon and its CI
workflow in popular services. The examples assume a
package script named moon
and are using Yarn 3, but feel
free to replace with your chosen setup.
- GitHub
- Buildkite
- CircleCI
- TravisCI
name: 'Pipeline'
on:
push:
branches:
- 'master'
pull_request:
jobs:
ci:
name: 'CI'
runs-on: 'ubuntu-latest'
steps:
- uses: 'actions/checkout@v3'
with:
fetch-depth: 0
- uses: 'actions/setup-node@v3'
with:
cache: 'yarn'
- run: 'yarn install --immutable'
- run: 'yarn moon ci'
steps:
- label: 'CI'
commands:
- 'yarn install --immutable'
- 'yarn moon ci'
version: 2.1
orbs:
node: 'circleci/node@5.0.2'
jobs:
ci:
docker:
- image: 'cimg/base:stable'
steps:
- checkout
- node/install:
install-yarn: true
node-version: '16.13'
- node/install-packages:
check-cache: 'always'
pkg-manager: 'yarn-berry'
- run: 'yarn moon ci'
workflows:
pipeline:
jobs:
- 'ci'
language: node_js
node_js:
- 16
cache: yarn
script: 'moon ci'
Comparing revisions
By default the command will compare the current HEAD against a base revision, which is typically the
configured vcs.defaultBranch
(master, main, trunk, etc). Both
of these can be customized with the --base
and --head
options respectively.
$ moon ci --base <BRANCH> --head <SHA>
Parallelizing tasks
If your CI environment supports sharding across multiple jobs, then you can utilize moon's built in
parallelism by passing --jobTotal
and --job
options. The --jobTotal
option is an integer of
the total number of jobs available, and --job
is the current index (0 based) amongst the total.
When these options are passed, moon will only run affected targets based on the current job slice.
- GitHub
- Buildkite
- CircleCI
- TravisCI
GitHub Actions do not support native parallelism, but it can be emulated using it's matrix.
# ...
jobs:
ci:
# ...
strategy:
matrix:
index: [0, 1]
steps:
# ...
- run: 'yarn moon ci --job ${{ matrix.index }} --jobTotal 2'
# ...
steps:
- label: 'CI'
parallelism: 10
commands:
# ...
- 'yarn moon ci --job $$BUILDKITE_PARALLEL_JOB --jobTotal $$BUILDKITE_PARALLEL_JOB_COUNT'
# ...
jobs:
ci:
# ...
parallelism: 10
steps:
# ...
- run: 'yarn moon ci --job $CIRCLE_NODE_INDEX --jobTotal $CIRCLE_NODE_TOTAL'
TravisCI does not support native parallelism, but it can be emulated using it's matrix.
# ...
env:
global:
- TRAVIS_JOB_TOTAL=2
jobs:
- TRAVIS_JOB_INDEX=0
- TRAVIS_JOB_INDEX=1
script: 'moon ci --job $TRAVIS_JOB_INDEX --jobTotal $TRAVIS_JOB_TOTAL'
Your CI environment may provide environment variables for these 2 values.
Caching artifacts
When a CI pipeline reaches a certain scale, its run times increase, tasks are unnecessarily ran, and build artifacts are not shared. To combat this, we support remote caching, a mechanism where we store build artifacts in the cloud, and sync these artifacts to machines on demand.
Remote caching is powered by our moonbase service. Start using today for free!
Reporting run results
If you're using GitHub Actions as your CI provider, we suggest using our
moonrepo/run-report-action
. This
action will report the results of a moon ci
run to a pull request as a comment
and workflow summary.
# ...
jobs:
ci:
name: 'CI'
runs-on: 'ubuntu-latest'
steps:
# ...
- run: 'yarn moon ci'
- uses: 'moonrepo/run-report-action@v1'
if: success() || failure()
with:
access-token: ${{ secrets.GITHUB_TOKEN }}
The report looks something like the following:
