Docker usage
Using Docker to run your applications? Or build your artifacts? No worries, moon can be utilized with Docker, and supports a robust integration layer.
Looking to speed up your Docker builds? Want to build in the cloud? Give Depot a try!
Performance improvements
For the most part, everything should just work, but we have disabled caching and hashing in Docker containers and images for the following reasons:
- Avoids having to mount a volume for the
.git
directory. - Reduces the overall image size as we're avoiding creating tarballs.
- Ensures that builds are fresh and up-to-date.
Dockerfile
We're very familiar with how tedious Dockerfile
s are to write and maintain, so in an effort to
reduce this headache, we've built a handful of tools to make this process much easier. With moon,
we'll take advantage of Docker's layer caching and staged builds as much as possible.
With that being said, there's many approaches you can utilize, depending on your workflow (we'll document them below):
- Running
moon docker
commands before runningdocker run|build
commands. - Running
moon docker
commands within theDockerfile
. - Using multi-staged or standard builds.
- Something else unique to your setup!
What we're trying to avoid
Before we dive into writing a perfect Dockerfile
, we'll briefly talk about the pain points we're
trying to avoid. In the context of Node.js and monorepo's, you may be familiar with having to COPY
each individual package.json
in the monorepo before installing node_modules
, to effectively use
layer caching. This is very brittle, as each new application or package is created, every
Dockerfile
in the monorepo will need to be modified to account for this new package.json
.
Furthermore, we'll have to follow a similar process for only copying source files necessary for
the build or CMD
to complete. This is very tedious, so most developers simply use COPY . .
and
forget about it. Copying the entire monorepo is costly, especially as it grows.
As an example, we'll use moon's official repository. The Dockerfile
would look something like the
following.
FROM node:latest
WORKDIR /app
# Install moon binary
RUN npm install -g @moonrepo/cli
# Copy moon files
COPY ./.moon ./.moon
# Copy all package.json's and lockfiles
COPY ./packages/cli/package.json ./packages/cli/package.json
COPY ./packages/core-linux-arm64-gnu/package.json ./packages/core-linux-arm64-gnu/package.json
COPY ./packages/core-linux-arm64-musl/package.json ./packages/core-linux-arm64-musl/package.json
COPY ./packages/core-linux-x64-gnu/package.json ./packages/core-linux-x64-gnu/package.json
COPY ./packages/core-linux-x64-musl/package.json ./packages/core-linux-x64-musl/package.json
COPY ./packages/core-macos-arm64/package.json ./packages/core-macos-arm64/package.json
COPY ./packages/core-macos-x64/package.json ./packages/core-macos-x64/package.json
COPY ./packages/core-windows-x64-msvc/package.json ./packages/core-windows-x64-msvc/package.json
COPY ./packages/runtime/package.json ./packages/runtime/package.json
COPY ./packages/types/package.json ./packages/types/package.json
COPY ./package.json ./package.json
COPY ./yarn.lock ./yarn.lock
COPY ./.yarn ./.yarn
COPY ./.yarnrc.yml ./yarnrc.yml
# Install toolchain and dependencies
RUN moon docker setup
# Copy project and required files
COPY ./packages/types ./packages/types
COPY ./packages/runtime ./packages/runtime
# OR COPY . .
# Build the target
RUN moon run runtime:build
For such a small monorepo, this already looks too confusing!!! Let's remedy this by utilizing moon itself to the fullest!
Scaffolding the bare minimum
The first step in this process is to only copy the bare minimum of files necessary for installing
dependencies (Node.js modules, etc). This is typically manifests (package.json
), lockfiles
(yarn.lock
, etc), and any configuration (.yarnrc.yml
, etc).
This can all be achieved by the moon docker scaffold
command, which scaffolds a
skeleton of the repository structure, with only necessary files (the above). Let's update our
Dockerfile
usage.
- Standard
- Multi-staged
This assumes moon docker scaffold <project>
is ran outside of the Dockerfile
.
FROM node:latest
WORKDIR /app
# Install moon binary
RUN npm install -g @moonrepo/cli
# Copy workspace skeleton
COPY ./.moon/docker/workspace .
# Install toolchain and dependencies
RUN moon docker setup
#### BASE
FROM node:latest AS base
WORKDIR /app
# Install moon binary
RUN npm install -g @moonrepo/cli
#### WORKSPACE
FROM base AS workspace
WORKDIR /app
# Copy entire repository and scaffold
COPY . .
RUN moon docker scaffold <project>
#### BUILD
FROM base AS build
WORKDIR /app
# Copy workspace skeleton
COPY /app/.moon/docker/workspace .
# Install toolchain and dependencies
RUN moon docker setup
And with this, our dependencies will be layer cached effectively! Let's now move onto copying source files.
Copying necessary source files
The next step is to copy all source files necessary for CMD
or any RUN
commands to execute
correctly. This typically requires copying all source files for the project and all source files
of the project's dependencies... NOT the entire repository!
Luckily our moon docker scaffold <project>
command has already done this for us! Let's
continue updating our Dockerfile
to account for this, by appending the following:
- Standard
- Multi-staged
# Copy source files
COPY ./.moon/docker/sources .
# Run something
RUN moon run <project>:<task>
# Copy source files
COPY /app/.moon/docker/sources .
# Run something
RUN moon run <project>:<task>
If you need additional files for your commands to run successfully, you can manually use COPY
or
pass --include
to the scaffold command.
Pruning extraneous files
Now that we've ran a command or built an artifact, we should prune the Docker environment to remove
unneeded files and folders. We can do this with the moon docker prune
command, which
must be ran within the context of a Dockerfile
!
# Prune workspace
RUN moon docker prune
When ran, this command will do the following:
- Install production only dependencies for the projects that were scaffolded.
- Remove extraneous dependencies (
node_modules
) for unfocused projects.
Final result
And with this moon integration, we've reduced the original Dockerfile
of 35 lines to 18 lines, a
reduction of almost 50%. The original file can also be seen as O(n)
, as each new manifest requires
cascading updates, while the moon approach is O(1)
!
- Standard
- Multi-staged
FROM node:latest
WORKDIR /app
# Install moon binary
RUN npm install -g @moonrepo/cli
# Copy workspace skeleton
COPY ./.moon/docker/workspace .
# Install toolchain and dependencies
RUN moon docker setup
# Copy source files
COPY ./.moon/docker/sources .
# Run something
RUN moon run <project>:<task>
# Prune workspace
RUN moon docker prune
# Or CMD
#### BASE
FROM node:latest AS base
WORKDIR /app
# Install moon binary
RUN npm install -g @moonrepo/cli
#### WORKSPACE
FROM base AS workspace
WORKDIR /app
# Copy entire repository and scaffold
COPY . .
RUN moon docker scaffold <project>
#### BUILD
FROM base AS build
WORKDIR /app
# Copy workspace skeleton
COPY /app/.moon/docker/workspace .
# Install toolchain and dependencies
RUN moon docker setup
# Copy source files
COPY /app/.moon/docker/sources .
# Run something
RUN moon run <project>:<task>
# Prune workspace
RUN moon docker prune
# Or CMD
Troubleshooting
Process failure for git
Since moon executes git
commands under the hood, there are some special considerations to be aware
of when running moon within Docker. There's basically 2 scenarios to choose from:
- Add the
.git
folder to.dockerignore
, so that it's notCOPY
'd. moon will continue to work just fine, albeit with some functionality disabled. - Ensure that the
git
library is installed in the container, and copy the.git
folder withCOPY
. moon will work with full functionality, but it will increase the overall size of the image because of caching.