Skip to main content

Docker usage

Using Docker to run your applications? Or build your artifacts? No worries, moon can be utilized with Docker, and supports a robust integration layer.

success

Looking to speed up your Docker builds? Want to build in the cloud? Give Depot a try!

Performance improvements​

For the most part, everything should just work, but we have disabled caching and hashing in Docker containers and images for the following reasons:

  • Avoids having to mount a volume for the .git directory.
  • Reduces the overall image size as we're avoiding creating tarballs.
  • Ensures that builds are fresh and up-to-date.

Dockerfile​

We're very familiar with how tedious Dockerfiles are to write and maintain, so in an effort to reduce this headache, we've built a handful of tools to make this process much easier. With moon, we'll take advantage of Docker's layer caching and staged builds as much as possible.

With that being said, there's many approaches you can utilize, depending on your workflow (we'll document them below):

  • Running moon docker commands before running docker run|build commands.
  • Running moon docker commands within the Dockerfile.
  • Using multi-staged or standard builds.
  • Something else unique to your setup!

What we're trying to avoid​

Before we dive into writing a perfect Dockerfile, we'll briefly talk about the pain points we're trying to avoid. In the context of Node.js and monorepo's, you may be familiar with having to COPY each individual package.json in the monorepo before installing node_modules, to effectively use layer caching. This is very brittle, as each new application or package is created, every Dockerfile in the monorepo will need to be modified to account for this new package.json.

Furthermore, we'll have to follow a similar process for only copying source files necessary for the build or CMD to complete. This is very tedious, so most developers simply use COPY . . and forget about it. Copying the entire monorepo is costly, especially as it grows.

As an example, we'll use moon's official repository. The Dockerfile would look something like the following.

FROM node:latest

WORKDIR /app

# Install moon binary
RUN npm install -g @moonrepo/cli

# Copy moon files
COPY ./.moon ./.moon

# Copy all package.json's and lockfiles
COPY ./packages/cli/package.json ./packages/cli/package.json
COPY ./packages/core-linux-arm64-gnu/package.json ./packages/core-linux-arm64-gnu/package.json
COPY ./packages/core-linux-arm64-musl/package.json ./packages/core-linux-arm64-musl/package.json
COPY ./packages/core-linux-x64-gnu/package.json ./packages/core-linux-x64-gnu/package.json
COPY ./packages/core-linux-x64-musl/package.json ./packages/core-linux-x64-musl/package.json
COPY ./packages/core-macos-arm64/package.json ./packages/core-macos-arm64/package.json
COPY ./packages/core-macos-x64/package.json ./packages/core-macos-x64/package.json
COPY ./packages/core-windows-x64-msvc/package.json ./packages/core-windows-x64-msvc/package.json
COPY ./packages/runtime/package.json ./packages/runtime/package.json
COPY ./packages/types/package.json ./packages/types/package.json
COPY ./package.json ./package.json
COPY ./yarn.lock ./yarn.lock
COPY ./.yarn ./.yarn
COPY ./.yarnrc.yml ./yarnrc.yml

# Install toolchain and dependencies
RUN moon docker setup

# Copy project and required files
COPY ./packages/types ./packages/types
COPY ./packages/runtime ./packages/runtime
# OR COPY . .

# Build the target
RUN moon run runtime:build

For such a small monorepo, this already looks too confusing!!! Let's remedy this by utilizing moon itself to the fullest!

Scaffolding the bare minimum​

The first step in this process is to only copy the bare minimum of files necessary for installing dependencies (Node.js modules, etc). This is typically manifests (package.json), lockfiles (yarn.lock, etc), and any configuration (.yarnrc.yml, etc).

This can all be achieved by the moon docker scaffold command, which scaffolds a skeleton of the repository structure, with only necessary files (the above). Let's update our Dockerfile usage.

This assumes moon docker scaffold <project> is ran outside of the Dockerfile.

FROM node:latest
WORKDIR /app

# Install moon binary
RUN npm install -g @moonrepo/cli

# Copy workspace skeleton
COPY ./.moon/docker/workspace .

# Install toolchain and dependencies
RUN moon docker setup

And with this, our dependencies will be layer cached effectively! Let's now move onto copying source files.

Copying necessary source files​

The next step is to copy all source files necessary for CMD or any RUN commands to execute correctly. This typically requires copying all source files for the project and all source files of the project's dependencies... NOT the entire repository!

Luckily our moon docker scaffold <project> command has already done this for us! Let's continue updating our Dockerfile to account for this, by appending the following:

# Copy source files
COPY ./.moon/docker/sources .

# Run something
RUN moon run <project>:<task>
info

If you need additional files for your commands to run successfully, you can manually use COPY or pass --include to the scaffold command.

Pruning extraneous files​

Now that we've ran a command or built an artifact, we should prune the Docker environment to remove unneeded files and folders. We can do this with the moon docker prune command, which must be ran within the context of a Dockerfile!

# Prune workspace
RUN moon docker prune

When ran, this command will do the following:

  • Install production only dependencies for the projects that were scaffolded.
  • Remove extraneous dependencies (node_modules) for unfocused projects.

Final result​

And with this moon integration, we've reduced the original Dockerfile of 35 lines to 18 lines, a reduction of almost 50%. The original file can also be seen as O(n), as each new manifest requires cascading updates, while the moon approach is O(1)!

FROM node:latest
WORKDIR /app

# Install moon binary
RUN npm install -g @moonrepo/cli

# Copy workspace skeleton
COPY ./.moon/docker/workspace .

# Install toolchain and dependencies
RUN moon docker setup

# Copy source files
COPY ./.moon/docker/sources .

# Run something
RUN moon run <project>:<task>

# Prune workspace
RUN moon docker prune

# Or CMD

Troubleshooting​

Process failure for git​

Since moon executes git commands under the hood, there are some special considerations to be aware of when running moon within Docker. There's basically 2 scenarios to choose from:

  1. Add the .git folder to .dockerignore, so that it's not COPY'd. moon will continue to work just fine, albeit with some functionality disabled.
  2. Ensure that the git library is installed in the container, and copy the .git folder with COPY. moon will work with full functionality, but it will increase the overall size of the image because of caching.

Supporting node:alpine images​

If you're trying to use the node:alpine image with moon's integrated toolchain, you'll need to set the MOON_TOOLCHAIN_FORCE_GLOBALS environment variable in the Docker image to disable moon's toolchain. This is required as Node.js does not provide pre-built binaries for the Alpine target, so installing the Node.js toolchain will fail.

FROM node:alpine

ENV MOON_TOOLCHAIN_FORCE_GLOBALS=1