Skip to main content

Remote caching

Is your CI pipeline running slower than usual? Are you tired of running the same build over and over although nothing has changed? Do you wish to reuse the same local cache across other machines and environments? These are just a few scenarios that remote caching aims to solve.

Remote caching is a system that shares artifacts to improve performance, reduce unnecessary computation time, and alleviate resources. It achieves this by uploading hashed artifacts to a cloud storage provider, like AWS S3 or Google Cloud, and downloading them on demand when a build matches a derived hash.

Requirements​

  • moon >= 0.23.3

Sign up for an account​

Remote caching requires a moonrepo.app account, powered by our moonbase service. Creating an account requires a GitHub account. In the future we'll support additional OAuth providers.

Create an organization​

A moonbase organization can represent a company, team, group, or personal workspace, and is required for housing repositories, projects, artifacts, and more. Begin by creating an organization from the dashboard.

The "Region" field is very important for remote caching, as it configures which S3 region to store artifacts in. This cannot be changed after creation, so choose wisely!

Create a repository​

A moonbase repository is a direct relationship to a codebase repository, and is necessary for linking artifacts to the appropriate tasks. From an organization's page, you can create a repository.

The "Slug" field must match your repository's "remote/repo" naming exactly, as we compare these values for authentication purposes.

Enabling remote caching​

Once you have a moonrepo.app account, an organization, and repository, you can enable remote caching in your CI pipeline. From a moonbase organization page, you can view your API key in the actions menu.

In a repository's CI environment, set the following environment variable.

MOONBASE_SECRET_KEY=<secret key>

moon will automatically authenticate the remote caching service when credentials in the environment exist. Furthermore, the repository in question must be within the moonbase organization, and the repository slug (<owner>/<repo>) must match.

FAQ​

What is an artifact?​

In the context of moon and remote caching, an artifact is a tar archive (.tar.gz) that contains all outputs of a task, as well as the stdout and stderr of the task that generated the outputs. Artifacts are uniquely identified by the moon generated hash.

Do I have to use remote caching?​

No, remote caching is optional. It's intended purpose is to store long lived build artifacts to speed up CI pipelines, and optionally local development. For the most part, moon ci does a great job of only running what's affected in pull requests, and is a great starting point.

Does remote caching store source code?​

No, remote caching does not store source code. It stores the outputs of a task, which is typically built and compiled code. To verify this, you can inspect the tar archives in .moon/cache/outputs.

Does moon collect any personally identifiable information?​

No, moon does not collect any PII as part of the remote caching process.

However, to use remote caching, you must create a moonrepo account in which we require an email address, and information about your organization and repository.

Are artifacts encrypted?​

Yes! We use AWS's built-in SSE-S3 encryption for all S3 buckets.