Monorepo pipelines in GitLab CI: selective everything

Part 8 ended with a shared CI library: ten near-identical repos, one pipeline definition, drift solved. A monorepo is the other answer to the same problem. Instead of ten repos including one config, one repo contains the ten apps: in our case, eight deployable apps and roughly fifty internal packages they share.

The economics flip, though. In separate repos, every pipeline is scoped by construction: a push to the storefront repo can only ever build the storefront. In a monorepo, a typo fix in docs/ lives in the same repo as everything else, and a naive pipeline happily builds, lints, and tests all of it. At fifty packages, that’s a forty-minute pipeline for a one-line commit.

The whole game becomes selection: run exactly what the commit affects and nothing else. Each section below is one layer of that selection.

Only run what changed

Part 5 introduced rules:changes as one condition among many. In a monorepo it becomes the load-bearing wall, and its default behavior has a sharp edge you need to understand first. What “changed” means depends on the pipeline type: in merge request pipelines, GitLab compares against the MR’s target branch; in branch pipelines, against the previous commit on the branch; and for new branches, or pipelines with no Git push event at all (tags, schedules, manual runs), changes always evaluates to true and the job runs.

That previous-commit comparison bites hard. Push a commit touching apps/storefront, watch the deploy job fail, push a follow-up that only fixes a typo in the docs, and the new pipeline, comparing against the previous commit, sees zero storefront changes. The deploy job silently disappears from the very pipeline that was supposed to retry it. rules:changes:compare_to fixes this by pinning the comparison base:

build-storefront:
  script: pnpm turbo run build --filter=storefront...
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    - changes:
        paths:
          - apps/storefront/**/*
          - packages/**/*
          - pnpm-lock.yaml
        compare_to: $CI_DEFAULT_BRANCH

With compare_to, every pipeline on a feature branch sees the branch’s full diff against main, so follow-up commits can’t lose jobs. The first rule exists because main compared against itself is an empty diff: on the default branch we skip change detection and run everything.

Two version-sensitive notes. For years compare_to couldn’t expand CI/CD variables, and workarounds for hardcoded branch names circulated in every monorepo thread; that limitation is gone: since GitLab 17.2, variables like $CI_DEFAULT_BRANCH work fine (issue #369916 is closed). And compare_to behaves unexpectedly in merged results pipelines, where the comparison base is an internal commit GitLab creates; our monorepo runs branch pipelines only (the choice part 6 walked through), which sidesteps that entirely. Two practical limits to keep in mind: a rules:changes section accepts at most 50 patterns, and paths are matched by exact string comparison, so ./apps/storefront matches nothing.

One anchor per app

Eight apps, each with a build job and a deploy job, and every one of those jobs needs the same three facts: which paths belong to the app, which package to filter, where to deploy. Copy-pasting those facts into sixteen rules blocks recreates the drift problem part 8 was about, only inside one file. A YAML variable anchor puts each app’s facts in one block:

.storefront: &storefront
  APP_NAME: storefront
  APP_PATH: apps/storefront
  DEPLOY_TARGET: storefront-production
 
.app-job:
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    - changes:
        paths:
          - $APP_PATH/**/*
          - packages/**/*
          - pnpm-lock.yaml
        compare_to: $CI_DEFAULT_BRANCH

Every job for an app extends the shared skeleton and points at the app’s anchor:

build-storefront:
  extends: .app-job
  variables: *storefront
  script: pnpm turbo run build --filter="$APP_NAME..."
 
deploy-storefront:
  extends: .app-job
  stage: deploy
  variables: *storefront
  script: ./scripts/deploy.sh "$DEPLOY_TARGET"

This works because rules:changes paths expand CI/CD variables, including job-level ones. The rules:changes docs show exactly this pattern. Onboarding a ninth app is one anchor plus two four-line jobs; the change paths, the build filter, and the deploy target all flow from the same block, so they can’t disagree. One caveat: anchors don’t cross include boundaries, which is why the shared library in part 8 needed !reference. In a single-file monorepo config, anchors are the lighter tool.

The task runner already knows the graph

Change detection decides whether the storefront jobs run. It says nothing about what happens inside them. The storefront depends on maybe twenty of the fifty packages, some of those depend on each other, and tests need builds of their dependencies first. You do not want to express that graph in YAML, because it already exists in the repo’s package.json files. Our stack is pnpm workspaces plus Turborepo, where task dependencies are declared once:

{
  "tasks": {
    "build": { "dependsOn": ["^build"], "outputs": ["dist/**"] },
    "lint": {},
    "typecheck": { "dependsOn": ["^build"] },
    "test": { "dependsOn": ["lint", "typecheck", "^build"] }
  }
}

^build means “build my dependencies first”. With that in place, the CI script is a single command: pnpm turbo run test --filter=storefront..., where the ... suffix is pnpm’s filter syntax for “this package and everything it depends on”. That one command builds the affected packages in graph order, then lints, typechecks, and tests them, parallelizing wherever the graph allows. The pipeline picks the app; the task runner picks the work. When a developer adds a dependency between two packages, no YAML changes: the graph the task runner reads is the package.json they just edited.

CI stays dumb on purpose. Every job’s script is one filtered command, and all the intelligence about ordering and skipping lives in the tool that also runs on laptops, so local runs and CI runs can’t drift apart.

One build job feeds everything

Part 3’s build-once pattern returns at scale. Even with filters, eight app jobs each compiling the shared packages means the same packages/* graph compiles up to eight times per pipeline. One job at the front compiles it once and ships the output as artifacts:

validate-and-build:
  stage: build
  script:
    - pnpm install --frozen-lockfile
    - pnpm turbo run build --filter="./packages/*"
  artifacts:
    paths:
      - packages/*/dist
    expire_in: 3 hours
 
test-storefront:
  stage: test
  needs: [validate-and-build]
  script: pnpm turbo run test --filter=storefront...

Downstream jobs declare needs on the hub (part 4), get packages/*/dist extracted into their working tree, and Turborepo sees fresh outputs and skips those builds. The expire_in: 3 hours is deliberately short: these are intermediate outputs consumed minutes after they’re produced, and any retry within a working day still finds them. The number wasn’t always three hours; the honest-middle section below is about why it stopped being a week.

Four layers of cache

Selective execution and a build hub still leave repeated work between pipelines, and no single cache covers it. What we ended up with is layered:

• The pnpm store, keyed on the lockfile with a fallback key (the part 2 setup, unchanged at monorepo scale).
• Per-branch tool caches (ESLint’s cache, test-runner caches) keyed on $CI_COMMIT_REF_SLUG, so the second push to a branch only re-lints what changed.
• A per-app framework build cache (the .next/cache-style incremental directory), keyed per app and branch.
• The task runner’s remote cache: Turborepo hashes each task’s inputs and stores outputs in a shared HTTP cache, so a package unchanged since any previous run (on any branch, any runner) replays its build log instead of rebuilding.

test-storefront:
  cache:
    - key:
        files: [pnpm-lock.yaml]
      paths: [.pnpm-store]
      policy: pull
    - key: lint-$CI_COMMIT_REF_SLUG
      paths: [.eslintcache]

GitLab allows up to four cache entries per job, which is exactly enough here. The remote task cache is the interesting layer because it lives outside GitLab’s cache mechanism entirely: keyed by content hash rather than branch, it isn’t subject to the protected/non-protected cache split from part 2, so a package built on main genuinely warms feature branches.

The week the runners ran out of disk

All of this selection multiplied the number of jobs per pipeline, and a few months in, jobs started dying with ENOSPC: no space left on device. Random jobs, random projects, usually mid-install. The runners were shared across teams, so every team saw it and no team could reproduce it.

The measurement was the uncomfortable part: each of our jobs was leaving roughly 11 GB behind on the runner host. Extracted dist/ artifacts, per-app framework build directories, unpacked caches, and the executor reuses working directories between jobs for speed, so the leftovers accumulated until the disk filled and every job scheduled on that host died, whoever it belonged to.

Three fixes, none clever:

default:
  after_script:
    - rm -rf apps/*/dist packages/*/dist .turbo node_modules/.cache
 
variables:
  GIT_CLEAN_FLAGS: -ffdx

The after_script cleanup runs even when the job fails, so heavy outputs never outlive the job that made them. GIT_CLEAN_FLAGS controls the git clean the runner performs at job start; -ffdx wipes untracked and ignored leftovers from whatever ran in that directory before. The runner configuration docs cover it, and the flag is also a lever people set to none to “save time”, which is how residue gets immortal. And artifact expiry went from one week to three hours, because build outputs consumed within minutes were sitting on storage for seven days, multiplied by every pipeline and every app. The disk graphs went flat and stayed flat.

Where this leaves you

The pipeline now detects which apps a commit touches, delegates the actual work to a task runner that knows the dependency graph, builds shared code once, and cleans up after itself. Building and testing selectively is, at this point, a solved problem.

Publishing is not. Those fifty packages also version and release from this same pipeline, and a release job that pushes a version-bump commit back to the repo triggers a brand-new pipeline, which wants to release again. Part 10 is about shipping packages from CI without retriggering yourself.

References

• rules:changes
• rules:changes:compare_to
• Specify when jobs run with rules
• pnpm — filtering
• Turborepo — configuring tasks
• GitLab Runner — configure runners (GIT_CLEAN_FLAGS)