Workflow and pipeline types in GitLab CI: one pipeline per commit

Part 5 gave every job a rules block, so each job now decides for itself whether it belongs in a pipeline. Push a commit to a branch that has an open merge request, though, and job-level rules can’t save you from what shows up in the pipelines tab: two pipelines for the same SHA. One is labeled a branch pipeline, the other a merge request pipeline. Both run the full suite, both burn runner minutes, and each reports its own status.

Neither one is a bug: both pipelines are doing exactly what the file, by omission, told them to do. GitLab has several pipeline types, one push can legitimately produce more than one of them, and nothing we’ve written so far says which ones this repo wants. That choice is the most consequential decision in the whole file, and it lives in a block we haven’t touched yet: workflow.

Why you get two pipelines

A push to a branch creates a branch pipeline. In that pipeline, CI_PIPELINE_SOURCE is "push" and CI_COMMIT_BRANCH holds the branch name. This is the only type we’ve used through parts 1–5.

If the branch also has an open merge request, GitLab can additionally create a merge request pipeline for the same commit, with CI_PIPELINE_SOURCE set to "merge_request_event". MR pipelines exist because they carry context that branch pipelines don’t: predefined variables like CI_MERGE_REQUEST_IID and the target branch, which MR-oriented tools rely on. They run on the contents of the source branch only: same code as the branch pipeline, different metadata.

An MR pipeline is only created if your configuration opts in: some job or workflow rule has to match CI_PIPELINE_SOURCE == "merge_request_event". The duplicate-pipeline problem appears exactly when your rules match both sources at once, typically a mix of jobs with MR-flavored rules and jobs with none. Every push to an MR branch then pays for two nearly identical pipelines.

workflow rules: the pipeline-level gate

Job rules answer “does this job run in this pipeline”. workflow: rules answers a question one level up: “does this pipeline get created at all”. It sits at the top of the file, is evaluated before any job, and uses the same if/changes/exists conditions as part 5, with one restriction: when can only be always or never. First matching rule wins, and if no rule matches, the pipeline doesn’t run.

The pattern GitLab’s own docs give for switching cleanly between the two types:

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
      when: never
    - if: $CI_COMMIT_BRANCH

Read top to bottom: an MR event always gets a pipeline; a branch push is suppressed when that branch has an open MR (CI_OPEN_MERGE_REQUESTS is only set in that case); a plain branch push still gets its branch pipeline. Result: MR pipelines while an MR is open, branch pipelines otherwise, never both.

A file without a workflow block means this decision is being made per job, by accident. The block belongs at the top of .gitlab-ci.yml, above the stages, where the next person reading the file sees the pipeline policy before any job definitions.

The pipeline type taxonomy

Six types cover almost everything you’ll meet. What distinguishes them is who triggers them and what code they test:

• Branch pipelines: triggered by a push, CI_PIPELINE_SOURCE == "push". Test the branch as pushed.
• Merge request pipelines: created for commits on a branch with an open MR, CI_PIPELINE_SOURCE == "merge_request_event". Test the source branch contents, with MR context attached.
• Merged results pipelines: an upgrade to MR pipelines (Premium tier): GitLab builds a temporary commit that merges source into target and tests that. If the merge would conflict, it falls back to a regular MR pipeline.
• Merge trains: a queue on top of merged results (also Premium): each MR in the train is tested against the target branch plus every MR ahead of it. A failing pipeline ejects its MR and restarts the pipelines behind it.
• Scheduled pipelines: cron-triggered, CI_PIPELINE_SOURCE == "schedule". Nightly E2E, cache warmers.
• Triggered pipelines: started by another pipeline or an API call. These are how repos talk to each other; part 7 is entirely about them.

The first four are alternatives: you pick a lane per repository. The last two are additive and coexist with whatever lane you picked. Merged results and merge trains also come with prerequisites: MR pipelines must already be configured in the file, and the project has to live on GitLab rather than an external repository mirror.

What we actually run

Two setups from production, sanitized, because they land on opposite answers.

The large monorepo runs branch pipelines only. The workflow block kills everything else:

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH

Every push produces exactly one pipeline, MR open or not. The appeal is operational: one pipeline type means one mental model for cache keys (part 2’s protected-branch split was confusing enough), one set of predefined variables, and CI that works from the very first push, before any MR exists.

The cost is that MR context is gone. CI_MERGE_REQUEST_IID doesn’t exist in a branch pipeline, and tools like Danger need to know which MR they’re commenting on. We pay for our choice with an API lookup: resolve the open MR from the branch name, then hand the tool the variable it expects.

danger:
  stage: quality
  rules:
    - if: $CI_COMMIT_BRANCH && $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
  script:
    - |
      MR_IID=$(curl --silent --header "PRIVATE-TOKEN: ${DANGER_API_TOKEN}" \
        "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests?source_branch=${CI_COMMIT_REF_NAME}&state=opened" \
        | jq -r '.[0].iid // empty')
    - |
      if [ -z "$MR_IID" ]; then
        echo "No open MR for ${CI_COMMIT_REF_NAME}, skipping"
        exit 0
      fi
    - export CI_MERGE_REQUEST_IID="$MR_IID"
    - npx danger ci

The separate microfrontend repos went the other way: merged results pipelines. Their MRs merge into a shared release flow where a semantic conflict (two MRs that each pass alone but break combined with the moved target branch) is expensive to discover after merge. Merged results catches that class of failure before the merge button, because the pipeline tests the merged state instead of the branch as it was forked.

The honest tradeoff: branch pipelines need no MR and no extra setup, but a green pipeline only proves the pre-merge state was fine; the target branch may have moved since. Merged results proves the thing you’re actually about to create, but it demands an MR-centric flow (no MR, no pipeline), a Premium license, and acceptance that a conflict silently downgrades the run to a plain MR pipeline. Both choices are defensible. Mixing them within one team’s repos is the part that hurt us: every context switch came with a “why is there no pipeline” or “why are there two” detour.

Cancel what a new push made obsolete

One pipeline per commit still stacks up when someone pushes four times in ten minutes. The first three pipelines are answering questions nobody is asking anymore.

interruptible: true marks a job as safe to cancel when its pipeline becomes redundant, and workflow:auto_cancel:on_new_commit sets the pipeline-wide policy:

workflow:
  auto_cancel:
    on_new_commit: interruptible
 
default:
  interruptible: true
 
deploy-prod:
  interruptible: false
  resource_group: production

With on_new_commit: interruptible, a new push cancels exactly the jobs marked interruptible (lint, tests, builds) while anything marked interruptible: false keeps running. The default policy, conservative, is blunter: it cancels the redundant pipeline only if no non-interruptible job has started yet; once one has, the whole old pipeline runs to completion. Setting interruptible: true under default: and opting deploys out individually is less error-prone than remembering to opt every new job in.

One deploy at a time

That resource_group: production line in the previous example solves the opposite problem: jobs that must never run concurrently, even across different pipelines. Two merges land close together, both pipelines reach the deploy job, and without coordination they race: the older pipeline can finish last and put stale code in production.

A resource group is a mutex: across all pipelines in the project, only one job holding resource_group: production runs at a time; the rest queue. Which queued job goes next is the process mode: unordered (the default), oldest_first for strict ordering, or newest_first/newest_ready_first to skip straight to the latest build when your deploys are idempotent. There’s no YAML keyword for the process mode; you change it per resource group through the API.

Where this leaves you

One commit now produces one deliberate pipeline of a type you chose, redundant runs cancel themselves, and deploys are serialized. Within a single repository, that’s the whole control story.

The next problem doesn’t fit inside one repository: a merge in the frontend repo needs to run E2E tests that live in a different repo, owned by a different team, with its own pipeline. Making one pipeline start (and wait for) another is part 7.

References

• workflow keyword
• CI/CD YAML reference: interruptible, workflow:auto_cancel
• Merge request pipelines
• Merged results pipelines
• Merge trains
• Resource groups