Nx + Gitlab CI/CD

Working with Gitlab and Nx might not be as straightforward as the official guide. It assumes you can work with merge_request_event and your current flow won’t be disturbed. On top of that it can lead to duplicated pipelines and make your pipelines tricky to optimize.

Additionally, I have also noticed that Gitlab won’t show status from multiple pipelines in the UI! This is pretty bad UX that can lead to introducing regression to your main branch or even your production system.

What’s wrong with the official setup?

Example: You just opened a PR and pushed a new commit to your branch. In jobs definition you have two events:

• “push” - run job1
• “merge_request_event - run job2 (eg. nx affected command)

If one of the jobs fails, this is what you might end up seeing:

Failed status is nowhere to be find. The only way to see it is to open “Pipelines” tab and look for it there.

Adjusting to “push” pipelines

Working with “push” was a constrain coming from Centralized CI/CD at the company. It standardizes the way of running CI jobs across dozens of projects. It makes infra consistent and easier to debug / adjust. Adding new job type is as easy as extending only one repo.

Main difference between “merge_request_event” and “push” is that some environment variables are not available. Predefined variables like CI_MERGE_REQUEST_DIFF_BASE_SHA are simply not available and you need a workaround in order to get what you want. Luckily you can always fallback to native git command and easily get it.

Here is a code snippet how I achieved it.

# .gitlab-ci.yaml

stages:
  # ...
  - pre-tests
  - tests
  # ...

npm-install:
  stage: pre-tests
  cache: 
    key: ${CI_PROJECT_PATH_SLUG}-${CI_COMMIT_BRANCH}
    paths:
      - node_modules/
    policy: push
  rules:
    - if: $CI_PIPELINE_SOURCE == "push"
  script:
    - node -v
    - npm -v
    - npm ci

extract-sha:
  stage: pre-tests
  image: alpine:3.18.2
  rules:
    - if: $CI_PIPELINE_SOURCE == "push"
  before_script:
    - apk add --update git
    - git fetch origin $CI_DEFAULT_BRANCH
  script:
    - BASE_SHA=$([ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ] && echo "$CI_COMMIT_BEFORE_SHA" || git merge-base HEAD origin/$CI_DEFAULT_BRANCH)
    - echo 'BASE_SHA' ${BASE_SHA}
    - echo "export BASE_SHA=$BASE_SHA" > $CI_PROJECT_DIR/sha.txt
  artifacts:
    when: always
    paths:
      - sha.txt
    expire_in: 1 day

.npm-task:
  stage: tests
  needs:
    - job: extract-sha
    - job: npm-install
      optional: true
  rules:
    - if: $CI_PIPELINE_SOURCE == "push"
  before_script:
    # nx affected needs git, which alpine doesn't have. So it's added here.
    - apk add --update git
    - source $CI_PROJECT_DIR/sha.txt
    - NX_BASE=$BASE_SHA
    - NX_HEAD=$CI_COMMIT_SHA
    - echo 'NX_BASE' ${NX_BASE}
    - echo 'NX_HEAD' ${NX_HEAD}

affected/lint:
  extends: .npm-task
  script: npx nx affected --target=lint --base=$NX_BASE --head=$NX_HEAD

affected/test:
  extends: .npm-task
  script: npx nx affected --target=test --base=$NX_BASE --head=$NX_HEAD