Loose Coupling in Monorepo Pipelines

Very special interest, how to create a loosely coupled CI/CD pipeline in GitHub for monorepos with an “undocumented” feature.

The problem

We have a monorepo with Turborepo and Next.js with multiple zones.
Each zone, or micro-frontend, has its own set of tests and tasks. In the project there are also common packages that are used by all zones.
Whenever a PR (pull request) is opened and a common package has been changed, all workflows should run. If only files in a specific zone are affected, only the zone specific workflow should run.

The common workflow

This workflow is perfectly fine and common things like linting changed files could live here.

name: Pull Request Workflow Common Checks

on:
  pull_request:
    branches:
      - main

jobs: ...

The zone specific workflow(s)

First approach

How can the pipeline be triggered when there are changes in a specific zone?

Since each zone is in its own folder, we can target specific paths.

Let’s check the docs.
“Using filters to target specific paths for pull request or push events” in the github docs

You can not use paths and paths-ignore to filter the same event in a single workflow. If you want to both include and exclude path patterns for a single event, use the paths filter prefixed with the ! character to indicate which paths should be excluded.

name: Pull Request Workflow Zone A

on:
  pull_request:
    branches:
      - main
    paths:
      - '**'
      - '!apps/zone-b/**'
      - '!apps/zone-c/**'

jobs: ...

Lets examine pull-request-zone-a.yml. Whenever something changes in any folder other than apps/zone-b/** or apps/zone-c/**, this workflow is triggered.
Changes in common-packages would trigger all “zone-workflows”, there is a yaml file for each zone.
Changes in apps/zone-a would only trigger the workflow pull-request-zone-a.yml.
Changes in apps/zone-b would only trigger the workflow pull-request-zone-b.yml.

This yaml file exists for each zone. Every time we add a new zone, we need to add an exclusion in each yaml file.
If we change the name of the folder for the application, we need to update each yaml file.
We may need to document this and so on … Each workflow is linked to all zones, even if it was only written for one zone.

name: Pull Request Workflow Zone B

on:
  pull_request:
    branches:
      - main
    paths:
      - '**'
      - '!apps/zone-a/**'
      - '!apps/zone-c/**'

jobs: ...

name: Pull Request Workflow Zone C

on:
  pull_request:
    branches:
      - main
    paths:
      - '**'
      - '!apps/zone-a/**'
      - '!apps/zone-b/**'

jobs: ...

Loosely coupled approach

As it turns out, you can use paths-ignore with ! to include and exclude paths. This is undocumented, but works fine.
As always with undocumented things, this may stop working one day.

name: Pull Request Workflow Zone A

on:
  pull_request:
    branches:
      - main
    paths-ignore:
      - 'apps/**'
      - '!apps/zone-a/**'

jobs: ...

This workflow ignores all folders in apps/**, i.e. all zones. In the next line, the folder for zone a, apps/zone-a/**, is added.

Now only the zone for which the workflow is intended is referenced.

The name of some zone folders has changed?
Are there new zones?
There is nothing to do in the workflow for the other zones!

• loosely coupled
• “undocumented”
• needed in monorepos

bang bang !! ;-)

Questions, comments, ideas?

Just open an issue on the repo or get in contact via mastodon