How To Maintain Your Velocity On a Monorepo?

Handling large codebases is a tough job, and it's even harder when dealing with monorepos. A monorepo is a single code repository that contains multiple projects or modules. It's a handy way to share code across teams and manage dependencies. However, managing a monorepo can be challenging, especially as the codebase grows. Thankfully, Mergify comes to the rescue with its merge queue feature, providing a solution for maintaining velocity on monorepos.

Monorepos: Why Are They Complex?

Monorepos have several benefits, such as easier dependency management, better code sharing, and simpler code refactoring. However, they also come with their fair share of challenges:

1. Dependency Management Issues

With so much code in one repository, it can be challenging to ensure that dependencies are correctly managed across all projects and services. Keeping track of version compatibility and avoiding conflicts becomes increasingly complex as the monorepo grows.

2. The Are Slower

Building and testing the entire monorepo can take a lot of time, especially as it grows. Even small changes to a specific module may require rebuilding and retesting the entire codebase, leading to reduced development velocity.

3. Conflicts Management

With many developers working on the same codebase, it can be challenging to manage conflicts and ensure that changes made in one module do not conflict with changes made in another. Coordinating development efforts and resolving conflicts manually can slow down the development process.

Why Use a Merge Queue?

To help overcome the challenges of managing monorepos, using a merge queue provides several benefits:

1. Minimized Conflict Risk and Streamlined Modifications

A merge queue ensures that changes made in one module do not conflict with changes made in another by managing the merge order and dependencies. Instead of merging pull requests independently, the merge queue determines the appropriate order of merging, reducing the risk of conflicts and errors.

2. Enhanced Efficiency through Automated Dependency Management

A merge queue automatically manages dependencies, making it easier for developers to work on their individual projects and modules. Instead of manually resolving dependencies, the merge queue takes care of ensuring that the correct versions of dependencies are used.

3. Accelerated Build and Test Times

By managing the merge order, a merge queue reduces build and test times by ensuring that changes that depend on each other are merged together. This approach eliminates the need for rebuilding and retesting the entire monorepo for each small change, leading to improved development velocity.

Using a merge queue for managing monorepos is a real game changer. Developers can set up a merge queue for their monorepo and customize it to fit their specific needs. For example, they can set up different merge queues for different projects or modules and specify the order in which changes should be merged.

Mergify for a Monorepo

When you submit a pull request to a monorepo, it can be put in one or multiple queues at the same time. Mergify will not merge the pull request until it has been validated by each queue it is in. This collaboration ensures that the pull request meets all the requirements of each queue.

You can also use queue rules to handle different projects in the monorepo that have different criteria for merging pull requests. Mergify allows you to define partition rules, which determine how pull requests are distributed among different queues based on the modified files.

Go Further with Partition Rules

Partition rules allow you to split your repository into smaller, more manageable parts called partitions. Each partition can run its own queue in parallel with other partitions. Partitions are independent of each other, meaning that changes queued into one partition will not affect the others.

How Does It Work?

When a pull request is submitted to the monorepo, it can be queued into one or multiple partitions simultaneously. Each partition can represent a subproject inside your monorepo and splitting your repository into partition can be done based on any criteria: folders and files modified, impacted service, programming language, CI run, or even the pull request author if you wished.

Mergify will wait for the pull request to be validated in each partition before it can be merged. This allows the different partitions to work together to ensure that the pull request meets all the necessary criteria for each partition.

If a pull request is queued in several partitions and the queue checks fail in at least one partition, the pull request will be reported as not mergeable. To be merged, the pull request needs to fulfill all the requirements of all the partitions it is queued in.

Partition rules can also be used to handle different projects in a monorepo that have different criteria for merging pull requests. This can be achieved by replicating the partition rules logic inside the queue rules merge_conditions and by using the attribute queue-partition-name.

A partition rule takes the following parameters:

Use Case

If a pull request contains a modification on any file in the folder projectA/ it will be added to the partition projectA, if it contains a modification on any file in the folder projectB/ then it will be added to the partition projectB, if it contains a modification in both folder projectA/ and projectB/, then it will be added to both partitions.

If none of the two partition's rules matches, then the pull request will be added to both partitions. The queue_rules determines in which queue in the partition(s) the pull request will be added.

Here is a table representing the partition and queues with the code below:

shared:
    priority_rules: &priority_rules
        - name: hotfix PR detected
          conditions:
            - label=hotfix
          priority: high
        - name: lowprio PR detected
          conditions:
            - author=dependabot[bot]
          priority: low

pull_request_rules:
   - name: queue PR with queue label
     conditions:
       - label=queue
     actions:
       queue:

partition_rules:
  - name: projectA
    conditions:
      - files~=^projectA/

  - name: projectB
    conditions:
      - files~=^projectB/

queue_rules:
  - name: hotfix
    priority_rules: *priority_rules
    routing_conditions:
      - label=hotfix
    merge_conditions:
      - or:
        - and:
          - queue-partition-name=projectA
          - check-success=ciA
        - and:
          - queue-partition-name=projectB
          - check-success=ciB

  - name: default
    priority_rules: *priority_rules
    merge_conditions:
      - or:
        - and:
          - queue-partition-name=projectA
          - check-success=ciA
        - and:
          - queue-partition-name=projectB
          - check-success=ciB

Conclusion

Managing a monorepo poses several challenges such as complex dependency management, slow build and test times, and the difficulty of handling conflicts and overlapping changes.

However, a merge queue can provide a solution by reducing the risk of conflicts, increasing efficiency with automatic dependency management, and improving development velocity by optimizing the merge order. Mergify is offering a powerful merge queue feature that streamlines monorepo management.

With Mergify, developers can customize merge queues, implement partition rules, and ensure smoother collaboration across projects or modules.

To experience the benefits firsthand, request a demo today and unlock the full potential of managing your monorepo effectively.