Testing your Pull Requests Smartly with a Merge Queue

Testing your Pull Requests Smartly with a Merge Queue

Mathieu Poissard

Before we start, let's agree: all your pull requests must be tested. If you don't agree with this first point, reading this article may seem like a waste of your time.

Unless you're ready to change your mind?

While it's important to test all the changes you wish to integrate into your code base, it's important to test them intelligently, and at the right time.

Good news: you can do this with a merge queue!

Intelligent PR testing: CI savings and exacerbated velocity

To test your pull requests as efficiently as possible, you should do it in two stages:

  • tests to validate your PR and get it into the queue,
  • tests to merge your pull request into main.

Split your Test Suite

First, divide your test suite into two distinct categories, differentiating between unit tests and more global tests such as functional or end-to-end tests.

Queue Entry Conditions and Unit Tests

For a pull request to be included in the merge queue, it must be valid and of good quality. After all, why attempt to merge poor-quality changes?

With this in mind, you can define entry conditions. In general, we recommend choosing the requirements:

  • More than 1 approval,
  • Unit Test CI.

If these checks pass, then your PR is validated and can enter the queue. Here's what your YAML file might look like.

queue_rules:
  - name: default
    queue_conditions:
      - check-success = mycijob
      - "#approved-reviews-by >= 1"

Here, the pull request enters the queue called default if the CI named mycijob has passed. Moreover,  to enter the queue, the PR needs to be approved by at least one contributor.

Pull Requests Rules and Conditions to Merge

Your PR is now in the merge queue. It will make its way to the top of the queue. This moment corresponds to the merge momentum. The PR has just been updated; it's ready to be tested and merged.

So, it's at this precise moment that you should launch all your integration tests. Not before (and certainly not after!)

You can define new merge conditions for your pull request, in other words, new checks, corresponding to the CI job of your functional and end-to-end tests.

Here's how it might look.

queue_rules:
  - name: default
    queue_conditions:
      - check-success = mycijob
      - "#approved-reviews-by >= 1"
    merge_rules:
      - check-success = mycijob-extra

The merge_rules define the condition required to merge a pull request once it is in the merge queue. Here, the PR will be merged if the CI called "mycijob-extra" has passed successfully.

Note: Mergify injects all branch protection settings defined by GitHub, so there's no need to repeat them.

Why Should You Test Your PR this Way?

Following this strategy, each pull request is fully tested, but above all intelligently, by launching the right type of test at the right time to match your standards and workflows.

But the coolest thing about all this is the benefits you'll reap. Using this testing strategy for your pull requests ensures that:

  • Limit your CI costs, since you only run your tests once, and when you need to,
  • Speed up your project and improve your velocity. Launch large tests only when the PR is ready to be merged, and avoid relaunching your tests because of out-of-date PRs,
  • Secure your code base: only up-to-date and thoroughly tested PRs can be integrated.

So, are you convinced? If not, we invite you to try Mergify and its merge queue free of charge for the next 14 days. This trial should finish the job and convince you for good ;)