Announcing Speculative Merge Queues

Julien Danjou 31 Mar 2021

Today we're happy to announce the general availability of our new merge queues feature. This unique functionality allows engineering teams to secure their code merge by serializing them while parallelizing their testing.

If your team never used a merge automation tool nor a merge queue, this is a crucial tool to use. Merge queues are becoming as important to their users as their continuous integration system: they wouldn't safely merge any code without those two tools.

Large engineering team, such as Shopify or Strava, implemented their own software to solve this problem.

We're cheering at Mergify to bring such a feature set to smaller engineering teams. We aim to empower every engineering crew to increase their code quality and efficiency, whatever their size is.

Why a merge queue?

To understand the problem queues resolve, imagine the following situation:

The base branch (e.g., main) has its continuous integration testing passing correctly.
A pull request is created, which also passes the CI.

The state of the repository can be represented like this:

While the pull request is open, another commit is pushed to main — let's call it new commit. That new commit can be pushed directly to main or merged from another pull request; it doesn’t matter.

The tests are run against the main branch by the CI, and they pass. The state of the repository and its continuous integration system can be now described like this:

The pull request is still marked as valid by the continuous integration system since it did not change. As there is no code conflict, the pull request is considered as mergeable by GitHub: the merge button is green.

If you click that merge button, this is what might happen:

As a new merge commit is created to merge the pull request, it is possible that the continuous integration testing reports failure. Indeed, the continuous integration did not test the pull request with the new commit added to the base branch. This new commit might have introduced some further tests in the base branch while the pull request was open. That pull request may not have the correct code to pass this new test. You just broke your main branch, and maybe your production system.

Queues to the Rescue

Using a merge queue solves that issue by updating any pull request that is not up-to-date with its base branch before being merged. That forces the continuous integration system to retest the pull request with the new code from its base branch.

If a merge queue were being used in the previous example, Mergify would automatically merge the main in the base branch. The continuous integration system would have rerun and marked the pull request as failing the test, removing it from the merge queue altogether.

Serializing all those pull requests can limit your merge throughput, this is why Mergify now supports speculative merges.

Speculative Merge Queues

While Mergify allows automating GitHub pull request merge by using a merge queue, this can slow down your merge throughput. Our new queue action offers new possibility by allowing multiple queues and offering speculative checks to parallelize testing and increase your merge queue throughput. By using this feature, it's now possible to merge pull request serially while keeping a high merge throughput, removing the queue bottleneck.