What's a Merge Queue and Why Use it?
What is a Merge Queue? And when is it useful? This article provides the answers to both questions, and explains the concept of a Merge Queue in detail. A must-have for most developers.
That's a good question, isn't it? Indeed, if it was only a little known term a few months ago, merge queues are more and more talked about. Between announcements from leaders like GitHub and real technical solutions, merge queues are increasingly adopted by software development teams.
You will therefore dive into this topic to understand what a merge queue is, in what situations they are useful but also how they work in practice.
Are you ready for it? Let's get started - you can also watch our video at the end of this article.
What Do We Mean by "Merge Queue"? 🤓
Before wanting to know why you should work with a merge queue it is important to define what it is.
Its name says all: a Merge Queue is simply a queue of Pull Requests waiting to be merged.
Each member of your team creates Pull Requests, like a lot each day. Then the maintainers of the repository add it to the queue. Simple right?
To be accurate, you are not just putting basic PRs in a queue. All the PRs present in the queue have been approved by maintainers. It means that they passed all the required checks.
So you get a queue full of validated Pull Requests. Sounds cool, not really useful though. Why not just merge them, one by one? To give you an answer, let see first what common issues you will encounter if you do not use a merge queue.
Why Do You Need a Merge Queue? 🚀
To be honest, there are plenty of reasons and arguments to explain why you should bet on a merge queue. In this part, you will discover a really problematic situation and how you can solve it by using a merge queue.
The Common Issue: Merging Outdated Pull Requests and its Consequences.
The common issue encountered when handling multiple concurrent pull requests is the challenge of dealing with outdated submissions as new changes are integrated into the target branch. This necessitates the crucial step of updating these pull requests with the most current modifications, a task that often proves to be both time-consuming and susceptible to errors.
Manual Rebasing
To keep multiple open pull requests up-to-date with the latest changes in the target branch, manual rebasing is often required. This process involves aligning the pull requests with the current branch state, ensuring a smooth integration. However, managing this task can be time-consuming and prone to errors, especially when dealing with numerous pull requests and reconciling contributions from multiple developers.
Incompatible Changes
Even if a pull request passes tests within its branch, it can still face issues due to incompatible changes introduced by other contributors. These conflicts occur when alterations don't align with the proposed modifications, potentially causing code clashes and unintended consequences upon merging.
Broken Builds and Red CI
Outdated or incompatible pull requests can disrupt the project's flow by leading to broken builds, resulting in a red status in the continuous integration pipeline. A red CI signifies test failures, indicating an unstable codebase. This can halt deployments, delay progress, and complicate issue identification.
Merging Outdated Pull Requests: A Concrete Exemple
To understand the problem queues resolve, you first have to understand the issue. To do so, imagine the following situation:
- The main branch passes its continuous integration testing correctly.
- A pull request is created, which also passes the CI. Call it PR1.
At this time, here is how you can represent the state of your repository:
Right now everything is totally OK, but it will not last. Let's see.
While PR1 is still open, another commit is pushed to the main branch. No matter if this newly created commit is pushed directly to the main branch or merged from another pull request. In fact, you just have to understand that the main branch has been modified.
Then the tests are run against the main branch by the CI, and they pass again. At this point, you can describe the state of your repository and its continuous integration system like this:
You can see the PR1 is still marked as valid by the continuous integration system. Indeed, it is normal since it did not change. Only the main branch changed.
As there is no code conflict, GitHub considers PR1 as mergeable: the merge button turns green.
All happy you are, you smash that green button.
But you saw that coming, you might have a surprise - and not a good one.
Now that you want to merge PR1, a new merge commit is created and the continuous integration tests fail, but why?
In fact, when PR1 was marked as valid, the CI did not test - again - PR1 with the new commit added to the main branch (remember it?).
However, the last commit brought new tests in the main branch with him and now PR1 does not have the correct code to pass this new test. As sad as logical.
➡️ The Real Solution: a Merge Queue
A promise is a promise, you're not going to stay in this dead-end situation.
Your solution: using a merge queue. Indeed, it solves that problem by updating any pull request before being merged. But only the ones that are not up-to-date with its main branch. Actually, a merge queue forces the CI system to test the PR again but with the new code from its main branch.
In the previous situation, if you use a merge queue, it would automatically merge the main in the feature branch.
As you can see below, the CI would have rerun. Then the pull request would have been marked as failing, and removed from the queue. Of course, if the PR was valid, and all the checks passed, it would have been merged.
Another case of scenario: multiple pull requests are validated and ready to be merged.
A merge queue will schedule their merging sequentially and ensure you that they are all updated on top of each other. Obviously, these updates are only done if the pull requests complete all the conditions.
Nevertheless, what happens when you merge an updated pull request and then get another one which is still outdated. To ease your understanding, what about another illustration?
A merge queue will make sure the second pull request is updated with the latest tip of the main branch before merging. By doing this, there is no way to merge an outdated and broken pull request into the main branch.
And you repeat this as many times as necessary, for each outdated pull request in the queue.
Life is not always easy, but a merge queue can make it a lot easier.
Other Use Cases Where Merge Queue Proves Formidable
Charly Laurent
Léo Ecrepont
Charly Laurent
Julien Danjou
Mathieu Poissard
Hugo Escafit
Hugo Escafit
Damien Couppé
How Does a Merge Queue Work? 🧐
Now that you know what kind of issue a merge queue can solve, let’s summarize its functioning.
As you probably understood, a merge queue looks more or less like a complex workflow we can analyze step by step.
1️⃣ Queue valid PRs
Run the Merge Queue on your pull requests. All valid pull requests - the ones that validate all the conditions to enter the merge queue can be embarked.
2️⃣ Update each pull request and trigger the CI
Sequentially and automatically, the merge queue will make sure that each PR in the queue is up-to-date by merging the main branch into the feature branch.
Then the merge queue triggers the CI to rerun to make sure the PR is mergeable into the targeted branch.
3️⃣ To merge or not to merge, that’s the question
Two quite distinct cases:
- All checks passed → merge the PR.
- Failing test → remove the PR from the queue.
The Essential Prerequisites for Using a Merge Queue
Before implementing a merge queue to streamline your development workflow, there are a few essential prerequisites to consider.
✅ Version Control System
Ensure that your project is managed within a version control system (VCS) such as Git. A merge queue relies on VCS capabilities to manage and merge branches, making it a fundamental requirement.
✅ Automated Tests and Continuous Integration Setup
Have a CI system in place that automatically runs tests and checks upon each push or merge to the repository. A well-configured CI system is crucial for detecting issues early in the development process.
✅ Standardized Workflow
Establish a standardized workflow for your team, including guidelines for creating and managing pull requests. Consistency in your development process is key for a merge queue to work effectively.
✅ Code Review Process
Ensure that your team has a code review process in place. Code reviews help maintain code quality and catch potential issues before they make their way into the main branch.
✅ Clear Merge Policies
Define clear merge policies and criteria for accepting pull requests. This includes specifying when a pull request is considered ready for merging and what tests or checks it must pass.
By having these prerequisites in order, you can maximize the benefits of a merge queue and enhance the efficiency and reliability of your software development process.
What About Mergify’s Merge Queue?
In concrete terms, Mergify’s Merge Queue does everything you just read.
It is one of the first merge queues launched on the market and already has thousands of happy users.
While the common features presented before can solve many headaches, you may find yourself facing even more complex and specific situations. These situations require very specific functionalities.
Good news, with Mergify, you already have them at your fingertips!
Speculative checks
The first pull requests from the queue are embarked in a merge train and tested together in parallel so they can be merged faster.
Batches
Mergify allows checking the mergeability of multiple pull requests at once using the batch_size option.
Multiple Queues
By using multiple queues, it's possible to put some pull requests in a higher priority queue.
Queue Freeze
Mergify allows freezing the merge of one or several queues simultaneously to provide maximum control and flexibility on how and when you want the code to be merged.
Priority management
Choose which PR should be merged first, depending on labels, owners, etc. Your rules ultimately!
Conclusion
Here you are folks, you should know everything about the concept of a merge queue. From how it works to why you should use one, merge queues have no secrets for you.
It's now time to try it and you know what? You can enjoy a 14-day trial for free with Mergify. I guess it's your chance to say bye-bye to this outdated pull request issue.
