This is my Workflow: Elastic

Anne-Laure Civeyrac 19 Apr 2022

Have you ever wanted to be a 🪰 on the wall to see how your favorite open-source projects manage their GitHub workflows? At Mergify, eager to know more about our users, we were always curious to find out how they organize their branches, what tools and processes they employ, and their merge policies! So we decided to create a new series that invites major open-source projects to showcase their pull request [PR] workflows. This first installment is all about Elastic, the company behind the well-known search engine Elasticsearch and the data visualization dashboard software Kibana. We met with Victor Martinez, a principal software engineer in the company’s observability team.

The observability team 👀

The main aim of our team is to help observe the infrastructure (in the cloud or on-premises) and applications, as well as provide logs, metrics and traces with various alerts. To do this, integration with other tools, operating systems, platforms, vendors, and so on is required.

Workflow strategy ♟

The strategy adopted by the observability team for Elastic’s main projects is to use the main branch for active development and 8.{number} branches for active development and releases. This workflow is not only used by the engineers but also the company’s tech writers, product managers, and managers. The release cadence occurs every six weeks.

Merge policy 📝

Depending on the situation, we either complete the approval ourselves as humans or use Mergify. For a manually created PR, two people must approve it before the merge can happen. However, when a PR has been created automatically by Dependabot or some other automation tool, only one Mergify approval is needed for most of the teams. In all cases, the PR is then merged automatically by Mergify according to the defined rules. And if there is a conflict, Mergify flags it on the PR. Once the PR has been merged, there are backports or forwardports between the branches.

Backports and forwardports 🔁

The backport policy differs depending on whether the PR is a fix or a feature. The PR will be backported to all the active branches if it's a fix. But if it’s a feature, it might not be backported to minor version release branches, for instance. The team lead is responsible for deciding where features should be backported.

We do it by adding the label backport-{branch} to the PR to indicate which branches we want to backport it to. If the PR should not be backported, we use the label backport-skip. When the PR is merged on the main branch, the backport PR is automatically created by Mergify on the relevant branches. The backport PR is then reviewed and merged by the owner.

We also manage forwardports to push documentation updates made on 8.{number} branches to the main branch. In these cases, we use the forwardport-main label.

In the future, we would like to have a more generic backport policy so that we don’t have to change branch names every time we bump a minor version release branch or add a new branch. Also, we would like to be able to automatically populate labels on backport PRs based on the original PR.

Ownership 🤖

We use a probot on GitHub to force labels to indicate teams and owners on the PRs. One of our more complex projects is a mono repository, and using this probot is the only way we can ensure its PRs are targeting the right team.

To manage ownership, we assign teams to a particular file or folder. So anytime someone creates a PR, it will validate it if someone is named as the owner of the file or folder and put that person as the owner.

Tests 🔍

We use Jenkins as our CI tool on 95% of our projects. Teams use GitHub Actions, Buildkite, or Travis on some other projects.

Automation setbacks 🔀

Because of this highly automated workflow, we have an issue that people sometimes forget to merge the backport PRs. This is mainly caused by the backport PR not being automatically assigned to the person who created the original PR. We fixed this with the help of Mergify, and we used the assignment to notify the owner of the original PR.

Take a look at Mergify's actions possibilities!

We also implemented a reminder: Assignees are now automatically notified if a backport has been waiting a week or more to be reviewed or merged. We’re also trying to figure out how to notify people more directly, for instance, with weekly emails.

Building a merge queue 🐛

As we observe ecosystems and infrastructures, we need to integrate with a lot of tools, such as cloud providers and operating systems, to see if there is a memory heap, a lack of resources, or any networking issues. We need to be sure that our projects have extensive test coverage of all the different issues that can arise.

So we plan to work in the mid- to long-term on our proper merge queue. Our strategy is to run light validation tests on the PRs themselves so that our developers are not slowed down and to use the merge queue as a more exhaustive way of validating PRs.

But the complexity with the mono repository project at the moment is that the team has some merge conflicts in the changelog file. The process the project needs to follow prevents us from using the merge queue fully, so we need to move changelogs from within the repository to somewhere else if we’re going to be able to facilitate the merge queue. An obvious solution might be to use our proper changelogs generator based on the commits PR or something like that.