Cutting Costs with GitHub Actions: Efficient CI Strategies

GitHub Actions (GHA) provides an automation platform allowing users to build, test, and deploy their code right from GitHub repositories. It offers a robust solution to implement Continuous Integration (CI) and Continuous Deployment (CD), thus making the software development process more efficient and streamlined.

However, utilizing GitHub Actions can be costly, as it is billed by the minute. Pricing varies, depending largely on the operating system chosen and other parameters. Therefore, strategizing and optimizing GitHub Actions can significantly save monetary resources.

Most articles on the Internet explain how to make your CI go faster by parallelizing jobs or speeding up your test suite. That's great, but that focuses on a different metric (runtime) than the one we're talking about here (money).

Below you will find a list of all the tips and tricks that we rely on at Mergify to keep our CI budget under control with GitHub Actions.

1. Group Short Jobs

GHA billing is precise: each started minute is billed as full. Thus, running ten 1-second jobs will be billed as ten minutes. Therefore, grouping short jobs into a single job can save notable resources.

For instance, ten 2-3 minute jobs can cost double compared to two 10-15 minute jobs due to the rounding up to the nearest minute: you'll pay 10 times the "last minute" rather than paying it only twice.

Based on 10 pushes a day for a developer and an average price per minute of $0.01, this small alteration can result in savings of around $15/month/developer based on average usage.

That amount could cover an extra SaaS service like Mergify.

2. Cancel Previous Jobs

Frequent code pushes triggering new workflow runs are common. That's fine: you want feedback on your newly pushed code.

But do you really care about getting feedback on the old version of your pull request? This happens by default, as GitHub does not cancel the ongoing GitHub Actions jobs when you push new code.

When you have a long end-to-end test, canceling the ongoing workflow once a new one is initiated can save substantial costs. GHA's concurrency setting aids in canceling in-progress jobs for the current branch when a new commit is pushed, avoiding redundant workflow runs.

concurrency:
  group: ${{ github.ref }}
  cancel-in-progress: true

3. Fail Your Matrix Early

When using a matrix in jobs, GitHub runs multiple tests for different OS or language versions, depending on your use case. If one fails, running the remaining can be seen as wasteful, especially when a merge is unlikely with failed tests.

Consider using jobs.<job_id>.strategy.fail-fast to stop further tests when one fails, optimizing both time and costs. This allows GitHub Actions to cancel all running matrix jobs when one of them fails, saving precious CI time and money.

4. Chain Your Jobs

Running all jobs in a workflow in parallel can be costly. Typically, your jobs list looks like:

• linter;
• unit tests;
• functional tests.

All run in parallel. However there might be no point in running unit and functional tests if the linter spots a crude syntax error. Ensuring linting passes before running unit tests, and then proceeding to more comprehensive end-to-end tests, can prevent unnecessary expenditure on faulty code.

Chaining jobs using the needs keyword helps ensure that subsequent jobs are dependent on the success of the preceding ones. This approach saves money and avoids wasting resources on extensive tests if there are fundamental errors in the code.

For example, the above scenario could be changed to:

• linter;
• unit tests (need linter to pass);
• functional tests (need linter to pass).

This would ensure no time is wasted running tests if the code does not pass the linter.

5. Leverage a Merge Queue

A merge queue can delay costly checks until they are absolutely necessary, conducting minimal sanity checks like linting and unit tests initially. When it’s time to merge, the full test suite is run to ensure robustness. This occurs only before the merge and after approval, maximizing the efficiency of CI time.

The typical scenario looks like the one described above, but with more delays added. Rather than running all your jobs on each pull request creation and update, you'd typically run the linter and unit tests, but delay running the costly functional tests until the pull request is ready to be merged and has been approved by humans.

When it's ready to be merged, the pull request is enqueued by a developer, and the merge queue runs the final CI tests before the merge. This means the costly tests are not run in the usual developer workflow of creating and updating the pull request.

A merge queue can also test multiple pull requests simultaneously once they enter the queue, optimizing resource utilization even further.

6. Optimize Through Caching

Caching dependencies is a straightforward way to speed up workflows, thereby reducing the runtime and associated costs. Efficient caching is crucial for optimizing the workflow duration and, subsequently, the expenses.

7. Select the Proper Runner

Choosing the right runner, with the appropriate number of CPUs, is crucial. An inappropriate runner choice can lead to either overspending for excessive resources or underperforming due to insufficient resources, slowing down the process and increasing costs indirectly.

8. Run Workflows Sensibly

Triggering workflows only when necessary, utilizing on.paths, can save considerable resources. For instance, there's no need to run a JS linter if only Python code has been altered in a multi-language repository. Such careful selections ensure that workflows are not triggered unnecessarily, avoiding wasteful runs and controlling costs.

Conclusion

GitHub Actions, while powerful, requires strategic utilization to ensure cost-effectiveness. The intricate balance between speed, quality, and expenditure necessitates thoughtful optimization of workflows. By grouping short jobs, canceling unnecessary runs, failing matrices early, chaining jobs appropriately, leveraging merge queues, optimizing caching, selecting the right runners, and running workflows sensibly, developers can significantly cut down on CI time and associated costs while maintaining high-quality development standards.

Remember, it's not just about cutting costs; it's about efficient resource management. Allocating resources wisely ensures financial savings and a more streamlined and effective development process. Balancing the trade-offs between cost, time, and quality is key to maximizing the benefits of GitHub Actions in any development environment.