How to cut CI costs with batching and bisection

Take a team merging 20 pull requests a day, a 30-minute queue pipeline, and one pull request in twenty that fails in the queue. That last number is the one everybody forgets, so the bisection runs are counted here as a cost, not waved away.

The first step is the one that pays. Going from no batching to batches of three removes more than half the queue's CI time. Doubling from five to ten buys another five percentage points while making every failure slower to resolve and every merge wait longer for a batch to fill. Most teams land between three and five and stop there.

The bisection column assumes one failing batch a day and a binary split, which is what Mergify does by default. If your queue failure rate is closer to one in five than one in twenty, redo the arithmetic with more failing batches before you pick a size.

A failing batch does not send five people back to their branches. Mergify splits it and retests cumulative parts of it, using the failure it already knows about to skip combinations it has ruled out:

Batch [#1, #2, #3, #4, #5] fails
  → test [#1, #2] and [#1, #2, #3, #4] in parallel
  → [#1, #2] passes, [#1, #2, #3, #4] fails   → the culprit is #3 or #4
  → test [#1, #2, #3]                          → fails
  → #3 is the culprit and leaves the queue
  → #1 and #2 merge, #4 and #5 go back in the queue

The split runs in parallel up to max_parallel_checks, so raising that value trades CI concurrency for a faster recovery. If a part contains a single pull request and still fails, that pull request is the culprit and it is removed with a comment explaining why. Everything else keeps its place in the queue.

Two knobs matter when bisection gets expensive. batch_max_failure_resolution_attempts caps how many times a batch may be split before the whole thing is dropped from the queue. skip_intermediate_results stops a flaky intermediate failure from blocking a pull request whose changes later pass inside a bigger batch, which is worth turning on if your suite is not perfectly stable.

Batching is one key on a queue rule. This config adds it to a queue that also splits validation into a cheap entry check and an expensive pre-merge suite, which is where the two savings compound:

merge_queue:
  # How many batches (or batch splits) run at the same time.
  max_parallel_checks: 2

queue_rules:
  - name: default
    # Cheap checks decide who gets into the queue.
    queue_conditions:
      - check-success=lint-and-unit

    # The expensive suite runs once per batch, not once per pull request.
    merge_conditions:
      - check-success=full-suite

    batch_size: 5
    batch_max_wait_time: 5 min
    merge_method: squash

• batch_size is how many queued pull requests are tested in one run. This is the setting that removes runs from your bill.
• batch_max_wait_time is how long Mergify waits for a batch to fill before testing a smaller one. Without it, a quiet afternoon produces batches of one and the saving disappears.
• max_parallel_checks is how many batches or splits run at once. It buys throughput and recovery speed, and it costs concurrency.
• queue_conditions and merge_conditions are the two-step split: lint and unit tests gate entry, and the full suite runs only against the batch that is about to land.

If you do not have a queue yet, start with setting up a merge queue on GitHub and add batching once pull requests are flowing through it.

Pick from your queue failure rate:

• Under 2% of queued pull requests fail: batches of 5 to 10. Bisection is rare enough that its cost rounds to nothing.
• 2% to 5%: batches of 3 to 5. This is where most teams sit.
• Over 5%: batches of 2 to 3, and treat the failure rate as the real problem. At a high enough rate the bisection runs cancel the saving.

The practical signal is easier than the percentage: if bisection fires more than once a day, your batch is too big or your tests are too flaky. Start at 3, watch a week of queue activity, and move up.

One thing that is not a reason to keep batches small is fear of unrelated changes colliding. Mergify groups pull requests that touch the same scopes and directories, so a batch tends to be a set of related changes rather than a random slice of the queue. The batching explainer covers the trade-offs in more depth.

Batching is the biggest single lever, and it stacks with a few others.

• Two-step CI keeps the expensive suite off pull requests that are still being reviewed. Combined with batching, a 30-minute suite runs once for five pull requests instead of five times per pull request per push.
• Scopes tag each pull request with the areas of the repository it touches, so monorepo CI can run only the jobs those areas need. A documentation change stops paying for the integration suite, and the queue reuses the same scopes to build better batches.
• Flaky tests are a spending problem before they are a trust problem. Every retry is a full pipeline you paid for twice. CI Insights shows which jobs and tests are burning runs, which is usually a shorter list than people expect.

None of this requires rewriting your pipeline. Batching, two-step CI, and scopes are all queue configuration, and they work with whatever CI you already run. For the broader picture of what a queue does to your pipeline, see continuous integration.