Julian Maurin How we made our merge queue lower its own quality bar (only when it's drowning)
Our merge queue tested every PR alone, which choked the morning a burst of agent PRs landed faster than CI could clear them. Here's the dynamic batch size that widens under load, and the production data on what it changed.
Our own merge queue runs on the safest setting it has. Every pull request gets its own CI run, tested alone against the predicted merge state. A few weeks ago we let it lower that bar on its own, and only when an unpredictable burst of pull requests arrives faster than CI can clear them.
The setting that stopped scaling
We ship a lot more pull requests than we used to. Agentic coding changed the shape of our day: instead of a handful of humans opening PRs, we now have humans plus a pile of agents, all landing work into the same repository at once. The code review side of that is its own story. The part that started to hurt was CI.
We run our CI on our own bare-metal runners (here’s why). That gives us a lot of capacity, but it’s a fixed amount of capacity. We don’t pay per CI run, so the constraint isn’t a bill. It’s availability. Every speculative check holds a real machine for the length of a full test suite, and we only have so many machines.
To keep that bounded, our default queue runs with max_parallel_checks: 5. Five PRs can be under test at any moment. With a batch size of 1, that’s also a hard ceiling on how fast the queue can drain: five PRs validated per CI cycle, no matter how many are waiting. And the load that hits this ceiling is bursty and hard to predict. A quiet stretch and a pile-up of twenty agent PRs can be twenty minutes apart, and the queue has to handle both without anyone retuning it.
What batch_size: 1 was actually buying us
A batch size of 1 is the most conservative thing a merge queue can do. Each PR is tested by itself. If it breaks, you know exactly which one broke, and backing it out is a clean single revert.
Batching gives that up. Put two PRs in one speculative check and they pass or fail as a pair. If the pair passes CI but one of the two is semantically wrong in a way the tests didn’t catch, you can’t always revert just that one. The other was merged on top of it, and a clean single-PR revert can turn into a semantic conflict. You’re left untangling two changes that only ever existed as a unit. For us this is still theoretical. It hasn’t bitten us, and we went in treating it as a real cost we’d decided we could afford.
This is the RCV theorem in practice: a merge queue optimizes two of reliability, cost, and velocity, never all three. We run parallel speculative checks, so we’ve already bought reliability and velocity and we pay for it in cost, meaning those five bare-metal slots. A fixed batch_size: 1 parked us at the most reliable point on the line we’d chosen. Safe, and slow to drain under load.
A fixed batch size also makes you guess. Set it to 1 and you stay safe but choke the moment a burst lands. Set it to 2 or more and you carry the blast radius all day, including the long quiet stretches when there was nothing to pair and nothing to gain. No single number is right, because the load it’s meant to match never sits still.
A batch size with a floor and a ceiling
The change we shipped is small to describe. batch_size now accepts an object, {min, max}, instead of a single number. When parallel-check slots are plentiful the queue stays at min. When a backlog builds, it grows each speculative check toward max to keep the queue draining.
The sizing is one line of arithmetic, run once per check:
clamp(ceil(remaining_pulls / free_slots), min, max)where free_slots is the room left under max_parallel_checks. Larger batches get front-loaded so the worst-case bisection stays small. With max_parallel_checks: 5 and {min: 1, max: 2}, seven PRs waiting become checks of sizes {2, 2, 1, 1, 1}: just enough pairing to cover the backlog, no more.
A plain integer still works and normalizes to {min: N, max: N}, so every existing config is byte-for-byte identical to before. No feature flag, no new metric to watch. The object form is the opt-in.
The obvious question is why not just set batch_size: 2 and be done. Because a flat 2 pays the blast-radius cost on every quiet Tuesday afternoon, for nothing. With min: 1, a calm queue is back to testing one PR at a time, full isolation. We only spend reliability when there’s an actual backlog to justify spending it. The degradation is conditional.
So we turned it on for our own default queue:
batch_size:
min: 1
max: 2One PR at a time when we keep up. Pairs of two only under load.
What happened when a burst hit
Here’s the part I went and checked in production rather than guessed, because the whole point is that the behavior is conditional, and a feeling isn’t evidence.
Before the change, 100% of our speculative checks were size 1. The first size-2 batch in our history fired on June 16 at 07:01 UTC, a European morning with agent PRs already stacking up. Then it behaved like this:
| Day | Speculative checks paired into size 2 |
|---|---|
| Jun 8-15 (before) | â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘ 0% |
| Jun 16 (busy) | ███████████████░░░░░ 74% |
| Jun 17 | ████░░░░░░░░░░░░░░░░ 20% |
| Jun 18 (calm) | â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘â–‘ 0% |
June 16 was a heavy day: 74% of checks paired up, 146 of 198. On that morning, pairing let the same five slots clear close to twice the pull requests per CI cycle they could have at a fixed size of 1. June 17 calmed down to 20%. June 18 was quiet and paired nothing at all, which is to say it behaved exactly like the old batch_size: 1.
That June 18 column is my favorite number in the whole thing. Same YAML as the busy day, and it sat at full strictness because it could afford to. A flat batch_size: 2 would never give you that. The dial moved itself, with the load, with no one touching the config.
What I’m not going to claim
I could stop here and tell you our throughput went up. It didn’t. Merges per day, weekday average, were 48.4 before and 45.3 after, which is flat or a hair lower inside the noise. It’s a short, unrepresentative window, and on most days we were demand-bound anyway. Batching a busier queue doesn’t invent pull requests that aren’t there.
Dynamic batching is aimed at one thing: the burst. Load arrives in spikes we can’t predict, and a fixed batch size of 1 forces every spike to drain through five slots one PR at a time. The queue now widens exactly when a spike hits and narrows back once it passes. June 16 is what that looks like: an ordinary, unplanned busy morning the queue absorbed on its own, then forgot about two days later.
{1, 2} is also just our first step. It was a deliberately small move: double the ceiling and watch whether it absorbs the load. Keeping the ceiling at 2 bounds the downside too. If that silent semantic case ever does land on main, the tangle is two PRs, the smallest batch above one. It doesn’t always fully absorb the load, but it feels much better, and that’s enough to keep it for now and push the bound later if we need to.
The takeaway
A quality standard isn’t a constant. batch_size: 1 was a decision we made when our PR volume was lower and the safest possible setting was free. {min: 1, max: 2} is the same decision, re-made for a queue that now gets flooded by machines at moments nobody schedules.
The version I’ve come around to is the one that re-makes the decision every few minutes on its own. Strict when it can afford to be, looser only when staying strict would just mean a longer line for no reliability you’ll ever cash in. What I want from a quality standard now is for it to degrade itself, and only when it has to.
