When Approving Pull Requests Becomes Tinder, Merge With a Train

Filippo Pedrazzini4 min read
When Approving Pull Requests Becomes Tinder, Merge With a Train
Contents

Approving a pull request used to take an afternoon. Now it takes a swipe.

With Artifacts Driven Development, every change comes back with proof: a recording of the feature being used, a QA report, a verdict. You look at it for a few seconds and say yes or no. Approving pull requests has become a Tinder-like experience.

That is good news, and it breaks the software development lifecycle.

The next bottleneck is merging

When pull requests are good enough that you could merge every one of them, the slow part is no longer writing them or reviewing them. It is getting them into main.

Every pull request has to be synced with main. Every sync can conflict. Every conflict has to be resolved. Every resolution needs CI to run again. And while that one runs, three more pull requests have landed and the next one in line is out of date again.

Here is what happened to our own codebase as the agents got faster:

WeekPull requests merged
Sep 7 to 1354
Sep 14 to 2099
Oct 5 to 11199
Oct 12 to 18438

At that rate a person who merges by hand does nothing else. Most of the work in a day becomes syncing branches that went stale while you were syncing the previous one.

Pushing straight to main is not an answer. It turns parallel work back into sequential work, and parallel work is the whole point.

The merge train

The first thing we built was a merge train. An agent looks at every pull request that is approved and ready, decides the order, and merges them one by one. For each one it syncs with main, resolves the conflicts, does a quick review and runs QA when the change needs it.

It worked. In its first weeks, our codebase merged up to 90 pull requests in a single day and 16 in a single hour.

But a train is one pull request at a time. Every merge pays for a sync and a full CI run, so what the train merges next, and what it does when something fails, decides how much lands in a day.

A better train

We made the train smarter about what goes next, and stopped letting one failure stop everything.

  • A red main no longer stops the train. If main is broken, the train keeps going. Failures caused by main don't count against a pull request. A fix for main goes first, and the affected checks run again once it passes. The merge agent card shows when CI on main would stop every merge, with a one tap Fix.
  • It ranks by how soon a pull request can merge. Ready now first, then one step away, then waiting on a review. Then by the files it shares with the others, then by priority.
  • Stuck pull requests go last. Changes requested, a red check or a draft no longer hold up the ones that can merge.
  • It syncs only what is about to merge. A conflict sync waits only behind a pull request that can land soon, so the agent stops syncing branches that would go stale again before their turn.
  • It ignores noise. Two pull requests that only touch the same translation file no longer hold each other up.
  • Long runs keep going. When a run fills 70% of its context window, the agent summarizes it and keeps going.

On November 11, the train landed 95 pull requests, our best day so far.

Pull requests per hour

Pull requests per hour is the number we watch now. Agents can open pull requests in parallel and you can approve them in seconds, but they only count once they are in main, and the hour is where you see what is in the way.

What is still slow

Two things.

  • CI. Every merge on a train pays for a sync and a CI run. The faster CI gets, the more pull requests fit in an hour.
  • Conflicts. When many pull requests touch the same files, each one that lands makes the next one stale. Better ordering reduces them. It does not remove them.

That is where we are working now.

The lifecycle needs reinventing

When pull requests are generated in parallel, come back with proof, and get approved in seconds, the process around them was designed for a different world. Branches, review queues, CI runs per pull request: all of it assumes a few changes a day from a few people.

One person with a thousand agents ships more than that before lunch. The lifecycle has to be rebuilt for it, and merging is where it breaks first.