When Approving Pull Requests Becomes Tinder

Approving a pull request used to take an afternoon. Now it takes a swipe.

With Artifacts Driven Development, every change comes back with proof: a recording of the feature being used, a QA report, a verdict. You look at it for a few seconds and say yes or no. Approving pull requests has become a Tinder-like experience.

That is good news, and it breaks the software development lifecycle.

The next bottleneck is merging

When pull requests are good enough that you could merge every one of them, the slow part is no longer writing them or reviewing them. It is getting them into main.

Every pull request has to be synced with main. Every sync can conflict. Every conflict has to be resolved. Every resolution needs CI to run again. And while that one runs, three more pull requests have landed and the next one in line is out of date again.

Here is what happened to kosuke-core as the agents got faster:

At that rate a person who merges by hand does nothing else. Most of the work in a day becomes syncing branches that went stale while you were syncing the previous one.

Pushing straight to main is not an answer. It turns parallel work back into sequential work, and parallel work is the whole point.

Approach one: the merge train

The first thing we built was a merge train. An agent looks at every pull request that is approved and ready, decides the order, and merges them one by one. For each one it syncs with main, resolves the conflicts, does a quick review and runs QA when the change needs it.

It worked. In its first weeks, kosuke-core merged up to 90 pull requests in a single day and 16 in a single hour.

But a train is one pull request at a time. Every merge pays for a sync and a full CI run. It looked like a ceiling.

Approach two: merge batches

So we tried the crazy approach. An agent took every pull request that was approved and passed QA, merged all of them into one batch branch on top of main, resolved the conflicts between them once, synced once, ran CI once, then reviewed and ran QA on the combined result. The batch merged as one pull request.

We ran it from October 24 to November 8: 154 batches carrying 775 pull requests, up to 26 in a single batch. The bursts were real. In the best hour, 31 pull requests landed.

The days were not. Over those 16 days we landed about 60 pull requests a day, against 56 a day on the train the week before. And batches had costs the train does not:

• One bad pull request holds up the rest. The batch has to eject it, and an eject only works when the failure can be pinned on one change.
• A revert is bigger. Undoing a batch undoes every change in it.
• Conflicts get resolved away from the author. Every resolution needs its own record, so a person can check it later.

Bigger bursts, the same day, more to go wrong. On November 9 we removed batches. The merge agent is now the merge train, or it is off.

Approach three: a better train

Instead of merging more at once, we made the train smarter about what goes next, and stopped letting one failure stop everything.

• A red main no longer stops the train. CI tests the pull request's own head, failures that come from main do not count against it, a fix for main is filed and merged first, and the checks main broke run again once it is green. The Merge agent card shows when CI on main would stop every merge, with a one tap Fix.
• It ranks by how soon a pull request can merge. Ready now first, then one step away, then waiting on a review. Then by the files it shares with the others, then by priority.
• Stuck pull requests go last. Changes requested, a red check or a draft no longer hold up the ones that can merge.
• It syncs only what is about to merge. A conflict sync waits only behind a pull request that can land soon, so the agent stops syncing branches that would go stale again before their turn.
• It ignores noise. Two pull requests that only touch the same translation file no longer hold each other up.
• Long runs keep going. When a run fills 70% of its context window, the agent compacts it and carries on.

On November 11, the train landed 95 pull requests, our best day so far.

Pull requests per hour

Pull requests per hour is the number we watch now. Agents can open pull requests in parallel and you can approve them in seconds, but they only count once they are in main, and the hour is where you see what is in the way.

What is still slow

Two things, the same two as before.

• CI. Every merge on a train pays for a sync and a CI run. The faster CI gets, the more pull requests fit in an hour.
• Conflicts. When many pull requests touch the same files, each one that lands makes the next one stale. Better ordering reduces them. It does not remove them.

That is where we are working now.

The lifecycle needs reinventing

When pull requests are generated in parallel, come back with proof, and get approved in seconds, the process around them was designed for a different world. Branches, review queues, CI runs per pull request: all of it assumes a few changes a day from a few people.

One person with a thousand agents ships more than that before lunch. The lifecycle has to be rebuilt for it, and merging is where it breaks first.