leek.io leek.io
contact
#ai #tooling #methodology

I Went on Vacation and My Agents Paid Down My Tech Debt

CJ
Chris Jones · @leek
Jul 13, 2026 · 7 min read · 1,207 words
I Went on Vacation and My Agents Paid Down My Tech Debt

I took a week of vacation. My codebase got measurably better while I was gone.

Before I left, I pointed Superconductor at one of my projects. It’s not a toy repo: Laravel 13, Filament, PostgreSQL, multi-tenant, HIPAA-regulated. Around 526,000 lines of application code, 315 models, 572 migrations, and a test suite roughly the same size as the app itself.

Then I mostly stopped looking at it for a week.

The agents didn’t. They opened about two dozen pull requests a day, every day, like a metronome. When I sat back down, the tally since kickoff was 335 PRs, 328 of them merged. Of the ones authored directly by the Superconductor GitHub app, I merged 190 out of 194, a 98% acceptance rate. The median diff was 77 lines.

Nobody wrote a feature. That was the point. Every single PR was the kind of work that never survives sprint planning: tech debt, latent bugs, performance fixes, hardening.

What 200 tech-debt PRs actually look like

“AI cleaned up my codebase” is vague enough to be meaningless, so here are the actual categories, with real PR titles from the run.

Unbounded background jobs. The biggest single theme. Dozens of scheduled jobs and reminder scans were loading entire tables into memory. Fine at 10 tenants; a time bomb at 1,000. The agents systematically found and chunked them: “Bound appointment reminder batch processing,” “Bound authorization renewal processing,” “Bound CEU deadline reminder processing.” There were over twenty PRs that started with the word “Bound.”

N+1 queries. “Eliminate N+1 queries from document reminders,” “Reuse eager-loaded appointment staff participants,” “Avoid lazy loading payment plan aggregates.” Classic Laravel debt, tedious to hunt manually, mechanical to fix once found.

Missing HTTP timeouts. Every third-party integration (clearinghouse APIs, Microsoft Graph, RingCentral, NPPES) got connect and request timeouts: “Bound Stedi HTTP connect timeouts,” “Bound Microsoft Graph HTTP timeouts.” This is the kind of hardening everyone agrees matters and nobody ever schedules.

Tenant-scoping bugs. The scariest and most valuable category for a multi-tenant health-data app: “Scope expiring authorization stats to tenant,” “Scope conversation family guardian traversal by tenant,” “Fix soft-deleted portal guardian access.” Plus a run of test-factory fixes ensuring factories produce tenant-consistent data, so these bugs can’t sneak back in.

Duplication and dead code. “Centralize phone search normalization,” “Centralize active authorization date checks,” “Remove unused document template helper.” Small extractions that make the next change cheaper.

Static analysis debt. My PHPStan baseline file had grown to 266KB of suppressed problems. The agents chipped at it PR by PR: “Reduce PHPStan baseline debt in auth actions,” “Remove mastery timeline PHPStan suppressions.”

Outright bug fixes. “Fix DST transition timezone bounds,” “Fix final patient responsibility calculation,” “Harden AWS SMS SNS webhook topic validation.” Bugs that were live in production, found by an agent reading code nobody had touched in months.

None of these PRs would ever have won a prioritization argument against a customer-facing feature. All of them together are the difference between a codebase that scales and one that falls over.

Why the merge rate was 98%

The tool is the least interesting part of this story. Point an agent at an unprepared repo and you’ll get a pile of plausible-looking PRs you’re afraid to merge. Three things made this run different.

1. The PRs were small on purpose

Median diff: 77 lines. 90th percentile: 194 lines. A 77-line PR with a clear title takes two minutes to review. I could clear twenty of them with morning coffee.

The unit of autonomous agent work isn’t the feature. It’s the small, independently verifiable pull request.

This is also what made the 98% merge rate honest rather than reckless. I wasn’t rubber-stamping a 5,000-line refactor on faith. I was reviewing changes small enough to actually hold in my head.

2. CI was the real reviewer

Every PR runs through the same gauntlet: the full Pest suite (1,355 test files), PHPStan, Pint, a Composer security audit, and an automated AI commit review with its own prompt checked into the repo. A separate workflow auto-opens an issue when CI fails, and that issue becomes the feedback signal for the next agent run.

CI wasn’t the only reviewer, either. Every PR picked up reviews from three separate bots: Gemini Code Assist, OpenAI’s Codex, and GitHub Copilot. Three different models, three different blind spots, all leaving inline feedback before a merge was even on the table.

The agent proposes. CI disposes. The last mile ran through a triage-github-pr skill checked into the repo itself: it reads the diff, the check results, and every line of bot feedback, classifies each finding, fixes the legitimate blockers, and merges when the PR comes out clean. Conflicts, failing checks, or unresolved feedback stop the merge and get reported back to me instead.

3. The repo was written for agents before the agents arrived

The repo has an 18KB CLAUDE.md, a matching AGENTS.md, nested per-directory instruction files, and dozens of checked-in skills and rules covering Laravel, Filament, and the domain itself. Every PR body included a section titled “Technical debt evidence” — proof the duplication or bug being fixed actually existed, and confirmation the files weren’t already claimed by another open PR.

When you have twenty-four PRs a day in flight, agents stepping on each other’s changes is the failure mode. Convention, documented in the repo, is what prevents it.

An agent-ready codebase is one where the rules for making a good change are written down instead of living in someone’s head.

What this didn’t do

Full honesty, because the hype version of this story helps nobody.

It didn’t ship features. Nothing customer-visible came out of the week. It was a week of pure maintenance.

It wasn’t unsupervised. Merges ran under my account through the triage skill, which stops and reports rather than force-merging anything questionable. Four PRs got closed or are still open because they weren’t right. The human stayed in the loop; the loop just got faster and cheaper.

It needed a mature harness. Without a test suite the size of the application and CI gates that actually fail, this exact setup would have been a liability. The agents inherited a decade of my paranoia. If your repo has neither tests nor static analysis, autonomous PR generation is where you’ll discover that, painfully.

The compounding part

Here’s what I keep coming back to. Tech debt has always lost the scheduling fight because its cost is diffuse and its payoff is invisible. Nobody demos “the reminder job no longer loads the whole table.”

But the economics just flipped. When a bounded-processing fix costs a two-minute review instead of a half-day of engineer time, you stop rationing this work. You run it continuously, in the background, forever, the way you run backups.

I’ve been building software for over 25 years, and the durable lesson from this week isn’t about any one tool. It’s that the repos that benefit most from AI agents are the ones that were already disciplined: tested, gated, documented. The agents didn’t replace the engineering rigor. They cashed it in.

My codebase is now better than it was two weeks ago, and I was on a beach for most of the delta.

That sentence would have been science fiction in 2024. In 2026, it’s just a well-configured Tuesday.

CJ
Reply by email
I read everything.
$ reply