← Back to the blogMethod

False productivity: when AI speeds you up without moving you forward

More pull requests, more commits, longer reviews. A two-week sprint shows where AI speed goes, and how to measure real progress instead.

September 8, 2026 · 7 min read

Your teams ship more code than they ever have, and releases don’t feel any calmer. Writing got faster. Somewhere between the first commit and production, a good share of that speed disappears.

Activity is up everywhere

GitHub’s Octoverse 2025 counted nearly a billion commits in the year, up 25%. Inside companies the curve looks similar. Faros AI’s The AI Productivity Paradox report (July 2025, more than 10,000 developers across 1,255 teams) found that teams with high AI adoption merge 98% more pull requests.

The same report carries the other half of the story. Review time in those teams rose 91%. Code gets written faster and read slower.

So leadership asks a fair question. If the team produces twice as much, why doesn’t the roadmap move twice as fast? Rather than answer in the abstract, let’s follow one sprint.

PULL REQUESTS MERGEDOther teamsHigh AI adoption+98%TIME SPENT IN REVIEWOther teamsHigh AI adoption+91%FAROS AI, THE AI PRODUCTIVITY PARADOX, JULY 2025written faster, read slower
Teams with high AI adoption merge 98% more pull requests, and their time in review rises 91% (Faros AI, July 2025). Code is written faster and read slower.

Sprint goal, notification preferences

This team is invented, but you’ll probably recognize it. Six engineers, one product manager, two weeks. The goal is to let users choose which notifications they receive (email, push, in-app) for each category, plus a “mute all” switch.

The product already has rules about notifications, even if nobody has listed them in one place. Security alerts, like a new login or a password change, always go out, whatever the user picks. In some regions, marketing emails start switched off until the user opts in. Transactional emails such as receipts can’t be turned off at all.

Days 1 and 2

The PM writes three tickets and the engineers open their assistants. By the end of day two there are five pull requests: a preferences screen, a new settings table, an API endpoint, changes to the notification dispatcher, and a data migration. The activity dashboard looks great.

Days 3 to 5

Review starts, then slows down. The reviewers know the dispatcher less well than the engineer who wrote it two years ago, who has since moved to another team. Each pull request takes hours, because the reviewer has to rebuild the context the assistant never had. Someone asks whether “mute all” should silence security alerts. Nobody is sure. A long thread begins.

Days 6 to 8

QA confirms that “mute all” does silence security alerts. The assistant wrote a clean, general rule that sends nothing to a muted user. It didn’t know about the exception, because the exception lives in another module and in a compliance note on the wiki. Fix, new review, new QA run.

Meanwhile the migration set every existing user’s marketing preference to “on,” including users in regions where it should start “off.” Review missed it because the migration looked trivial.

Days 9 and 10

The assistant had also added a weekly digest option, because the preferences model “naturally” supported one. Nobody asked for it. It clashes with another team’s in-flight change to the email scheduler. Two teams, one meeting, and the digest gets rolled back.

Sprint review

The activity charts show a strong fortnight, with lots of pull requests, lots of commits and plenty of comments. The feature ships with a known issue on regional defaults, pushed to the next sprint. Compared with building the same feature by hand a few years ago, it probably took more total effort. It was just typed faster.

Following the lost time

Look back at those two weeks and the time went to three places.

Review absorbed it. Every generated pull request needs someone who knows the context. Those people are few, and their week doesn’t stretch with the volume.

Trust thinned out. In the Stack Overflow Developer Survey 2025 (49,000 respondents), 29% of developers said they trust the accuracy of AI output, down from 40% in previous years. People who don’t trust what they read end up rereading it, retesting it and hesitating before they merge.

Rework compounded. The security alert bug came back once. The regional defaults will come back next sprint. The digest went out and came back in a single afternoon. Each round trip cost more than the last, because more code had been built on top in the meantime.

A quieter leak runs underneath all three. Every open question, like whether “mute all” covers security alerts, turns into a thread, then a meeting, then a decision made by whoever happens to be in the room. That decision rarely gets written anywhere the next assistant will read, so the same question can come back next sprint.

None of this appears on a dashboard that counts pull requests or commits. Those numbers track output, and output is precisely what AI made cheap. They can’t tell you whether a change did what the business asked, or whether yesterday’s behavior survived it. A team can post record activity for a whole quarter while its confidence in its own software slowly wears down.

Counting rules that hold

We think the fix starts with a different unit. Lines and pull requests measure effort. Progress is better measured in rules: how many new rules are implemented as decided, and how many existing ones still hold.

Replay the sprint with that unit, using Lyriks.

Before coding. The change starts as a need, then becomes a solution, a functional spec and a technical spec. Lyriks works out what it touches, what’s compatible, what conflicts and what needs a decision, including across changes running in parallel. In this sprint, the impact report would have listed the three existing notification rules on day one, put the “mute all versus security alerts” question in front of the PM, and shown the overlap with the other team’s scheduler work. The Evolve page explains how that report is built.

During coding. The approved spec goes to each engineer’s assistant over MCP, with its rules and their sources. The assistant changing the dispatcher knows security alerts are an exception, because the rule sits in its context with a link to where it lives in the code.

After coding. A deterministic engine checks the result rule by rule. Additions such as the weekly digest are flagged before merge. Omissions like the regional default, and regressions like muted security alerts, go back to the assistant with the rule they break. The loop repeats until every rule is met, which in this sprint means before QA rather than after.

Each release then gets a compliance report showing what is compliant and what isn’t. Your PM can read it as a progress metric, since it counts rules that hold instead of lines that changed.

BEFORE CODINGDURING CODINGAFTER CODINGImpact reportSpec over MCPRule-by-rule checkLists the 3 existingnotification rules"Mute all" vs securityalerts goes to the PMOverlap with thescheduler work shownEach assistant getsthe rules and sourcesThe dispatcher changeknows security alertsare an exceptionADDITIONweekly digestOMISSIONregional defaultREGRESSIONmuted security alertsgaps caught before QA, not after
The notification preferences sprint replayed: the impact report before coding, the spec over MCP during coding, and a rule-by-rule check that catches the digest, the regional default and the muted alerts before QA.

Same two weeks, replayed

Here is roughly how the sprint changes when the rules come first.

Days 1 and 2 are slower. The PM and an engineer read the impact report, settle the security alert exception and the regional defaults, and agree with the scheduler team that the digest is out of scope. Fewer pull requests open in the first two days, and the activity chart dips.

Days 3 to 8 are where the time comes back. Reviewers no longer rebuild context from scratch, since every pull request arrives with the rules it was checked against. QA starts from a list of rules to test instead of a guess. The “mute all” bug never reaches QA, because the dispatcher change fails verification on the security alert rule and goes back to the assistant the same afternoon.

Days 9 and 10 are quiet. No cross-team meeting, no rollback, no known issue carried into the next sprint.

We can’t promise your own sprint will look exactly like this. The shape is what we’d watch for, with less motion at the start and much less rework at the end.

Reporting outside engineering

For a CTO, the first change is what you can say to the rest of the company. Instead of reporting how many pull requests were merged, you report which rules were added, which ones changed, and which ones were checked and still hold. “Users can now choose notifications per category. Security alerts still always go out. Regional defaults are verified.” A business stakeholder can follow that sentence, and argue with it if needed.

For the team, the change is fewer rollbacks. Reviewers spend their attention on design and readability, since rule compliance is already checked. The fear of touching an old dispatcher fades, because its existing rules get verified with every change.

Before your next sprint

AI is a remarkable accelerator, and it still needs a heading. Next time you plan two weeks of work, write down the existing rules it could break before the first pull request opens. There’s usually one like the security alert exception hiding in plain sight. We publish pieces on method like this one regularly, so subscribe to the newsletter if you’d like the next.

Keep your software in tune

Start with one module. Lyriks recovers its spec, then verifies every change. All it needs is your repository.