The code review policy decision record: why the review depth standard you chose determines your defect escape rate surface and your merge delay accumulation failure mode

A 31-person SaaS that built a B2B financial reporting platform had been running a one-reviewer LGTM policy since the founding team of four had written it into their GitHub branch protection rules in month two. The policy was simple and had been appropriate for its original context: any engineer on the team could open a pull request, any other engineer could approve it, and CI passing was required before merge. For the first eighteen months, the policy worked as intended. The team was small, the reviewers knew every subsystem personally, and the review conversations were extensions of the design conversations that had already happened in Slack and in weekly architecture discussions. A reviewer who approved a change in the payment module was someone who had written most of that module or had reviewed every significant change to it for the past year.

By month twenty-four, the team had grown to thirty-one engineers. Thirteen of those engineers had been hired in the preceding eight months. The reviewers who approved changes to the billing API were now frequently engineers who had not been present for the original design discussions, had not reviewed the architectural choices that the billing API's authorization model depended on, and were applying the review policy — one approval, CI passing — as a bar for correctness of implementation rather than as a bar for correctness of design intent and security model.

In month twenty-seven, a senior engineer submitted a refactor of the invoice retrieval API. The change was 400 lines across eleven files. Its stated purpose was to consolidate three separate invoice query paths into a single service method to reduce duplication. The tests were comprehensive. The code was clean. A mid-level engineer who had been on the team for five months approved it after forty minutes of review. He noted two style issues in the comments, both of which were addressed before merge. The PR was merged on a Thursday afternoon.

Four weeks later, a support ticket arrived from an enterprise customer. The customer's finance team had noticed that their invoice list page showed invoice records whose amounts did not match their contract terms. Investigation revealed that the records were not theirs — they belonged to a different enterprise customer whose account had a similar invoice date range. The tenant isolation check that had previously existed in each of the three separate query paths had been present in two of the three paths. The third path had implemented the check in a slightly different location — not inside the query itself but in the API controller that called it. When the refactor consolidated the three paths into a single service method, the check from the two consistently implemented paths was used as the template. The third path's check, being in the controller rather than the service, was not identified as logically equivalent during the refactor. The consolidated service method had no check. Every invoice retrieval call made through the previously-third-path code was now returning results without tenant isolation applied.

Forty-seven enterprise customer accounts were affected. All of them had accessed the invoice list page during the four-week window. Whether any of them had actually seen records belonging to another customer depended on whether their request had been routed through the affected code path — which in turn depended on a query parameter that was present in approximately thirty-five percent of invoice list requests. The exposure window required GDPR Article 33 notification for customers in the European Union. Two enterprise customers escalated to their security teams. One initiated a formal vendor security review process that delayed a contract renewal by six weeks.

The reviewer who approved the PR was not negligent. He had applied the policy the team had established — one approval, CI passing — and had done a thorough review relative to the depth that policy implied. The policy had not specified that changes to authorization-sensitive code paths required a different review standard, a minimum reviewer qualification, or a mandatory completion of automated security checks before human review began. The founding session that established the one-reviewer LGTM rule had not asked which categories of change that rule was appropriate for and which required a higher bar. In month two, with a four-person team where every reviewer had comprehensive context across every system, the distinction was unnecessary — the reviewer's context was the higher bar, applied implicitly. In month twenty-seven, with a thirty-one-person team whose reviewers had highly variable context across the billing and authorization systems, the absence of the distinction was load-bearing.

A 38-person product SaaS had a code review process that was widely described internally as "thorough." The engineering team ran GitHub pull requests for all changes to the main branch. Every PR had at least one reviewer assigned. Review conversations were detailed — the team was proud of the quality of its review comments, which were consistently substantive rather than stylistic. The one thing the process did not specify was PR size. There was a general norm that "PRs should be reviewable," understood to mean "not trivially obvious," but no numeric constraint and no decomposition requirement when a change grew large.

As the team and the codebase grew, PR sizes grew with them. A feature that required database schema changes, service layer modifications, API endpoint additions, and frontend component updates was submitted as a single PR — because it was one feature, and splitting it would require coordinating multiple reviews and merge orderings that felt more cumbersome than submitting the complete change at once. By the team's third year, the average PR size in their main repository was 620 lines, with a median of 480 and a tail extending to 1,400–1,800 lines for complex features.

Reviewers were conscientious. They took the review responsibility seriously. What changed as PRs grew was the structure of the review process itself, in ways that were invisible to any individual reviewer but visible in aggregate across the review history. For PRs under 200 lines, the reviewer typically left comments distributed relatively evenly across the diff — early sections, middle sections, late sections. For PRs over 600 lines, the review comments concentrated heavily in the first third of the diff. The later sections of large PRs — often the sections that implemented the core logic the PR was about, after the setup code and data model changes that opened the diff — received fewer comments per hundred lines. The pattern was not the result of reviewers becoming less careful as they progressed; it was the result of reviewer attention being a depletable resource that grew shorter as the diff grew longer. The reviewer who had spent seventy minutes understanding what a 1,200-line PR was doing had significantly less capacity to reason deeply about each remaining section than the reviewer who had spent ten minutes understanding a 200-line PR.

In one sprint in month thirty-one, three separate defects escaped review in three separate PRs, all of which had passed the team's standard review process. The first was a missing null check in an edge case of a background job that had been submitted in a 900-line PR; the reviewer had identified several genuine issues in the first half of the diff and had mentally classified the second half, where the null check was missing, as "cleanup code." The second was an incorrect pagination offset calculation in a data export endpoint submitted in an 1,100-line PR that included the new endpoint, the associated service methods, three new database queries, and a full suite of unit tests — the pagination logic was in the service method section of the diff, which the reviewer reached after forty-five minutes of review and reviewed quickly because the tests were comprehensive. The tests covered the happy-path pagination behavior. They did not cover the offset calculation for the last page of an export with a record count that was an exact multiple of the page size. The third was a missing index on a foreign key column in a schema migration submitted as part of an 800-line PR that included the migration, the model changes, and the feature that used them; the migration section was four files at the end of the diff, and the reviewer confirmed the migration was reversible and the columns were correct, but did not verify that foreign keys were indexed — because that check requires mentally cross-referencing the new column against the query patterns in the service code, which was in a different section of the diff reviewed forty minutes earlier.

None of the three defects was the kind of obvious mistake that a diligent reviewer catches at any PR size. All three were the kind of mistake that a reviewer catches when they can hold the full change in working memory — the null check is visible when you know you are looking at a section of code that handles an edge case, which requires knowing the full scope of the change; the pagination edge case is visible when you are actively verifying the test coverage against the behavioral requirements rather than confirming that tests exist; the missing index is visible when you are cross-referencing the schema change against the query access pattern, which requires recalling the query code reviewed twenty minutes ago. The review policy specified "thorough review." It did not specify what "thorough" required structurally — including a PR size that made thoroughness operationally achievable rather than aspirationally intended.

A 44-person B2B SaaS that built a data integration platform had established a code review process in year one that included a branch protection rule requiring at least one approval before merge. What it did not include was a review response SLA — a documented expectation for how quickly a reviewer would respond after a PR was opened. The implicit norm was "review PRs when you have time, before you start new work." In a ten-person team, this norm worked because context switching costs were low and social pressure made multi-day PR queuing uncomfortable. By year three, with forty-four engineers across three timezone-distributed teams, the same norm produced PR queue times of three to five days for the initial review response, and two to three additional days for re-review after changes were requested.

The predictable response to multi-day PR wait times was that engineers began stacking work: opening a new PR based on the most recent commit of an unreviewed PR, so that work could continue without waiting for the base change to be reviewed and merged. Stacking was not a formal policy — it developed organically as a coping strategy. Engineers who were blocked waiting for review on PR #1 would create PR #2 based on PR #1's branch, continue working, and note in the PR description that #2 depended on #1. By the time the three-to-five-day review window had elapsed on PR #1, the engineer had often also opened PR #3 based on PR #2.

In month twenty-nine, a senior engineer submitted a PR that refactored the core data transformation pipeline — a change that touched the API contract between two internal services. The refactor was technically correct and had been discussed in an architecture session the previous week. The PR sat in the review queue for four days. During those four days, three other engineers — whose work in adjacent systems depended on the transformation pipeline API — opened their own PRs based on the unreviewd refactor, incorporating the new API contract in their changes because the design decision had been made and they needed to move forward. By the time the reviewer opened the base PR on day four, the stack above it included four dependent PRs across two repositories, totaling approximately 2,200 lines of new code.

The reviewer identified a problem in the base refactor: the new API contract changed the error response format in a way that was backward-incompatible with several downstream consumers that were not in the change's test scope. The change needed to be modified before merge. The modification required changing the response format specification — which meant that the three PRs immediately above the base, all of which had been written against the new format specification, needed to be updated to account for the corrected format. The fourth PR, which depended on the third, needed the same update but also had its own changes that now produced different behavior under the corrected format. The engineer who had submitted the base PR spent three hours making the corrections. The three engineers whose PRs depended on the base spent a combined sixteen hours updating their changes, resolving merge conflicts that had accumulated across four days of parallel development, and re-testing against the corrected contract. The total cost of the four-day review delay on a single base PR, in the context of a team that had developed a stacking norm as a response to chronic review latency, was nineteen engineering hours.

The founding session that established the branch protection rule had not specified a review SLA. It had not specified a stacking policy. It had not specified an escalation protocol for when a PR sat in the review queue longer than an acceptable window. All three of those omissions were benign in year one, when the team was small enough that the informal norm produced fast reviews naturally and stacking was not a rational response to review latency. In year three, the same informal norm produced a review queue with three-to-five-day latency, which made stacking rational, which made rejection-cascade incidents predictable, which made the nineteen-hour rework event not an anomaly but a recurring cost that showed up as "coordination overhead" in sprint retrospectives without anyone identifying the review SLA absence as the structural cause.

Structural properties set by the code review policy decision

Three structural properties are determined when a team decides how to specify its review depth standard, PR size policy, and review SLA. None appear explicitly in the founding session that writes the branch protection rules and adds the first entry to the contributing guide — they are operational characteristics that emerge from the founding policy's assumptions about team size, reviewer context, and review queue latency at the time the policy was written.

Property 1: The review depth standard and the defect escape rate surface. A single depth standard applied uniformly to all change categories produces a defect escape rate determined by how the lowest-risk category shapes the standard. A one-reviewer LGTM bar is appropriate for routine changes — a configuration update, a localization string addition, a dependency version bump with no API changes. Applied to a 400-line authorization-path refactor, the same bar produces the defect escape rate of "one reviewer with variable context on the authorization model reviewing a change that requires deep knowledge of the tenant isolation invariants to assess correctly." The escape rate for that category is not a function of reviewer diligence — it is a function of whether the policy requires a reviewer with the specific context the change needs. The defect escape rate surface for security-sensitive and billing-path changes is structurally uncorrectable by adding more reviewers at the same depth standard; a second reviewer applying the same bar as the first provides marginal additional coverage, not the categorically deeper coverage that requires domain-qualified reviewers who are explicitly responsible for verifying the security model rather than verifying the implementation. A change category taxonomy — with per-category review requirements specifying minimum reviewer count, required reviewer qualification (must have reviewed N prior changes in this subsystem, or must hold the designated security reviewer role), and mandatory automated gate completion before human review begins — is the mechanism that closes the defect escape surface in the high-impact categories without requiring every reviewer to apply maximum depth to every change. The security scanning decision record specifies the automated gates that complement human review for authorization, injection, and supply chain categories; those gates are most effective when the review policy specifies that their output must be reviewed before the human reviewer begins, not run in parallel with human review as independent signals.

Property 2: The PR size policy and the review thoroughness failure mode. Review thoroughness is a function of the reviewer's ability to hold the full change in working memory while reasoning about behavioral correctness. That ability degrades as diff size grows: above approximately 400 lines of changed code, reviewers begin to orient to the change (understanding what it is doing) rather than reasoning about the change (evaluating whether it is doing the right thing). The thoroughness failure mode is the change that passes review because the reviewer understood the first two-thirds of the diff well and reviewed the last third at reduced depth — which is where the defect was, because the defect is most often in the implementation detail section of the diff, after the setup code and interface definitions that open it. The review depth standard and the PR size policy are coupled: the depth standard specifies what "thorough review" requires, and the PR size policy is the operational constraint that makes the specified depth achievable. A review policy that specifies thorough review but does not specify a size constraint at which thoroughness is practically achievable has created a nominal standard that degrades silently as PR sizes grow above the threshold where the standard is operationally satisfiable. The PR size limit is not a bureaucratic constraint on engineering autonomy — it is the size below which the review depth standard the team has committed to is actually achievable in a single review session without reviewer attention depletion producing systematically lower coverage in the later sections of large diffs. The test strategy decision record connects here: reviewers rely on the test suite to verify behavioral coverage of the cases they do not trace manually; a PR size policy that requires decomposition into reviewable units also requires that each decomposed unit has sufficient test coverage to be independently verifiable, rather than depending on an integration test that only passes when all the decomposed pieces are merged together.

Property 3: The review SLA and the merge delay accumulation surface. A review SLA is the operational mechanism that prevents review latency from becoming a driver of the stacking behavior that makes rejection cascades predictable. Without an SLA, review latency is determined by reviewer availability and priority, which produces high variance — days when PRs are reviewed quickly and days when they sit untouched. High variance in review latency makes stacking rational: an engineer who cannot predict whether a PR will be reviewed in four hours or four days will start new work on top of the unreviewed change after the first few hours of waiting, because the alternative is unproductive blocking. Stacking compounds over time: the longer the average review latency, the deeper the average stack, and the higher the rework cost when a base rejection propagates through the stack. The merge delay accumulation surface grows as the product of (average stack depth at the time of rejection) × (average rework cost per dependent PR in the stack). It is not visible in PR review time metrics — those metrics capture the time from PR opening to first review response, not the downstream rework cost generated by the review latency that drove stacking. It appears in sprint retrospectives as "coordination overhead" and "unexpected rework" without a structural explanation. A review SLA — with a specified initial response time, a specified re-review response time after changes are requested, and a named escalation protocol when either window is missed — makes stacking economically irrational: if a PR will be reviewed within 24 hours, the rework cost of stacking and then getting a rejection is high relative to the one-day wait. If a PR might sit for four days, the same calculation produces a different outcome. The incident response playbook decision record connects here: postmortem reviews of incidents caused by code changes that passed review should include a review of whether the PR that introduced the defect was reviewed within the SLA, whether it met the size policy, and whether it triggered the depth standard for its change category — these three parameters together determine whether the defect escape was a policy gap or a policy violation, and the distinction determines the remediation.

What the founding session records and what it omits

The founding code review session — typically a conversation about how to configure branch protection rules, what to put in the contributing guide, and how many approvals to require — records the reviewer count, the CI requirement, and the approval rule. What it does not record is the change category taxonomy that determines where the uniform rule applies and where a higher standard is needed, the PR size constraint that makes the depth standard operationally achievable, and the review SLA that prevents queue latency from becoming a structural driver of stacking behavior.

These omissions are identical in structure to every other founding decision in this series: the failure modes are theoretical at founding, when the team is small, the codebase is understood by everyone, and the review queue is short because only four engineers are submitting PRs. A change category taxonomy with security-qualified reviewers is unnecessary when every engineer has personally reviewed every significant change to the authorization model. A PR size constraint is unnecessary when PRs are naturally small because the codebase is young and changes are scoped to features that have not yet accumulated the implementation debt that makes later changes large. A review SLA is unnecessary when the review queue drains quickly because four engineers collectively have more review capacity than they have PRs to review.

The failure modes develop at predictably different rates. The defect escape rate surface in security-sensitive categories grows from the moment the team adds engineers whose reviewer context in those categories is lower than the founding team's — which is the first senior hire who has not been present for the authorization model's design evolution. The thoroughness failure mode grows as PR sizes grow, which accelerates when the codebase grows and features become more complex — the first sprint where PRs routinely exceed 600 lines is the first sprint where the review depth standard is being applied at a size where it is structurally incomplete. The merge delay accumulation surface grows from the first week where the review queue consistently takes more than one day to drain — at that point, the rational response for an engineer trying to maintain velocity is to start building on top of unreviewed changes.

The code review policy ADR closes these gaps by documenting the change category taxonomy, the PR size policy, and the review SLA at the time they are decided — not as post-hoc policy additions after each failure mode has produced an incident, a defect escape cluster, or a cascade rework event. The decisions never written down in the code review domain are not the reviewer count or the CI requirement — those appear in the branch protection rules and the contributing guide. They are the category classification that determines when the uniform bar is insufficient, the size threshold at which thoroughness breaks, and the SLA that prevents latency from producing stacking. The new CTO onboarding problem is specific in the code review context: the incoming technical leader finds the branch protection rules and the contributing guide, can confirm that reviews are required and CI must pass, but cannot determine whether there is a higher review standard for authorization changes, whether the team has an informal or explicit PR size norm, whether the review queue times that appear in GitHub PR data represent a policy gap or normal variance, or whether the stacked PRs that appear regularly in the PR history represent a team convention or a workaround for chronic review latency. The code review ADR makes those decisions explicit and auditable. The CI/CD pipeline decision record specifies the automated gates that run before human review; the code review ADR specifies which of those gate outputs must be verified by the human reviewer (as opposed to being green-fielded by CI) for each change category. The WhyChose extractor finds the code review policy discussion in your AI session history — the conversation where the founding engineer chose the reviewer count, decided whether to require two approvals for security-sensitive paths, set the contributing guide's first PR expectations, or made the decision not to set a size limit because the team was too small for it to matter — and surfaces those commitments so you can check which elements were documented and which were left to evolve informally as the team grew.

The code review policy ADR: five sections

Section 1: Review depth standard and change category classification. Specify the review depth standard for each change category, not a single standard for all changes. Define the category taxonomy: routine changes (configuration updates, dependency bumps with no API changes, documentation, test additions with no behavior changes) require one reviewer with general familiarity with the codebase; feature changes (new endpoints, new service methods, new UI components) require one reviewer with working familiarity with the affected subsystem; security-sensitive changes (authorization logic, authentication flows, tenant isolation paths, cryptographic operations, rate limiting and abuse prevention) require one reviewer from a designated security reviewer pool — a named set of engineers with documented authorization model context — plus automated security gate completion before human review begins; billing and payment path changes require one reviewer from a designated billing reviewer pool plus a smoke test against the payment processor's sandbox environment before merge; schema migrations that are not reversible require two reviewers and a named reviewer who has operated the database at its current scale; production configuration changes that affect safety mechanisms require two reviewers plus a documented rollback procedure verified by both reviewers. Specify the automated gates required by category: security-sensitive changes require SAST completion with no new high-severity findings before human review begins; billing path changes require the automated payment integration test suite to pass; schema migrations require the migration rollback test to pass in a staging environment. Document the named pools for security and billing reviewers, the qualification criteria for joining each pool, and the rotation policy for distributing review load within the pool. The security scanning decision record specifies the SAST tools and their severity classification — the review ADR specifies which severity levels block human review versus which are surfaced for reviewer awareness.

Section 2: PR size policy and decomposition requirement. Specify the PR size limits and the decomposition protocol for changes that exceed them. For routine and feature changes: maximum 400 lines changed per PR (excluding auto-generated files, lock file updates, and bulk mechanical renaming that are auto-approved by CI), maximum one primary concern per PR. A "primary concern" is one feature, one bugfix, one refactor, or one migration — not a combination. When a natural change scope exceeds 400 lines or spans multiple concerns, the decomposition protocol applies: split the change into a sequence of smaller PRs where each PR can be reviewed and merged independently, or where later PRs are explicitly stacked on an earlier PR with the stack relationship noted in each PR description and the reviewer's re-review responsibility upon base merge specified. For security-sensitive and billing path changes: the size limit is 200 lines, because the higher depth standard required for those categories is not achievable at 400 lines without review session lengths that are not practically sustainable. For schema migrations: each migration file is reviewed as a standalone PR, separate from the application code changes that use the new schema — this constraint exists because schema migration review requires a different focus than application code review and combining them in a single PR allows the application code review to anchor the reviewer's attention at the expense of migration review depth. Specify exceptions explicitly: an exception to the size limit requires the PR author to include a written justification, the designated tech lead to approve the exception before the PR is opened for review, and a review pairing protocol (two reviewers reviewing the over-limit PR together rather than independently) to partially compensate for the attention depletion that large PRs produce. Log exceptions in a running exception register so that the tech lead can review whether exception frequency is increasing — which would be a signal that the size limit is set at a level the team does not find credible.

Section 3: Review SLA and reviewer assignment protocol. Specify the review SLA and the escalation protocol for when it is missed. Initial review response SLA: 24 hours from PR opening during business hours (excluding weekends and documented company holidays). Re-review response SLA after changes are requested: 24 hours from the change request response being posted. Both SLAs apply to the assigned reviewer, not to any reviewer — which means reviewer assignment is required at PR opening, not left to volunteers. Specify the assignment protocol: the PR author assigns the review to one or two named reviewers based on the change category (for security-sensitive and billing path changes, the author assigns from the designated reviewer pool; for other changes, the author assigns to a reviewer with subsystem familiarity or uses the team's round-robin assignment rotation). Specify the escalation protocol for missed SLA: when the initial review SLA window closes without a review response, the PR author pings a designated backup reviewer (a named role per team, updated monthly) and notifies the originally assigned reviewer in the same message. The escalation is not a reprimand — it is a structural backup that exists because reviewer availability is variable and the SLA is a team commitment. The backup reviewer's SLA is eight hours from the escalation ping. Specify the stacking policy: stacking is permitted only when the base PR has been open for more than 48 hours without a review response and the PR author has escalated per the protocol; stacking without escalation is not permitted. When a stacked PR's base PR is rejected, the engineer who submitted the base PR is responsible for notifying all stacking engineers within four hours, and the team's sprint planning process reserves capacity in the following sprint for the cascade rework rather than treating it as unplanned overhead.

Section 4: Review tooling configuration and automation gates. Specify the GitHub branch protection rules, the required status checks, and the code ownership configuration that enforce the review policy automatically rather than relying on reviewer awareness of the policy's requirements. Branch protection on the main branch: require PR before merge (no force push), require status checks to pass before merge (CI, SAST for repositories containing security-sensitive paths, migration test for repositories containing schema migrations), require review from code owners (CODEOWNERS file specifying the designated reviewer pools for security-sensitive directories and billing path directories). CODEOWNERS file policy: security-sensitive directories (auth/, billing/, tenancy/, permissions/) require review from the security-reviewer GitHub team regardless of which engineer submits the change; billing path directories (billing/, payments/, invoicing/) require review from the billing-reviewer GitHub team. Specify the automated PR size check: a CI gate that fails PRs exceeding the size limit with a message explaining the decomposition requirement and pointing to the contributing guide's decomposition protocol section — not a warning that can be dismissed, a failure that blocks the PR from being reviewable until the size issue is addressed or an exception is logged. Specify the stale PR detection: a bot that posts a comment when a PR has been open for 24 hours without a review response, notifying the assigned reviewer and including the escalation protocol reminder. This automation surfaces the SLA miss without requiring the PR author to manually track and escalate — the tracking is automated; the escalation decision remains with the PR author.

Section 5: Review quality feedback loop and calibration cadence. Specify the mechanism for auditing review quality over time and recalibrating the policy when defects escape through compliant reviews. The feedback loop has three components. First, incident-linked review audit: when a production incident is traced to a code change that passed review, the postmortem must include a review audit section that answers three questions — did the PR that introduced the defect meet the size policy? Did it meet the depth standard for its change category? Was it reviewed within the SLA? If all three answers are yes, the defect escaped despite policy compliance, which means the policy's depth standard for that change category is insufficient — which triggers a policy recalibration discussion. If any answer is no, the defect escaped because of policy non-compliance — which triggers an enforcement discussion rather than a policy recalibration. Second, quarterly review quality metrics: track defect-per-PR rate by change category, average PR size by team, average review response time by reviewer pool, and stacked-PR frequency over the quarter. Review these metrics with the tech lead and engineering managers quarterly to identify whether defect escape rates in high-impact categories are trending up, whether PR sizes are growing past the policy limit regularly (which is a signal that the limit is not credible to the team), and whether review response times are consistently near or above the SLA boundary (which is a signal that reviewer capacity relative to PR volume is becoming strained). Third, annual policy review: revisit the change category taxonomy, the size limits, and the SLA once per year, explicitly asking whether the founding policy's assumptions still hold — whether the team's composition and reviewer context have changed in ways that affect which categories require higher depth standards, whether the codebase's complexity has changed in ways that affect the achievable review thoroughness at the current size limit, and whether the team's velocity and review volume have changed in ways that affect the achievable SLA at the current reviewer pool size. Connect the annual policy review to the CI/CD pipeline decision record's annual review — the automation gates specified in both ADRs evolve together, and an inconsistency between the review policy's automation requirements and the pipeline's gate configuration is discovered most efficiently when both are reviewed in the same session.

FAQ