2026-11-03 · ~22 min read · Decision records

The engineering on-call model decision record: why the on-call model you chose determines your incident response latency ceiling and your engineer burnout surface

Q: How do you design a follow-the-sun on-call model that doesn't create ownership gaps at handoff windows?

Follow-the-sun ownership gaps are structural: they appear at every point in the schedule where primary responsibility transitions from one region's team to another's, and they are proportional to the duration of the transition window in which neither team's on-call engineer has confirmed ownership of open incidents. Three mechanisms eliminate or bound the gap. First, explicit handoff ceremony: a scheduled ten-minute overlap period at the start of each regional shift during which the outgoing primary and the incoming primary have a synchronous handoff of every open incident, with the incoming primary explicitly taking ownership before the outgoing primary goes off-call. The handoff ceremony eliminates ambiguous ownership by requiring explicit transfer rather than relying on the outgoing engineer to detect that their replacement is now available. Second, a warm-body rule: no open P1 or P2 incident is in an unclaimed state at any point — if the outgoing primary's shift ends with an open incident, the outgoing primary retains ownership until the incoming primary takes the handoff call, and the outgoing primary's SLA is extended for the duration of the handoff. This rule makes the gap's cost visible in on-call hours rather than in incident MTTR, which produces pressure to keep handoff windows short. Third, gap coverage rotation: for each handoff window, a named secondary from the overlapping regional team carries a secondary pager for the thirty minutes preceding and following the handoff, so that any alert that fires during the handoff window has an immediate recipient who is not in mid-handoff conversation. The secondary rotates through the regional team independently of the primary rotation and adds minimal per-engineer on-call burden while eliminating the gap coverage problem.

Engineering on-call model decisions are not decisions about which paging tool to use or how long the escalation timeout should be. They are a set of founding decisions about who is responsible for incident response coverage and at what rotation frequency, how alerts are classified and routed so that the engineer who receives a page has both the authority to act and the knowledge required to act, how the escalation path from first responder to subject matter expert is structured so that the time budget from alert to resolution is predictable rather than determined by who happens to answer their phone, how the alert quality is maintained so that on-call periods are bounded by actionable incidents rather than by alert volume, and how the rotation model accounts for the cumulative physiological cost of sleep disruption so that coverage is not purchased by accumulating a debt that surfaces as attrition. The consequences are invisible when the team is small enough that informal heroism covers the coverage gaps; they become visible when system complexity exceeds one person's ability to reason about it alone at 3am, when alert volume per rotation reaches the point where engineers are paging out more from fatigue than from actual incidents, and when burnout accumulation exceeds the team's ability to absorb it without losing engineers.

A 12-person developer infrastructure startup had an on-call model that its engineering team would have described as natural. Engineers owned the services they built — the primary queue processing system was owned by the two engineers who had written it, the authentication layer was owned by the engineer who had designed it, and the observability pipeline was owned by the founding engineer who had built it and who also served as de facto technical lead for the whole platform. When any service paged, the engineer who owned it received the alert. The model had been adopted without discussion; it was simply the consequence of a team small enough that service ownership and on-call coverage were the same thing. When the team had been six people, every engineer had owned one or two services and the rotation had been implicit in the service ownership. When the team had grown to twelve, the founding technical lead had accumulated ownership of the three systems that interacted most with everything else — the queue processor, the observability pipeline, and a custom routing layer that had been built during a three-day sprint and had never been formally handed to another engineer.

In the two weeks following the team's migration to a new cloud provider — a migration that had been completed without incident — the three systems the technical lead owned generated 23 pages across 14 nights. The pages arrived in clusters: three in a 90-minute window on a Tuesday evening when the routing layer encountered a connection pool state it had not encountered under the old provider's network topology, four spread across two nights when the queue processor's retry logic produced a cascade that took the observability pipeline's ingest endpoint to saturation, and sixteen distributed across the remaining nights when a combination of DNS TTL misconfiguration and the new provider's health check polling rate produced false-positive alerts at a rate the team had not experienced before. The technical lead resolved eleven of the twenty-three pages himself — seven of them required genuine debugging, four required only a restart and a log review. Twelve pages were resolved by other engineers who had been added to secondary escalation paths as the migration progressed and who successfully triaged the alerts that did not require knowledge of the routing layer's internal state. For eleven of fourteen consecutive nights, the technical lead was the primary responder.

In week three after the migration, a genuine incident arrived: a race condition in the queue processor's dead-letter handling that had been introduced during the migration created a state where messages in the dead-letter queue were being re-enqueued faster than the processor could consume them, eventually producing a memory exhaustion condition that caused the processor to crash and restart in a loop. The incident began at 2:17am. The technical lead received the page at 2:19am. He had been paged at 11:45pm the previous night for a false-positive that had required 25 minutes to confirm as non-actionable, and had been paged at 3:40am two nights before for a genuine alert that had taken 90 minutes to resolve. The technical lead's diagnosis of the dead-letter race condition took 4 hours and 32 minutes — from the first page to service restoration. A postmortem conducted two weeks later reconstructed the diagnostic timeline and concluded that the technical lead had made three diagnostic decisions during the incident that extended the resolution time by a combined estimate of two hours: he had initially attributed the memory exhaustion to a memory leak in the observability pipeline integration because a similar symptom pattern had appeared during the migration window, had spent 47 minutes ruling out that hypothesis before pivoting to the queue processor's internal state, had then misread a log timestamp because he transposed two digits in a timestamp column while fatigued, and had spent 38 minutes validating a hypothesis whose premise was the transposed timestamp before catching the error. The on-call model had been designed around service ownership. It had not been designed around the cumulative cognitive load on the engineer who owned the services that everything else depended on.

A 36-person B2B SaaS had implemented a formal on-call rotation sixteen months after its founding. The rotation had been designed by the engineering manager who joined when the company was 22 people, using a model she had used at her previous company: a seven-day rotation in which one engineer was primary and one was secondary, the primary received all P1 and P2 alerts, and the secondary received alerts only after the primary had not acknowledged within five minutes. The alert severity model used thresholds calibrated by the engineering manager with input from the founding CTO: P1 for any condition that affected end-user request success rate above 1%, P2 for any condition that degraded end-user request latency above a defined threshold, P3 for everything else. The model had been adopted quickly because it gave the rotation a clear structure and because the thresholds had been set conservatively — the 1% P1 threshold was low enough to catch real incidents early.

Fourteen months into operating the rotation, the engineering manager reviewed the on-call logs for the prior quarter. The previous three primary on-call engineers had been paged an average of 23 times per seven-day rotation period. She broke down the 23 pages per rotation by disposition: an average of 6.2 pages per rotation required a genuine response — configuration change, rollback, intervention in a running process, or escalation to a subject matter expert. The remaining 16.8 pages per rotation required a four-to-seven-minute investigation that concluded with no action required: the engineer had to wake up, log in, pull the relevant dashboards, confirm the condition had self-resolved or was within acceptable bounds, and close the alert. The 16.8 non-actionable pages were not false positives by the alert system's definition — the conditions they described were real — but they were pages for conditions that either resolved without intervention within the five-minute acknowledgment window or that required only a log review to confirm normal behavior. Seven of the 23 weekly pages arrived between midnight and 6am local time for the on-call engineer; an average of four of those seven were non-actionable. The on-call engineer was waking up four times per rotation week at a frequency distributed across the overnight hours to confirm that nothing required intervention.

The engineering manager raised this in the following quarter review with three engineers who had each completed multiple rotation periods. All three independently described the same adaptation: after six to eight months on the rotation, they had developed an intuition for which P1 alerts, based on their name and the time of night, were likely to require action versus likely to be self-resolving. All three described checking the alert name before deciding how urgently to respond — whether to get out of bed immediately or to wait the five-minute acknowledgment window before checking. One engineer described the adaptation explicitly: "I have a mental list of about eight alert types that I know will resolve without me doing anything if I give them three minutes. I acknowledge within the window so the secondary doesn't get paged, check in three minutes, and if it's still firing I go get my laptop." The alert routing model had been designed around severity thresholds calibrated to ensure that real incidents were paged as P1. It had not been designed around the distinction between a P1 condition that requires the on-call engineer to act and a P1 condition that requires the on-call engineer to confirm that no action is required. An on-call rotation that pages engineers 23 times per week, of which 17 require no action, trains engineers to evaluate the likely actionability of a page before deciding how urgently to respond — which is a sensible adaptation to the alert volume and also the adaptation that produces a delay in responding to the pages that do require urgent action, because the heuristic for which pages require urgent action is an approximation that will be wrong some fraction of the time.

A 70-person enterprise SaaS with engineering teams in Singapore, Berlin, and Toronto had designed a follow-the-sun on-call model when it opened its Singapore office eighteen months earlier. The model covered three time windows: Singapore primary covered 8am to 6pm SGT (midnight to 10am UTC), Berlin primary covered 8am to 6pm CET (7am to 5pm UTC in winter), and Toronto primary covered 8am to 6pm EST (1pm to 11pm UTC). Each region's primary was backed by a secondary from the same region. The overlap between Singapore's end-of-window and Berlin's start-of-window was designed as a one-hour handoff: Singapore primary went off-call at 6pm SGT (10am UTC), Berlin primary came on-call at 8am CET (7am UTC). The three-hour gap between 10am UTC and 7am UTC had been identified during the model design as a potential coverage problem — in UTC terms, Singapore's shift ended at 10am and Berlin's began at 7am, which meant three hours in which neither region's primary was formally on-call — but the model's designer had concluded that the historical incident data showed low incident rates in that UTC window and had classified the gap as acceptable.

In the first quarter of operating the follow-the-sun model, the engineering team tracked fourteen incidents that arrived during the identified UTC coverage gap between 7am and 10am UTC. In UTC clock terms, these incidents arrived between the time Berlin's primary came on-call (7am UTC) and the time Singapore's primary went off-call (10am UTC). The overlap was three hours, not a gap — the design had been correct on that point. But six of the fourteen incidents were not resolved until more than ninety minutes after the first page, and four of those six had customer-visible impact during the unresolved period. The postmortem analysis produced a finding that the original model design had not anticipated: the Berlin primary's on-call window began at 8am CET (7am UTC), which was the beginning of the Berlin team's workday. During the first hour of the Berlin primary's on-call window, the engineer was arriving at the office, setting up for the day, and often in a context-switching state between their overnight communications review and their daily standup. The Singapore primary's on-call window ended at 6pm SGT (10am UTC), which was the end of the Singapore team's workday. During the final hour of the Singapore primary's on-call window, the engineer was typically in end-of-day wrap-up, often in a late-afternoon sync with the Berlin team that was now starting its day. The three-hour nominal overlap was in practice a window in which both engineers were available in principle and both were in context-switching states that increased their acknowledgment latency and the latency of their initial diagnostic engagement with an incident. Five of the six slow-resolution incidents had been acknowledged within the five-minute SLA; the latency was not in acknowledgment but in the time between acknowledgment and the engineer reaching a state of focused diagnostic engagement with the incident. The on-call model had been designed around a UTC coverage map. It had not been designed around the cognitive availability of the engineers at the boundaries of their on-call windows relative to their workday context.

The Toronto team's on-call window — 8am to 6pm EST (1pm to 11pm UTC) — had been designed to cover the gap between Berlin's end-of-day (5pm UTC) and the Toronto team's own end-of-day (11pm UTC). The model had not included a handoff ceremony between Berlin and Toronto at 5pm UTC, because the incident management tooling automatically transferred alert routing to the Toronto primary at the scheduled transition time. In the first quarter, three incidents that were open when the 5pm UTC transition occurred were transferred to the Toronto primary without a verbal handoff. In two of the three cases, the Toronto primary received an alert for an incident that was already in progress, whose context — what had been tried, what had been ruled out, what the current hypothesis was — existed only in the Berlin primary's memory and in a running incident channel thread that the Toronto primary had to read backward from the most recent message to reconstruct the diagnostic state before contributing to it. In one of those two cases, the Toronto primary's first 35 minutes of incident involvement were spent reconstructing the diagnostic context that the Berlin primary had already developed, during which the Berlin primary was off-call and not available for a synchronous handoff. The on-call model had been designed as a coverage scheduling problem. It had not been designed as an information transfer problem.

Structural properties set by the engineering on-call model decision

Two structural properties are determined by the founding decisions about rotation design, alert routing, escalation paths, burnout mitigation, and incident review. Both are invisible when the team is small enough that informal heroism and tribal knowledge about the system cover the coverage and knowledge gaps the on-call model contains; both become visible when system complexity exceeds one person's ability to reason about it reliably at 3am, when alert volume exceeds the threshold at which engineers can treat every page as requiring immediate full attention, and when the burnout accumulation rate exceeds the team's ability to absorb it through inter-rotation recovery.

Property 1: Incident response latency ceiling. The incident response latency ceiling is the upper bound on the time from an incident beginning to an engineer with the relevant knowledge and authority actively working on resolution, bounded by three factors. First, alert routing precision: whether alerts are routed to the engineer with the relevant knowledge for the specific system experiencing the incident, or routed to whichever engineer is next in the primary rotation regardless of their familiarity with that system. The first failure story illustrates the knowledge concentration failure mode: when service ownership and on-call coverage are the same thing, the on-call model produces a rotation frequency per engineer that is proportional to the number of services they own, not to the total team size. An engineer who owns the three services that everything else depends on carries a rotation burden proportional to one-engineer coverage of the whole system rather than proportional to their share of the team. Alert routing that is based on service ownership rather than on a balanced rotation among engineers who have cross-trained on the relevant services produces routing precision at the cost of rotation equity, and when the cross-training requirement is not part of the on-call model's design, precision is purchased by concentrating burden. Connect this to the engineering documentation standards decision record: documentation coverage of system behavior — the operational runbooks, the incident response playbooks, the decision records for why systems were built the way they were — directly affects alert routing precision by determining whether a secondary on-call engineer who has not worked on a specific system has enough documented context to diagnose an incident involving it, which in turn determines whether the on-call model can distribute rotation burden across the team without degrading incident response latency for systems that only one or two engineers fully understand. Second, on-call rotation coverage model: whether the rotation schedule was designed around the mean incident arrival rate or the tail incident arrival rate, and whether the coverage windows were designed around the actual incident distribution across times of day rather than around organizational convenience. The third failure story's UTC coverage window analysis had been calibrated against the mean incident distribution; the six incidents that produced ninety-plus-minute resolution times arrived during a window that was nominally covered but was not designed around the cognitive availability profile of the engineers at the boundaries of their coverage windows. A coverage model that treats its windows as binary — covered or uncovered — and calibrates window boundaries against historical incident rates rather than against the cognitive availability of the engineers at window boundaries will produce response latency degradation at every boundary where engineer availability is nominal but cognitive engagement is reduced by workday-boundary context switching. Third, escalation path design: whether the escalation path from the primary on-call to a subject matter expert has defined triggering conditions, defined time budgets for each escalation tier, and defined authority for the primary to escalate without seeking approval. Connect this to the engineering technical hiring bar decision record: the escalation path design specifies the observable conditions that trigger escalation from one tier to the next, and the precision requirement is the same as in the hiring bar context — an escalation trigger defined as "when you've been working on it for a while" produces escalation timing that varies with each on-call engineer's individual judgment about what "a while" means relative to the incident's apparent severity, producing escalation latency that is a function of engineer personality rather than incident state.

Property 2: Engineer burnout surface. The engineer burnout surface is the accumulated physiological and motivational cost imposed on engineers by the on-call rotation model, bounded by three factors. First, alert volume per rotation: whether the number of pages per on-call period is within the bounds that allow the engineer to return to full cognitive performance in their subsequent working days. The second failure story illustrates the alert quality failure mode: a rotation model that pages engineers 23 times per week, of which 17 require no action, trains engineers to evaluate the likely actionability of each page before deciding how urgently to respond. The adaptation is individually rational — it allows engineers to preserve more sleep by correctly identifying which pages can be acknowledged and monitored before getting out of bed — and it is also the adaptation that degrades the model's incident response latency guarantee, because the heuristic for actionability will produce delayed responses to some fraction of the pages that do require urgent action. Alert volume that exceeds the actionable incident rate does not just create a nuisance cost; it changes the on-call model's effective behavior because it changes the response strategy engineers rationally adopt. Connect this to the engineering performance management decision record: the performance management model specifies the observable behaviors that constitute high performance; the on-call model's burnout surface directly affects those behaviors in the days following high-page-volume rotation periods, and a performance management model that does not account for on-call rotation history when evaluating engineer output in the days after a rotation will misattribute burnout-induced performance degradation as a signal about the engineer's sustainable performance level rather than as a signal about the on-call model's effect on that engineer's performance. Second, sleep disruption frequency: the number of incidents per on-call period that arrive between midnight and 6am local time, which determines the sleep disruption pattern that accumulates across rotations. Sleep disruption accumulation is not visible in the on-call metrics that engineering organizations typically review — incident count, MTTR, escalation rate — because it accumulates in the engineer's physiology rather than in the incident management system. An on-call model that produces a mean of four overnight pages per rotation period per engineer, of which three require no action, is a model that produces three sleep disruptions per week in exchange for no incident resolution value, and the cost of those three disruptions appears in the engineer's recovery time, their daytime cognitive performance in the week following the rotation, and their long-run motivation for staying on the rotation. Connect this to the engineering team growth decision record: the team growth model specifies the hiring cadence and the onboarding timeline for new engineers; an on-call model that produces burnout accumulation at a rate that causes experienced engineers to request rotation removal or exit the company constrains the team growth model by requiring replacement hiring to offset burnout-driven attrition, which creates a hiring demand that is not present in the headcount model if the on-call model's effect on attrition is not accounted for. Third, recovery time allocation: whether the rotation model includes explicit recovery provisions after on-call periods that contained sleep disruptions above a threshold, so that the burnout accumulation from high-incident periods is absorbed by the rotation design rather than by the engineer's personal resilience. Connect this to the engineering reorg decision record: organizational reorgs that change team composition change who is in each on-call rotation, and an on-call model that does not document why the rotation size, rotation frequency, and recovery provisions were calibrated the way they were will be reconstituted after a reorg using the new team's intuitions about fair rotation burden rather than the original model's calibration rationale, potentially producing a new rotation that discards the recovery provisions that were the design's mechanism for bounding burnout accumulation.

The engineering on-call model ADR: five sections

Section 1: Rotation design model. The rotation design model defines who is in the on-call pool, at what rotation frequency, what the coverage windows are, and the rationale for each design choice relative to the alternatives evaluated. Three components. Pool composition criteria: the observable properties that qualify an engineer for inclusion in the primary on-call pool for a given service or system — minimum months of tenure with the system, demonstrated ability to diagnose the three most common incident types for the system without escalation, completion of a defined set of production operations exercises conducted in a low-stakes context before the first primary rotation. Pool composition criteria distinguish the on-call pool from the team roster; not every engineer on the team may be ready for primary on-call on every system, and the criteria should specify the minimum qualification threshold rather than assuming that team membership implies on-call readiness. Rotation frequency calculation: the method by which the rotation frequency was determined relative to the pool size, the mean incident arrival rate per rotation period, and the sustainable on-call burden — the maximum number of pages per rotation period above which on-call engineers exhibit measurable performance degradation in the days following the rotation. A rotation frequency documentation that includes the sustainable burden threshold used in the calculation allows future calibration reviews to assess whether a change in incident arrival rate requires a change in pool size to maintain the same per-engineer burden. Coverage window design: the specific hours of coverage per region, the UTC conversion of each window boundary, and the engineering rationale for each boundary placement — specifically, whether each boundary was placed at a time when the incoming engineer is at the beginning of their focused workday rather than at a context-switching boundary between their overnight communications and their morning sync. Coverage window boundaries placed at workday beginnings rather than at coverage-convenience boundaries are the mechanism that prevents the third failure story's cognitive availability gap from appearing in a new follow-the-sun model.

Section 2: Alert routing model. The alert routing model defines how alerts are classified by severity, how they are routed to the on-call engineer, and what observable conditions constitute each severity level. Three components. Severity tier definition: the observable conditions that place an alert in each severity tier, specified with the response-time-constraint method rather than the category method. A P1 is any condition that, if unaddressed for a defined time window, will either cause customer-visible impact or foreclose the remediation option space; a P2 is any condition that, if unaddressed for a longer defined window, will cross those thresholds; a P3 is any condition that requires monitoring but not immediate intervention. The definition should be specific enough that an engineer applying it to a new alert condition can determine its correct tier without consulting the person who calibrated the thresholds. Alert quality maintenance process: the mechanism by which the alert set is reviewed periodically to identify non-actionable pages — pages that consistently resolve without engineer intervention or that consistently result in no-action-required dispositions — and either recalibrated to a lower severity tier, converted to a dashboard condition rather than a page, or eliminated. A quarterly alert quality review that examines the prior quarter's alert dispositions — what fraction of each alert type resulted in engineer action versus no-action-required close — and identifies alerts above a non-actionable threshold for remediation is the feedback mechanism that prevents non-actionable alert accumulation from training engineers to evaluate page urgency by heuristic rather than by the alert's formal severity. Routing knowledge-match specification: for services with narrow knowledge distribution — where fewer than a defined fraction of the on-call pool has demonstrated incident diagnosis capability for the service — the routing model should specify a service-specific routing override that routes alerts for those services to the engineers who have demonstrated that capability rather than to the rotation primary, along with a requirement that the service's knowledge distribution be broadened to qualify additional engineers before the next rotation period. The routing override is a temporary correction mechanism; the knowledge-match specification is the structural mechanism that prevents the first failure story's knowledge concentration failure from recurring as the team grows.

Section 3: Escalation path model. The escalation path model defines the trigger conditions for escalation from the primary on-call to successive tiers, the time budget for each tier before the next escalation triggers, and the authority of each tier to act. Three components. Escalation trigger conditions: the observable conditions that require the primary on-call to escalate to the secondary or to a subject matter expert, specified with sufficient precision that two engineers in the same incident would escalate at the same moment. An escalation trigger defined as "when you've been working on it for more than fifteen minutes without a diagnosis" is more precise than "when you're not making progress" and less precise than "when the incident has been open for fifteen minutes without a confirmed hypothesis and a remediation path." The most precise escalation triggers are defined by incident state rather than by elapsed time — "escalate when you have ruled out all the hypotheses on the first-tier diagnostic checklist without identifying the root cause" — because incident state captures the information that determines whether escalation will add value, whereas elapsed time captures only the duration. Time budget documentation: the maximum time each escalation tier should spend on an incident before triggering the next escalation, with the rationale for each time budget relative to the incident severity SLA. A tier-one time budget documented as "fifteen minutes for P1, forty minutes for P2" with the rationale "derived from the P1 customer SLA of sixty minutes minus the escalation transition latency of ten minutes and the tier-two diagnosis time budget of thirty-five minutes" allows the next on-call model designer to evaluate whether a change in the customer SLA requires a change in the tier time budgets. Escalation authority specification: the explicit statement that the primary on-call engineer has unilateral authority to escalate to any tier without seeking manager approval, with the rationale that an escalation authority model that requires manager approval introduces an approval latency into the escalation path that is not bounded by the incident SLA. The authority specification is frequently omitted from escalation models because it is assumed; the first failure story's technical lead's reluctance to wake his colleagues for the incidents that required only a restart was in part a social norm about when waking someone at 3am was justified — a norm that the on-call model had not addressed because it had not specified that the on-call engineer's authority to wake a colleague was established by the incident state, not by the on-call engineer's personal assessment of whether the page was worth a sleep disruption to the secondary.

Section 4: Burnout mitigation model. The burnout mitigation model defines the maximum sustainable on-call burden per engineer, the mechanisms by which burden above that threshold is addressed in the rotation design, and the leading indicators that the model uses to detect when actual burden is exceeding the designed threshold. Three components. Sustainable burden threshold: the maximum number of overnight pages — pages between midnight and 6am local time — per seven-day rotation period per engineer that the model was designed around, with the rationale for the chosen threshold relative to the alternatives evaluated. A threshold documented as "maximum three overnight pages per rotation period, derived from the point above which engineering output in the following week shows measurable degradation in the team's historical data" allows future calibration to assess whether the threshold is correctly set as the team's system complexity and incident arrival rate change. Recovery provision specification: the mandatory rest provisions after on-call periods that exceeded the overnight page threshold — the specific recovery entitlement (for example, four hours of protected morning time in the workday following a rotation period with more than four overnight pages), the mechanism for enforcing the provision without requiring the engineer to request it, and the rationale for the chosen provision relative to the alternatives. A recovery provision that requires the engineer to request recovery time after a high-page rotation produces recovery rates proportional to each engineer's willingness to self-advocate for rest, which varies with individual personality and organizational culture in ways that are not distributed equitably across the team. A recovery provision that is automatic — the engineer is not scheduled for morning meetings on the day following a high-page rotation, and this is enforced by the calendar system, not by the engineer's request — produces recovery rates that are a function of the policy rather than individual advocacy. Burnout leading indicator monitoring: the specific metrics the team tracks as early signals of burnout accumulation — mean overnight pages per rotation per engineer over the trailing four rotations, post-rotation recovery velocity, secondary opt-out rate — with the threshold values that trigger a rotation model review. Leading indicator monitoring converts the on-call model from a set-and-forget design into a feedback loop: the model was calibrated against the system state at the time it was designed, and the leading indicators are the mechanism that detects when the system state has changed enough that the model's calibration is no longer producing the burnout surface it was designed to produce.

Section 5: Incident review model. The incident review model defines how incidents are reviewed after resolution to maintain alert quality, identify systemic failure patterns, and preserve the diagnostic reasoning from complex incidents for future on-call engineers. Three components. Alert quality review process: the quarterly review of alert disposition data — what fraction of each alert type in the prior quarter resulted in engineer action versus no-action-required close, what the mean overnight page rate per rotation was, and what fraction of pages arrived during the identified coverage boundary windows — with defined remediation actions for each finding type. An alert quality review that produces a list of findings without defined remediation actions is a review that satisfies the process requirement without producing the output the process was designed to produce; a review with defined actions and assigned owners for each finding type produces alert quality improvement as a byproduct of the review cycle. Incident postmortem requirements: the content requirements for incident postmortems that make them useful for future on-call engineers facing similar incidents rather than useful primarily for the incident's stakeholders. A postmortem that documents the hypothesis sequence — which hypotheses were evaluated in which order, which were eliminated and why, which was ultimately confirmed and by what evidence — provides the next on-call engineer facing a similar incident with a diagnostic playbook that was produced by a real incident rather than by a tabletop exercise. Connect this to the engineering documentation standards decision record: postmortem content requirements are a form of documentation quality standard applied to incident records specifically — the same principle that produces an ADR gate requirement that checks for record existence rather than record content will produce a postmortem requirement that produces incident write-ups that describe what happened without preserving the diagnostic reasoning that would have reduced resolution time if it had been available to the on-call engineer at the start of the incident. Runbook maintenance protocol: the process by which the incident response runbooks for each monitored system are kept current as the systems evolve, including who is responsible for updating runbooks when systems change, what the review cadence is for runbooks that have not been updated in a defined period, and how runbook currency is signaled to the on-call engineer reading the runbook. A runbook that is accurate for the system as it was designed but not for the system as it currently operates produces a diagnostic tool that is worse than no runbook, because the on-call engineer who follows the runbook and finds that it does not match the system's actual behavior must then decide whether the divergence is a runbook error or a system anomaly — a decision that adds diagnostic latency and uncertainty to an incident that was already requiring their full attention at 3am.

FAQ

How do you calibrate alert severity thresholds without creating so many P1s that on-call engineers stop treating P1 as urgent?

Alert severity calibration fails in two directions: thresholds set too low produce a P1 volume that exceeds the on-call engineer's capacity to treat each page as requiring immediate full attention, which causes engineers to develop heuristics for which P1s actually require a wake-up response — heuristics that exist in the on-call engineer's judgment rather than in the alerting system, and that degrade the P1 response time guarantee for the subset of P1s that do require urgent action. Thresholds set too high produce a P1 volume that understates the actual incident rate, causing conditions with customer-visible impact to spend time in a lower-severity queue before triggering the on-call engineer. The practical calibration approach is to define P1 by its required response time constraint rather than by its severity category: a P1 is any condition that, if left unaddressed for a defined window, will either cause customer-visible impact the on-call engineer cannot remediate retroactively or eliminate the on-call engineer's ability to choose between remediation options. Under this definition, the P1 question for a specific alert condition is concrete: does this condition, if unaddressed for fifteen minutes, cross the customer-visible threshold or foreclose the remediation option space? A quarterly disposition review — examining what fraction of each alert type in the prior quarter resulted in no action required — provides the ongoing feedback loop that catches thresholds that have drifted away from the calibration target as system behavior evolves.

How do you design a follow-the-sun on-call model that doesn't create ownership gaps at handoff windows?

Follow-the-sun ownership gaps have two distinct sources: nominal coverage gaps, where the on-call schedule has no named primary during a time window, and cognitive availability gaps, where a named primary is available but is in a workday-boundary context-switching state that produces acknowledgment or diagnostic latency above the model's designed threshold. Nominal gaps are solved by coverage window extension or secondary pager coverage. Cognitive availability gaps require a different mechanism because they are not visible in the schedule: the coverage window shows the engineer as available, but their actual engagement latency during the first thirty to sixty minutes of their window is elevated. Three mechanisms address cognitive availability gaps. First, place coverage window start times at least thirty minutes into the engineer's workday rather than at the start of it, giving the engineer time to complete their morning sync before going on-call. Second, require a synchronous handoff call between the outgoing and incoming primary at every window transition — ten minutes is sufficient — in which the outgoing primary explicitly transfers ownership of any open incidents with a verbal summary of the current hypothesis and diagnostic state. Third, for window transitions that fall during or immediately after a recurring team sync, schedule the coverage transition after the sync ends rather than at a time that coincides with the sync, preventing the situation where the incoming primary is in a meeting during the first twenty minutes of their coverage window. The handoff call requirement is the most important of the three because it converts the information transfer problem from implicit to explicit: ownership transfer is not confirmed by the calendar showing a new primary; it is confirmed by a ten-minute conversation that ends with the incoming primary saying they have the context.

How do you detect engineer burnout accumulation from on-call before it produces an attrition event?

Burnout accumulation from on-call is invisible in the metrics engineering organizations typically review because it accumulates gradually across rotations, produces no discrete event until the engineer either resigns or requests rotation removal, and is invisible to the engineer's manager in the weeks it is accumulating because daytime performance may not yet show visible degradation. Three leading indicators are measurable before the attrition event. First, sleep disruption count per rotation period: the number of times an on-call engineer receives a page between midnight and 6am local time in a given rotation, tracked per engineer per rotation and aggregated across the team's trailing twelve-rotation history. A team whose mean overnight page rate has increased above the model's designed threshold is accumulating burnout at a rate the rotation's recovery provisions were not designed to absorb; the indicator appears in the alert data before it appears in performance or retention data. Second, post-rotation recovery velocity: the time required for an on-call engineer to return to their pre-rotation output level in the days following the rotation, as measured by the team's existing output metrics. An engineer whose recovery velocity trends longer across successive rotations is accumulating a burnout debt that is not being fully discharged in the inter-rotation period. Third, secondary opt-out rate: the rate at which engineers exercise available mechanisms to reduce their on-call burden. Secondary opt-out rate is a revealed preference signal that carries information the survey-based burnout measurement misses because it reflects the accumulated on-call experience across the prior rotations rather than the engineer's current self-reported state, which is influenced by social desirability effects in survey contexts.

How do engineering on-call model decisions connect to AI chat sessions and decision preservation?

Engineering on-call model decisions are made in AI chat sessions when engineering leaders work through the rotation design — who carries primary responsibility at what frequency, what the coverage windows are, how the rotation frequency was calibrated against the incident arrival rate and the team size — the alert classification model — what observable conditions place an alert in each severity tier, which alternative classification models were evaluated, and why the chosen thresholds were set where they were rather than higher or lower — the escalation path design — what conditions trigger escalation from the primary to a subject matter expert, what the time budget for each escalation tier is, and what authority the primary has to escalate without seeking approval — and the burnout mitigation model — how the rotation was designed to bound the cumulative sleep disruption per engineer, what recovery provisions exist, and what the leading indicators are that the rotation is producing burnout accumulation above the threshold it was designed to prevent. The reasoning in those sessions is the substance of the on-call model — more so than the PagerDuty schedule, the alert threshold configuration, or the escalation policy document — because the model's long-run effectiveness is a function of whether it was designed with an accurate model of how on-call coverage fails rather than an accurate model of what on-call coverage looks like when it is operating correctly. Running those AI chat sessions through the open-source extractor produces the on-call model decision record that allows the next engineering leader to inherit the reasoning about why the rotation frequency was set where it was, why the alert thresholds were calibrated the way they were, and why the escalation path was designed with the specific tier structure it has — rather than inheriting a PagerDuty schedule and discovering its rationale by asking the engineers who designed it, whose accounts of decisions made two years ago may be as unreliable as the accounts the first failure story's technical lead gave of the original benchmarks that had motivated the original service ownership model.

Structural properties set by the engineering on-call model decision

The engineering on-call model ADR: five sections

FAQ

Further reading