The production incident severity framework decision record: why the severity model you chose determines your response resource allocation and your customer communication threshold

A 19-person B2B SaaS building a subscription analytics platform had no formal incident severity framework. The on-call engineer triaged production issues by urgency and feel — critical, urgent, can-wait — without documented criteria for what belonged in each category or what response each triggered. The engineering team had solid monitoring: error rate dashboards, latency percentiles for all APIs, uptime checks for every service endpoint. In the tenth month, a bug in the webhook signature verification handler caused 7% of Stripe webhook events to fail silently. The failure mode was specific: webhooks for subscription upgrade events were being discarded with a signature mismatch error logged to stdout but not to the monitoring stack. The payment API returned 200 on the charge; Stripe showed the payment succeeded; the customer's account was never upgraded. No error rates spiked. No latency thresholds were breached. No uptime check flagged anything. The monitoring stack was comprehensive for the failure modes the engineer had modeled when building it, and the webhook validation failure was not in that model.

The issue ran for eight hours on a Tuesday evening. The on-call engineer saw nothing in the dashboard because the failure mode produced no observable signal against the criteria the monitoring stack was watching. During those eight hours, 14 subscription upgrade webhooks were discarded, 3 customers who had been in active renewal conversations with the sales team experienced checkout failures visible only in their Stripe receipt history, and one customer who had been preparing a procurement decision for a Team tier seat upgrade experienced the checkout failure during a live demonstration to her VP of Engineering. The issue was discovered when a customer filed a support ticket reporting that their account still showed the Free plan after a successful payment. The incident postmortem identified two gaps: no monitoring for webhook processing success rates, and no severity framework criterion that would have required monitoring coverage for revenue processing integrity at the time the monitoring stack was designed. The monitoring gap was a symptom; the severity framework gap was the cause. A severity framework that included revenue integrity as a P1 criterion class would have required monitoring coverage for that criterion class as a precondition for the P1 response model's activation — the monitoring setup session would have included webhook processing success rate as a required metric for the revenue integrity criterion, and the alert would have fired at the 30-minute mark rather than the 8-hour customer discovery. Connect this to the on-call escalation policy decision record: the escalation policy specifies what signals trigger escalation from the primary on-call engineer to additional responders; the escalation trigger list is derived from the severity criteria, so an escalation policy built on a severity framework that lacks revenue integrity criteria will have no escalation trigger for revenue integrity failures regardless of how thorough the escalation model is for the failure modes the framework does cover; the severity framework and the escalation policy are layered — the severity framework determines which failure modes warrant what level of response, the escalation policy specifies how that response is assembled, and a gap in the severity framework propagates directly into a corresponding gap in the escalation policy because the escalation trigger list can only reference criterion classes the framework has defined.

A 37-person developer tools company building a CI/CD observability platform had a P0–P3 incident severity framework documented in their runbook. P0: "service completely unavailable." P1: "major functionality degraded, affecting more than 50% of users or all users of a critical feature." P2: "minor functionality degraded or affecting fewer than 50% of users, service operational." P3: "cosmetic issues or minor bugs with no functional impact." The framework had been developed two years earlier when the engineering team had 11 people and had been refined through five incidents. It served the engineering team well for service availability failures — the triage was consistent across engineers, the escalation triggers were clear, and the P0/P1 response model had been exercised and tested. The framework was availability-centric by design: it measured severity by service state and user impact breadth, not by failure mode type.

In the fourteenth month of operation, a data export bug produced incorrect output for the trace aggregation feature. The export completed successfully, returned HTTP 200, and produced a JSON file with the expected structure and plausible values. The error was in the p95 and p99 latency calculations for services with non-uniform latency distributions: the aggregation algorithm was using the wrong percentile bucket boundary, consistently under-reporting p95 by 15% and p99 by 22%. The service was fully available. The feature worked for the majority of export types. The data returned was wrong, but the service was up. The on-call engineer triaged the bug as P2 — "minor functionality issue affecting a subset of the export feature" — because the service was operational and the majority of export functionality was unaffected. The P2 designation was the correct application of the framework as written. It was also wrong for the business consequence: the P2 triage meant no customer notification was required; the P2 SLA was 48-hour resolution with a status page update rather than a customer communication. Twelve customers received no notification. Three of those customers were using the trace export feature to populate their internal SLO reporting dashboards, and their dashboards under-reported latency for eleven days before one customer's SRE noticed a discrepancy between her WhyChose export p95 values and her in-platform p95 values. The support ticket she filed revealed that the issue had been running for eleven days. The post-incident review identified the severity framework gap: the P0–P3 definitions measured severity by service availability and user impact breadth, with no criterion for systematic data correctness failures affecting customers' operational dependencies on the product's output. Adding a data correctness criterion to the P1 definition — "any failure producing wrong output that customers may use as operational inputs, regardless of service availability status" — would have triaged this as P1 and triggered the 30-minute customer notification SLA at the time the bug was identified, not eleven days later. Connect this to the incident postmortem action item accountability decision record: the post-incident review produced an action item to add data correctness to the P1 criterion definition; the accountability model determines whether that action item is completed or whether it joins the large population of structural postmortem action items that are identified, assigned, and never executed because the next sprint's operational backlog absorbs the engineering capacity that the structural fix requires; the severity framework improvement and the postmortem accountability model are linked at exactly this junction — the gap is found in the postmortem, the fix is obvious and small (a two-sentence addition to the runbook), and the fix never ships because the accountability model has no mechanism to protect structural improvements from operational preemption.

A 63-person enterprise SaaS building a financial data aggregation platform for institutional clients had a severity framework that was among the most mature the engineering team had encountered at their size. P0: "complete service unavailability or data loss with confirmed customer impact." P1: "major functionality degraded, affecting more than 25% of requests or any SLO-protected API, customer communication required within 30 minutes." P2: "degraded functionality affecting fewer than 25% of requests, 4-hour resolution target, status page update required." P3: "minor degradation with no SLO impact, 24-hour resolution target." The framework had been revised seven times over three years, each revision triggered by an incident that revealed a gap in the previous version. P0 had been updated twice to add data loss coverage. P1 had been updated to add the 25% request threshold (originally 50%) after an incident where 30% request degradation was triaged as P2. The calibration across engineers was consistent; the escalation model was well-exercised; the customer communication protocol for P0 and P1 was practiced and efficient.

In the twenty-second month, one of the platform's third-party financial market data feeds began returning stale prices. The feed's API was responsive, returning HTTP 200 at normal latency. The prices were in the normal numerical range for the assets covered. The response structure was unchanged. The staleness was 4 to 6 hours — prices from mid-afternoon in a feed that should be delivering real-time quotes. The engineering team had no alert for data freshness on this feed; the monitoring stack was built against service-observable signals (error rates, latency percentiles, availability) because those were the signals the severity framework had defined as the criteria for P0 and P1 escalation, and the monitoring coverage was built to match the framework. The data feed staleness produced no observable signal against any criterion in the framework. The issue ran for 19 hours. During those 19 hours, three institutional clients used the platform's data to calculate derivative pricing for end-of-day positions. Their calculations used 4-to-6-hour-old prices as current prices. When their reconciliation jobs ran the next morning and compared their calculations against the exchange settlement prices, two clients discovered significant discrepancies. Both clients filed regulatory compliance notifications with their regulators as required by their internal procedures for data integrity events affecting financial calculations. The engineering team learned about the incident from a regulatory notification forwarded by one of the clients, not from any monitoring alert. The post-incident review categorized the incident as a P0 in retrospect — data quality failure with regulatory compliance impact on enterprise clients — and identified the severity framework gap: the framework had no criterion class for third-party data quality failures, because every prior incident that had refined the framework had been a service availability failure visible in engineering metrics. The data quality criterion wasn't absent because it had been considered and rejected; it was absent because no prior incident had produced a data quality failure that surfaced the gap, and the framework development process had not included a deliberate criterion coverage analysis independent of the incident history. Connect this to the on-call alert threshold calibration decision record: the alert threshold calibration model determines monitoring coverage for the failure modes the severity framework has defined; a severity framework that lacks a third-party data quality criterion class generates no requirement for data freshness monitoring in the calibration session, and the calibration session produces a monitoring stack with complete coverage for the failure modes in the framework and zero coverage for the failure modes not in it; the severity framework is the upstream decision that determines which failure modes the organization treats as worth monitoring, and the threshold calibration model is downstream — calibrating sensitivity for those failure modes; a gap in the severity framework propagates to a gap in the monitoring stack not through negligence but through the logical structure of the two decisions: if the severity framework does not classify third-party data quality failures as P0/P1, there is no organizational pressure to build P0/P1-calibrated monitoring for data freshness, and the calibration session correctly produces no such monitoring.

Structural properties set by the severity framework decision

Two structural properties are determined by the founding decisions about severity criterion classes, response models per severity level, customer communication protocol, escalation and downgrade authority, and calibration review cadence. Neither is visible in incident count metrics, MTTR dashboards, or SLO compliance reports until a failure mode outside the framework's design scope produces a response that is technically correct by the framework as written and wrong for the business consequence the incident produced.

Property 1: Response resource allocation and the criterion class gap. The response resource allocation is the match between the severity level assigned to an incident and the responders engaged at that severity level. The failure mode is not inconsistent triage — engineers who apply the same framework to the same incident and reach different severity levels. The failure mode is systematic under-categorization of entire failure mode classes that the severity framework was not designed to cover. When a criterion class is absent from the severity framework, every incident in that class is assigned severity by the remaining criteria, and the remaining criteria consistently produce under-categorization because they were not built for that incident type. A data correctness failure assigned by availability criteria becomes P2 or P3 because the service is operational and the error rate is zero; the correct severity by business impact is P1 because customers are making decisions based on wrong data. The under-categorization is not inconsistency across engineers — it is the consistent, logically correct output of a framework applied to an incident outside its design scope. Both engineers who triage the P2 data correctness failure as P2 are applying the framework correctly; the framework itself is wrong for the failure mode. The consequence is a response model that assembles the correct team for service availability failures and assembles an insufficient team — or no additional team at all beyond the primary on-call — for data correctness, revenue integrity, and third-party quality failures. The customer success team is not paged. The engineering leads are not escalated. The 30-minute customer communication SLA does not activate. The silence persists until a customer discovers the failure independently, at which point the customer communication is reactive rather than proactive, the engineering response is already hours behind where it should be, and the incident record reflects a categorization that was technically accurate and organizationally wrong. Connect this to the on-call escalation policy decision record: the escalation policy derives its trigger list from the severity framework criterion classes; a criterion class gap in the severity framework is simultaneously an escalation trigger gap in the escalation policy, because the escalation policy specifies what triggers escalation from P2 to P1 response, and if the failure mode that should be P1 is being assigned P2 by the framework, the escalation trigger that would move the response to P1-level resources never fires; the two decisions must be designed together so that every criterion class in the severity framework has a corresponding escalation path in the escalation policy, and every escalation path in the escalation policy corresponds to a criterion class in the framework.

Property 2: Customer communication threshold and the engineering-versus-business signal gap. The customer communication threshold is the point at which the severity framework mandates customer notification and the content of what customers are told. Most engineering severity frameworks use engineering-observable signals as their primary severity criteria because those signals are visible to the engineers who designed the framework and who respond to incidents in real time: error rates, latency percentiles, service availability, request success rates, SLO breach probabilities. The consequence is a framework calibrated to the failure modes most visible to engineers and under-calibrated to the failure modes most impactful to customers. Silent failures produce no error rate signal. Data quality failures return correct HTTP 200 responses. Revenue processing errors are not visible in any engineering metric until a reconciliation job runs. Third-party data quality degradations produce no changes in the metrics the monitoring stack was built to watch. A framework with engineering-calibrated criteria and no customer impact calibration will systematically fail to trigger customer communication protocols for exactly the incidents that customers care most about — the framework correctly identifies that no engineering SLO was violated, correctly determines that no P0 or P1 threshold was crossed, correctly concludes that no customer communication is required, while the customer has been using incorrect data to make business decisions for hours. The customer communication threshold becomes a lagging indicator rather than a proactive instrument: customers learn about the incident when they discover the consequence independently, which is always worse for customer trust than proactive communication at the time the incident is detected. Engineering severity frameworks must include customer impact criteria that map to business impact signals — data correctness, revenue integrity, regulatory obligation impact — not only to engineering availability signals, if the customer communication protocol is to fire before customers discover the problem on their own. Connect this to the engineering team communication channel decision record: the incident response model requires specific channel infrastructure — a dedicated incident coordination channel, a customer communication staging area, a cross-team path to customer success and legal — and a channel structure that lacks defined routing rules for incident types beyond service availability failures will produce ad-hoc coordination for data correctness and revenue integrity incidents; the team that needs to assess whether a data quality failure triggers regulatory obligations has no defined channel to receive the escalation; the team that needs to draft the customer communication has no defined path to receive the technical summary from the incident commander; the severity framework defines what happens when the incident is recognized, and the channel structure defines how that response assembles — a gap in either propagates to a gap in the response.

The production incident severity framework ADR: five sections

Section 1: Severity level definitions and criterion coverage. The severity level definitions specify the complete set of criterion classes the framework uses to determine incident severity, the thresholds for each class at each severity level, and the governing principle that severity is determined by the highest-severity criterion that applies — not by the most common criterion, not by the average, not by the primary failure mode if secondary failure modes in a higher criterion class are also present. The minimum criterion coverage for an engineering organization managing a production system with paying customers includes six classes. First, service availability: the fraction of requests returning errors or not completing, with P0/P1/P2 thresholds calibrated to the organization's SLO commitments. Second, latency degradation: percentage increase from baseline above SLO-protected percentile thresholds, measured at the customer-observable API boundary rather than at internal service boundaries. Third, user impact breadth: the number or fraction of users experiencing the failure, used alongside the other criteria (not as the sole determinant) to differentiate a P2 affecting a small subset from a P1 affecting a large fraction. Fourth, data correctness: any failure producing output that is structurally valid — correct format, plausible values, no error code — but incorrect in its content; the data correctness criterion applies regardless of service availability status, because a service that is fully available and returning wrong data can produce larger business impact than a service that is partially unavailable; the P1 threshold for data correctness is lower than for service availability because the customer impact of wrong data that is acted upon extends backward to the full duration of the silent failure, not forward from discovery. Fifth, revenue processing integrity: any failure affecting subscription state, billing, payment entitlement, or account permissions, including failures in downstream webhook processing, payment provider integrations, or entitlement propagation that result in a mismatch between the customer's payment state and their account state; the P1 threshold for revenue integrity is any confirmed mismatch affecting at least one paying customer, because revenue integrity failures compound silently and the cost of under-escalation exceeds the cost of over-escalation in nearly all cases. Sixth, regulatory or contractual obligation impact: any failure that triggers or may trigger customer regulatory reporting requirements, violates contractual SLA commitments in customer agreements, or produces data that customers may be required to report to external authorities; the P0 threshold for regulatory impact is any confirmed failure affecting an enterprise customer's regulatory obligations, because the timeline for regulatory notification is determined by the customer's obligations and is typically measured in hours rather than the engineering MTTR. The governing principle — severity is the maximum across all criterion classes — is critical for multi-criterion failures where a single incident triggers criteria in different classes: a payment processing failure that is simultaneously a service availability failure and a revenue integrity failure must be assigned the highest severity of the applicable criteria, not an average or a judgment call about which criterion is "primary." Connect this to the on-call escalation policy decision record: the escalation policy's trigger list is a direct mapping of the criterion classes in the severity framework; every criterion class at P0 and P1 must correspond to a specific escalation trigger, and the escalation trigger must specify the additional responders required for that criterion class — a revenue integrity P1 escalates differently from a service availability P1, requiring customer success and billing engineering rather than infrastructure engineering and on-call backup; a criterion class gap in the severity framework is simultaneously an escalation trigger gap and a required responder gap, because the escalation policy cannot specify a response team for a failure mode the severity framework has not defined.

Section 2: Response model per severity level and criterion class. The response model specifies who is paged, in what order, what role each responder holds in the incident response structure, and what constitutes "engaged" for each responder — not acknowledged, but actively participating in the incident bridge. The response model must be specified for each criterion class as well as for each severity level, because the required responders differ by incident type in ways that a flat severity-level model cannot capture. A P1 data correctness incident requires product management (to assess which customers' workflows are affected), customer success (to draft and send customer communication), and possibly legal (to assess whether any contractual SLA has been breached), in addition to the engineering responders who own the affected service. A P1 service availability incident requires infrastructure engineers, service owners, and the on-call backup, but typically not customer success or legal until the customer communication SLA activates. A P1 revenue integrity incident requires billing engineering, the payments service owner, and customer success, potentially without requiring the broad infrastructure escalation that a service availability P1 triggers. A flat "P1 = add two more engineers" model correctly handles service availability P1s and assembles the wrong team for data correctness and revenue integrity P1s, because those failure modes require responders from outside engineering who are not on the on-call roster and who will not be paged by a standard engineering escalation. Four additional specifications: response model activation verification (the incident commander must confirm that each required responder is active in the incident bridge, not merely paged — an unreturned page does not constitute response model activation, and the secondary escalation path must be triggered if the primary responder has not acknowledged within the specified timeout); incident commander role activation (for P0 and P1 incidents, the incident commander role must be explicitly activated, not implicitly assumed — the first engineer to acknowledge an incident is not automatically the incident commander; incident commander activation requires explicit acknowledgment and is recorded in the incident timeline); maximum time from severity assignment to response model activation (specified per severity level — P0: 5 minutes, P1: 15 minutes — and tracked as a metric in the post-incident review; the gap between severity assignment and response activation is the window during which the incident is running without the resources the severity level mandated); unavailable responder protocol (if a required responder for a given criterion class is unavailable, the response model specifies the secondary escalation path and the maximum delay before that path is activated — there is no scenario in which a P0 revenue integrity incident proceeds with the primary billing engineer unavailable without an explicit secondary escalation). Connect this to the incident postmortem action item accountability decision record: the response model specifies what should happen during an incident; the postmortem produces action items when the response model was not followed — wrong severity assignment, missing escalation, delayed response model activation; the accountability model that governs whether those action items are completed determines whether the response model is enforced through iteration or whether the same response failures recur across multiple incidents; an organization whose postmortem accountability model has no mechanism to protect structural improvements from operational preemption will correctly identify response model failures in each postmortem and will not improve the response model because the action items to update it are consistently deprioritized against the next incident's operational demands.

Section 3: Customer communication protocol. The customer communication protocol specifies the obligation, timeline, format, owner, and update cadence for customer-facing communication for each severity level and each criterion class. The protocol must differentiate between communication types because the appropriate communication vehicle differs by incident type and customer segment. A status page update is appropriate for service availability incidents with broad impact across the customer base — all customers are affected, all customers can see the status page, the engineering summary is the communication. A targeted in-app notification or direct email is appropriate for incidents affecting a specific feature or a defined subset of customers — broadcasting a service availability issue to customers who were not affected is confusion, not communication. Direct outreach is required for incidents affecting named enterprise customers, incidents with data correctness implications where the customer may have taken action based on wrong data, and incidents with potential regulatory obligation impact — the customer needs to know from the team, by name, before they discover it through reconciliation or regulatory review. Four specifications that must be explicit: first-communication SLA by severity and criterion class (a P1 service availability incident and a P1 data correctness incident may have the same 30-minute first-communication SLA, but the content of the first communication differs — a service availability communication describes the observed degradation; a data correctness communication must include the time range of affected data, which outputs were affected, and whether customer action is required to assess or correct impact); ownership of communication drafting (the incident commander drafts the technical summary specifying what happened, what is affected, and what is being done; customer success translates that summary into customer-facing language and sends it; the translation step exists because the engineering summary contains accuracy and the customer communication requires clarity, and the same person is rarely excellent at both under incident time pressure; the ownership split must be explicit and must be activated by the severity/criterion class combination, not by manual handoff); update cadence during active incidents (P0 and P1 incidents require proactive updates every 30 to 60 minutes regardless of whether the technical status has changed — "investigation ongoing, no update to impact scope" is a customer communication, not a non-event; the absence of an update during a P0 incident is experienced by customers as abandonment, and the communication SLA must specify the no-change update cadence explicitly rather than assuming it is understood); post-incident summary format (the post-incident summary is a required incident deliverable, not an optional quality-of-life addition; for data correctness and revenue integrity incidents, the summary must specify whether customer action is required to assess or remediate the impact — customers who used incorrect data to make decisions may need to re-evaluate those decisions, and the post-incident summary is the mechanism by which that information reaches them). Connect this to the engineering team communication channel decision record: the customer communication protocol requires a defined channel path from the incident bridge to the customer communication staging area to the customer success team drafting the customer-facing message; a channel structure without a defined routing path for incident customer communication will produce ad-hoc Slack DMs between the incident commander and the customer success lead, which are fine for one incident and fail at three simultaneous incidents when the incident commander is managing two P1s and cannot maintain an informal communication chain to both customer success leads; the channel structure decision and the customer communication protocol must specify the same routing path from different directions — the channel structure specifies where incident customer communication staging happens; the protocol specifies who is responsible for populating that staging area and what the timing requirements are.

Section 4: Severity escalation and downgrade authority. The escalation and downgrade protocol specifies who can change severity during an active incident, in which direction, under what conditions, and what must be recorded when a severity change is made. The asymmetry in authority between escalation and downgrade is a deliberate design choice: any responder can escalate severity; only the incident commander can downgrade it. The asymmetry exists because the costs are asymmetric. The cost of under-escalation during an active incident — delayed customer communication, insufficient response resources, extended time to mitigation because the criterion class requiring additional expertise was not escalated — is typically higher than the cost of over-escalation, which is additional responders who are released early once the scope is clarified. An on-call engineer who identifies that a service availability P2 has a data correctness component that warrants P1 treatment should escalate immediately without needing the incident commander's approval; the friction cost of requiring approval before escalation is higher than the false-positive cost of an escalation that turns out to be unwarranted. Three additional specifications: escalation trigger for new criterion class discovery (when a responder identifies that the incident affects a criterion class not reflected in the current severity assignment — finding a data correctness component in what was triaged as a service availability incident, discovering that a payment processing failure has revenue integrity implications — the escalation is triggered immediately and the incident timeline records the discovering responder, the new criterion class, and the basis for the criterion class assignment; the escalation is not a criticism of the initial triage — the initial triage was correct given the information available at the time, and the escalation reflects new information, not a triage error); downgrade requirements (severity downgrade requires the incident commander to record in the incident timeline: which criterion that previously applied no longer applies, the evidence basis for the determination that it no longer applies, and confirmation that the customer communication protocol for the original severity level has been satisfied — a P1 cannot be downgraded to P2 if the 30-minute customer communication SLA has not been met, even if the technical scope has narrowed; the communication obligation is tied to the highest severity reached, not the current severity); retrospective severity assessment (at incident close, the incident commander records in the incident timeline: the initial severity assigned, the final severity reached, the time elapsed between initial triage and the final severity assignment, the criterion class or classes that were not in the initial triage but were added during the incident, and the mechanism by which those additions were identified; this retrospective assessment is the primary input to the calibration review in Section 5 and must be completed before the incident is formally closed). Connect this to the on-call alert threshold calibration decision record: the alert threshold calibration model determines the monitoring signal sensitivity for the failure modes the severity framework has defined; if the calibration session is run with a bias toward reducing false positives in the alert stream, it may silence alerts that fire below the current P2 activation threshold but above the threshold at which a new criterion class would require P1 treatment — the severity escalation model and the threshold calibration model must coordinate on the threshold at which an alert becomes a P1 trigger so that the calibration session does not inadvertently silence the signals that would have triggered a severity escalation from P2 to P1; a calibrated threshold that produces a true positive rate for P1-severity events of less than 90% has a calibration error that the escalation model cannot correct, because the escalation model only operates once the alert fires and the severity is initially assigned, and an alert that was silenced in calibration never reaches the triage point where escalation is possible.

Section 5: Post-incident severity calibration review. The calibration review specifies how the severity framework is updated based on operational experience, how gaps identified in the retrospective severity assessments from Section 4 feed into framework revisions, and what the cadence and authority are for revisions. Three components. Incident-triggered calibration check: every P0 and P1 incident triggers a severity calibration check as a required component of the standard postmortem process; the calibration check asks three questions — was the initial severity assignment correct given the information available at initial triage; how long did it take to reach the correct severity level, and what delayed it if the initial assignment was wrong; and did the incident reveal a criterion class gap, a threshold calibration error, or an escalation trigger gap; the calibration check output is a structured record (incorrect initial assignment: yes/no; reason if yes; gap type: criterion class / threshold / escalation trigger; proposed revision if yes) that is aggregated with the outputs from all prior calibration checks to identify patterns; a single calibration check finding might be a one-off triage error, but three findings identifying the same criterion class gap across three different incidents is a framework revision requirement. Semi-annual framework revision cycle: severity criterion definitions, threshold specifications, and criterion class coverage are reviewed every six months against the aggregate of calibration check outputs from the prior period; the review produces either a documented confirmation that the current definitions are correctly calibrated, or a specific revision addressing observed gaps; every revision is documented as an ADR update with the rationale for each change, the calibration evidence that motivated the change, and the incidents that surfaced the gap — the severity framework version history is itself a decision record, and the audit trail of how the organization's understanding of its failure modes has evolved is a first-class artifact; a framework that has been in place for three years with no documented revisions either has perfect calibration, which is unlikely, or has not been maintained through the calibration review process, which is the common case. Deliberate criterion class coverage analysis: at least once per year, the severity framework undergoes a criterion class coverage analysis independent of incident history — this analysis asks what failure modes the platform could produce that the current criterion classes would systematically under-categorize; the analysis reviews the product's data flows and their correctness requirements, the external dependency landscape and each dependency's failure mode space, the contractual and regulatory obligations that apply to specific customer segments, and the revenue processing surface area; the preventive analysis is designed to identify criterion class gaps before an incident surfaces them, which is the mechanism by which the framework extends its coverage beyond the failure modes the organization has already experienced; AI-assisted design sessions using the analysis questions as prompts produce a higher-quality gap inventory than a single-engineer review because the AI assistant can surface failure mode categories that the reviewing engineer has not encountered in prior incidents; running those design session exports through the open-source extractor before the calibration review retrieves the explicit reasoning about which criterion classes were included and which were not, including the reasoning behind criterion coverage decisions that were made at design time but never recorded in the framework documentation itself — a severity framework designed with the help of a ChatGPT or Claude design conversation contains more explicit consideration of alternatives than most engineering runbooks, and that consideration is most valuable to the calibration reviewer who is evaluating whether a gap was deliberately excluded or simply not considered; the distinction shapes the revision approach, because a deliberately excluded criterion class should be re-examined against new evidence, while a criterion class that was never considered requires evaluation from first principles.

FAQ

Further reading

• On-call escalation policy decision record — the escalation policy specifies what triggers escalation from the primary on-call engineer to additional responders and what the additional responders' roles are; the escalation trigger list is derived directly from the severity framework criterion classes, so a criterion class gap in the severity framework is simultaneously an escalation trigger gap in the escalation policy; the two decisions must be designed together so that every criterion class at P0 and P1 has a corresponding escalation path, and every escalation path corresponds to a criterion class — an escalation policy for a severity framework that has no data correctness criterion will have no escalation trigger for data correctness failures regardless of how thorough the escalation model is for service availability failures, and the data correctness P1 will run with P2-level resources until a customer discovers it.
• Incident postmortem action item accountability decision record — the severity calibration check in Section 5 generates action items: add a criterion class, revise a threshold, update an escalation trigger; the accountability model determines whether those action items are completed or whether they join the large population of structural postmortem improvements that are identified in retrospective reviews and never executed because the operational backlog absorbs the engineering capacity the structural fix requires; severity framework improvements are particularly vulnerable to accountability failure because they are small changes to a documentation artifact (the runbook) with high organizational leverage, and because the next incident that demonstrates the gap will not arrive for days or weeks after the action item was generated, giving the operational backlog time to absorb the engineering attention that the fix requires before the consequence of not fixing becomes visible again.
• On-call alert threshold calibration decision record — the alert threshold calibration model determines monitoring coverage for the failure modes the severity framework has defined; a severity framework that lacks a third-party data quality criterion class generates no requirement for data freshness monitoring in the calibration session, and the calibration session produces a monitoring stack with complete coverage for the failure modes in the framework and zero coverage for the failure modes not in it; additionally, a calibration session that biases toward false positive reduction may silence signals that are below the current P2 activation threshold but above the threshold at which a new criterion class would require P1 treatment, producing a calibrated monitoring stack that correctly handles the failure modes the framework anticipated and correctly fails to alert on the failure modes it did not.
• Engineering team communication channel decision record — the incident response model requires specific channel infrastructure — a dedicated incident coordination channel, a customer communication staging area, a defined routing path to customer success and legal — and a channel structure without routing rules for incident types beyond service availability failures produces ad-hoc coordination for data correctness, revenue integrity, and regulatory impact incidents where the response team composition is non-standard and the communication path to non-engineering responders is informal; the customer communication protocol in Section 3 specifies who drafts the customer-facing message and when, but the channel structure must provide the path by which the technical summary from the incident commander reaches the customer success team; without that defined path, the communication handoff fails under the cognitive load of active incident response regardless of how clearly the protocol specifies the obligation.
• Open-source extractor — severity frameworks designed with AI-assisted design conversations contain more explicit consideration of alternatives and criterion class tradeoffs than most engineering runbooks; the AI conversation is the design session where the engineer thinks through which failure modes should be P0 versus P1, which criterion classes are in scope, and which failure modes were deliberately excluded; running those design session exports through the extractor before a calibration review retrieves the explicit reasoning that was in the design conversation but was never transcribed into the framework documentation itself; the distinction between a criterion class that was deliberately excluded and one that was simply not considered is exactly the information the calibration reviewer needs, and it is most reliably available in the AI design session rather than in the reviewer's recall of a design conversation that may have happened a year or two earlier.