Blog
Long-form essays on engineering decision records, ADR conventions, AI chat as a thinking tool, and how teams keep their reasoning findable. New posts ship roughly weekly. The narrative arc starts with the new-CTO onboarding problem and works outward.
2026-07-23 · ~26 min read
API contract testing decisions are made in the initial integration session (installs Pact because it's the most popular library, writes a few consumer interactions, sets up provider verification in CI — without documenting who owns the contracts or what the 'can I deploy?' enforcement policy is), the API change session (adds a new required request field and ships it without running consumer pacts because the change 'only adds things'), and the multi-team expansion session (a second consumer team uses the API without registering their pact). What none of these sessions produce is the contract ownership model, the 'can I deploy?' gate configuration, the backward compatibility definition at the matcher level, or the scope boundary between what contract tests verify and what complementary test types must cover — the gaps that determine whether contract testing prevents breaking changes or documents them after consumers break in production.
2026-07-23 · ~26 min read
Distributed locking decisions are made in the initial background job session (asks ChatGPT how to prevent concurrent execution, gets Redis SETNX with a TTL set to 'comfortably longer than expected runtime'), the incident response session (migrates to Redis Redlock after a single-node Redis failure), and the compliance session (adds an audit log without reviewing the lock's failure model). What none of these sessions produce is the lock backend failure model specifying which scenarios the chosen backend cannot prevent (Redlock's documented clock-drift and AOF-persistence-gap windows), the TTL derivation methodology (worst-case runtime under load, not expected runtime in testing), or the idempotency requirement — whether the protected operation needs additional correctness mechanisms beyond the lock itself for when the lock's documented failure scenarios actualize.
2026-07-23 · ~27 min read
Chaos engineering decisions are made during the initial adoption session (installs a chaos platform, runs experiments in staging, publishes a blog post about chaos engineering), the tool configuration session (schedules the first production experiment), and the incident-triggered session (adds a new experiment targeting the failure mode just discovered in production). What none of these sessions produce is the experiment authorization model specifying who has authority to run production experiments and under what pre-conditions, the blast radius scope policy specifying which systems are in-bounds and how newly added dependencies enter scope, or the steady-state hypothesis framework specifying measurable falsification criteria — the gaps that determine whether a circuit breaker validated in staging holds under production-scale connection pool exhaustion, whether a 99.7% quarterly GameDay pass rate reflects current system resilience or a confidence boundary that expired when a new analytics SDK was added after the last GameDay, and whether 'the system remained healthy during the experiment' is a measurable guarantee or a subjective assessment by an engineer watching a monitoring dashboard.
2026-07-23 · ~28 min read
CI/CD pipeline security decisions are made during the initial setup session (pins actions by tag not digest, adds static cloud credentials as repository secrets, deploys to production without a required reviewer gate), the dependency addition session (installs marketplace actions without reviewing permissions or pinning strategy), and the compliance remediation session (adds SAST scanning without addressing the action pinning model or credential type that made the audit necessary). What none of these sessions produce is the pipeline trust model specifying how third-party action references are made immutable, how cloud credentials are scoped to environments and made short-lived via OIDC, or how human approval gates prevent a single compromised merge from deploying arbitrary code to production.
2026-07-22 · ~28 min read
VPC design decisions are made during the initial cloud setup session, the service proliferation session, and the security hardening or compliance preparation session. What none of these sessions produce is the CIDR allocation rationale with its peering compatibility inventory, the subnet segmentation model specifying which services belong in each tier, or the security group convention determining whether internal rules use SG references or VPC CIDR ranges — the decisions that determine whether an enterprise partnership peering attempt is blocked by an overlapping default CIDR, whether a compromised frontend can reach the database directly through a permissive CIDR-based rule, and whether the isolated subnet's absence of a NAT gateway route is a deliberate security control or an incidental gap.
2026-07-18 · ~28 min read
GitOps decisions are made during the initial ArgoCD or Flux setup session, the infrastructure migration session, and the incident response session where an engineer runs kubectl apply directly and discovers that the reconciliation loop silently reverted the fix. What the AI session does not produce is the sync model specifying what happens to direct kubectl changes during the reconciliation interval, the secret management policy that prevents database credentials from being committed as base64-encoded YAML, or the reconciliation failure alert — the gaps that determine whether a manual kubectl fix during an incident persists or is silently overwritten every three minutes, whether a password committed as a Kubernetes Secret YAML is readable in git history by anyone with repository access, and whether a broken manifest push is detected in five minutes or the following morning.
2026-07-18 · ~30 min read
Distributed tracing decisions are made during the initial observability setup session, the performance debugging session, and the service mesh configuration session. What the AI session does not produce is the sampling model that specifies what happens to slow-request traces when the sampling rate is 1%, whether trace context propagation format conflicts across service boundaries break span correlation, or what the storage cost is at production request volume — the gaps that determine whether a P99 latency incident produces actionable traces of the slow requests or empty search results, whether a cross-service request produces a unified waterfall or four disconnected root spans, and whether the tracing system costs $50 or $800 per month.
2026-07-18 · ~29 min read
Cookie and session management decisions are made across three AI sessions that never communicate: the initial authentication session, the horizontal scaling session, and the security hardening session. None produces a session rotation requirement, a SameSite specification, or a secret rotation procedure — the gaps that determine whether a penetration test finds a session fixation vulnerability, whether a Redis outage takes down authentication across all servers, and whether a GET endpoint performing a state change is protected by your cookie policy.
2026-07-18 · ~25 min read
Package dependency decisions are made during the project bootstrapping session, the CI reproducibility debugging session, and the supply chain security hardening session. What the AI session does not produce is the pinning model specifying whether the lockfile is committed and whether CI uses npm ci or npm install — the difference between a malicious transitive dependency being blocked by content hash verification and reaching production 84 minutes after publication, whether a private registry outage causes a developer to bypass security controls with a --registry flag, and whether a critical CVE in a transitive dependency is remediated within 24 hours or sits unpatched because the remediation procedure was undocumented.
2026-07-18 · ~24 min read
TLS certificate decisions are made during the initial HTTPS setup session, the security hardening session, and the service mesh configuration session. What the AI session does not produce is the renewal automation failure mode specification, the cipher suite selection rationale with an enterprise client compatibility audit, the OCSP stapling configuration, or the mTLS specification for service-to-service authentication — the decisions that determine whether a certbot systemd timer breakage is discovered by a 3am customer error report or by a 30-day expiry alert, whether disabling TLS 1.2 for compliance breaks 8% of enterprise integrations, and whether the internal CA root certificate expires without a rotation procedure.
2026-07-17 · ~22 min read
DNS decisions are made during the domain registration session, the CDN migration session, and the infrastructure failover session. What the AI session does not produce is the TTL model per record type, the automated failover configuration, the DNSSEC rationale, or the split-horizon model — the decisions that determine whether a primary server failure triggers automatic rerouting in 90 seconds or leaves 40% of users hitting a failed IP for 60 minutes, and whether enabling DNSSEC during setup creates a 48-hour DS record propagation dependency when you need to disable it during a CDN migration incident.
2026-07-17 · ~25 min read
Database index decisions are made during performance optimization sessions, schema migration sessions, and incident response sessions. What the AI session does not produce is the index coverage rationale, the write amplification model, the slow query detection strategy, or the schema evolution contract — the decisions that determine whether indexes added one at a time across two years accumulate into 40GB of overhead on a 6GB table, whether seven indexes with zero scans in 90 days are identified before INSERT latency approaches a webhook timeout, and whether an index accumulation problem is diagnosed correctly or misattributed to hardware capacity.
2026-07-17 · ~25 min read
Database backup decisions are made when the team first deploys a production database, configures infrastructure, or writes the first disaster recovery runbook. What the AI session does not produce is the backup verification model, the RPO specification per data class, the backup storage isolation model, or the restoration time analysis — the decisions that determine whether a backup that exits 0 nightly can actually be restored, whether a ransomware event can delete every backup copy using the same compromised credentials, and whether the estimated 45-minute restoration becomes 11 hours when tested against the current production database size.
2026-07-16 · ~26 min read
Zero trust decisions are made when the team first enables remote access, adds contractors, or responds to a security audit finding a flat internal network. What the AI session does not produce is the microsegmentation policy, the device trust model, the access review cadence, or the lateral movement analysis — the decisions that determine whether a phished contractor credential grants access to the production database or only to the staging environment the credential was provisioned for.
2026-07-16 · ~26 min read
Load balancing decisions are made when the team first deploys to production with multiple backend instances. The AI session configures a load balancer, adds backends, sets a health check, and verifies traffic distribution. What it does not produce is the health check specification, the session affinity model, the connection draining policy, or the active-active failover analysis — the decisions that determine whether a database-level failure goes undetected for forty minutes, whether a corporate NAT routes all enterprise users to a single backend instance, and whether every rolling deployment silently drops in-flight requests.
2026-07-16 · ~25 min read
Edge caching decisions are made when the team first adds a CDN or reverse-proxy cache to reduce origin load and improve response time. The AI session sets TTLs on static assets and API responses and verifies cache hits. What it does not produce is the cache key model, the invalidation strategy, the bypass policy, the stampede protection mechanism, or the stale-while-revalidate contract — the decisions that determine whether authenticated users receive each other's personalized content, whether a critical price change reaches users before the TTL expires, and whether the origin survives the synchronized expiry wave that follows a cache-warming event.
2026-07-15 · ~28 min read
Webhook delivery decisions are made when the team first needs to push event notifications to external consumers. The AI session produces a working sender with an HTTP POST and a retry loop. What it does not produce is the delivery guarantee tier, the retry storm interaction with the backoff model, the endpoint verification and replay prevention model, the payload schema versioning contract, or the dead letter queue policy — the decisions that determine whether missed events silently corrupt downstream data or retry storms collapse the consumer that just recovered from an outage.
2026-07-14 · ~27 min read
API rate limiting decisions are made in the security hardening session or the API design session when the team first needs to protect infrastructure from abuse. What the session does not produce: the consumer isolation model, the distributed enforcement consistency guarantee, the bypass exemption policy, or the burst allowance model. Two stories: a SaaS startup adds per-IP token bucket rate limiting during a security sprint — 14 months later their largest enterprise customer, 800 employees behind a single NAT gateway, reports the product is completely unusable during business hours, and the manually maintained Redis exemption is later wiped by a maintenance flush because it was never documented in code; and a developer tools company migrates Redis to a managed service using a week-old snapshot with near-empty counters, producing a burst window where automated integration partners send 4× their contractual quota for eight minutes because counter state continuity was never documented as a migration requirement.
2026-07-14 · ~30 min read
GraphQL subscription decisions are made when the team needs to push real-time updates to clients and wants a typed, schema-governed alternative to ad-hoc WebSocket messages. The AI session that chose subscriptions produces a working WebSocket connection and a live-updating UI. What it does not produce: the connection scale model, the backpressure policy for slow consumers, or the message delivery guarantee. Two stories: a developer tools startup adopts WebSocket-based subscriptions for real-time dashboards — at 50 concurrent users excellent, at 4,200 the Node process hits the OS file descriptor limit; and a fintech adds subscriptions for live trade execution updates, a market volatility burst queues 40,000 events for a single slow mobile client, the Node process OOMs. Both failures were determined in founding sessions that documented the subscription transport but not the scale model, backpressure policy, or delivery guarantee.
2026-07-14 · ~25 min read
Multi-cloud strategy decisions are made when the team selects their cloud provider and defines what multi-cloud means for them. What the session does not produce: the workload placement policy, the provider lock-in inventory with migration cost estimates, the portable abstraction requirements, or the cross-cloud operational model. Two stories: a startup adopts multi-cloud with Terraform over 18 months, accumulates 23 AWS-specific services and 4 GCP services in a tightly-coupled architecture where neither cloud can function independently — gaining two cloud bills without gaining failover capability; and a fintech spends 2,400 engineering hours replacing working managed services with self-managed alternatives to achieve cloud neutrality, then discovers the enterprise requirement they were satisfying meant multi-region deployment within one provider, not active-active across two.
2026-07-13 · ~26 min read
Container image decisions are made in the first containerization session when the team needs to package the application for consistent deployment. What the session does not produce: the base image selection criteria, the layer cache strategy, the vulnerability scan threshold, or the image registry retention policy. Two stories: a startup uses node:18 for twelve services over twenty-two months with no base image policy — a security scan returns 847 CVEs and CI builds take 9–18 minutes with no layer cache; and an analytics platform standardizes on alpine without documenting the rationale, a new engineer uses node:20-slim for a financial calculation service, and musl/glibc rounding differences produce a systematic 0.3–0.7% revenue calculation error invisible to all tests, discovered through a customer escalation four months after deployment.
2026-07-13 · ~27 min read
Serverless decisions are made when the team needs to run code without managing servers. The AI session that chose Lambda produces a deployed function. What it does not produce is the cold start budget, the execution duration classification policy, or the vendor lock-in exit criteria — the decisions that determine whether Lambda remains the correct compute model as workloads grow, or becomes the ceiling on latency two years later.
2026-07-13 · ~26 min read
ORM decisions are made in the database interaction design session when the team needs to talk to the database and wants to avoid writing SQL by hand. What the session does not produce: the N+1 query detection policy, the raw SQL escape hatch criteria, the migration review requirement for table-locking ALTER TABLE operations on large tables, or the production migration protocol. Two stories: a team discovers fourteen N+1 endpoints through customer complaints six months after they shipped, and a startup loses nineteen minutes of write availability to a CREATE INDEX on a fourteen-million-row table that the migration generator produced identically to the one-row development table it was tested on.
2026-07-13 · ~24 min read
A twelve-person team migrates to gRPC for a measured serialization performance problem, grows to thirty-one engineers and fourteen services, and spends six months on a field rename that is correct at the binary level because the proto schema was shared across services owned by five teams with no documented schema ownership model, no deprecation window policy, and no migration tracking mechanism. A developer tools startup builds bidirectional streaming services for server-side clients, adds a browser-based admin dashboard two years later, and discovers that bidirectional streaming is not supported in any stable gRPC-Web proxy implementation — requiring a server-side aggregation layer that was not in the roadmap. Covers the three structural properties the gRPC transport decision determines: the schema ownership model and the coordination surface (who has authority to change shared proto definitions; the breaking versus non-breaking change classification; the deprecation window policy that drives removal from never to a defined timeline; the migration tracking mechanism across teams with different deployment cadences); the streaming model and the client compatibility ceiling (unary is gRPC-Web compatible; server-side streaming is gRPC-Web compatible with proxy configuration requirements; client-side and bidirectional streaming are not supported in stable gRPC-Web implementations; browser client access requires a server-side aggregation layer or service refactoring); and the error model and the observability surface (gRPC status code inconsistency produces incorrect HTTP status codes at the gateway; UNKNOWN maps to HTTP 500 and inflates error rates for validation errors that should be HTTP 400; binary-encoded protobuf bodies require proto schema registration for request body debugging that JSON services get for free). Five ADR sections: transport protocol selection and performance vs. compatibility rationale; proto schema ownership model, evolution policy, and breaking change criteria; streaming model selection and client compatibility matrix; error model design and gateway error translation policy; observability strategy for binary-encoded traffic and proto schema registry.
2026-07-12 · ~22 min read
A SaaS company deletes a flag that has been false in production for eleven months — the LaunchDarkly default value, the last-modified date, the absence of targeting rule changes all suggest a completed rollout — and causes three hours and fourteen minutes of broken checkout for twenty-three enterprise accounts because the flag was a compliance kill switch keeping four accounts on the legacy checkout flow for a SOC 2 re-audit; the targeting rules showed four account-level overrides that were never documented in the flag metadata; the cleanup PR was reviewed and approved by two engineers who also read "false for 11 months" as "completed rollout, safe to delete." A developer tools startup discovers that forty-seven of their one hundred and twelve flags are permanently true with no intent to change them, twelve gate dead code paths that accumulated deprecation warnings suppressed with filters, and the 22-millisecond p99 latency regression from the performance audit traces to a synchronous database read added incrementally to the main request handler — one context attribute at a time, each introduced by an engineer adding a new context-dependent flag — that now runs on every request regardless of whether the request's code path encounters any of the eighteen context-dependent flags. Covers the three structural properties the flag lifecycle decision determines: the flag lifecycle model and the dead-code accumulation rate (temporary versus permanent versus kill switch versus experiment is the taxonomy that determines whether cleanup is a completion or a deferral; dead code paths accumulate at the retirement rate, which defaults to zero without a lifecycle policy); the targeting model and the configuration surface complexity (rule accumulation per flag makes deletion risk unassessable from the default value alone; the configuration surface grows with flag count, targeting rule depth, and flag catalog age); and the flag evaluation overhead and the performance ceiling (local rules store versus remote API call is the evaluation model; context assembly cost — not evaluation time — is the ceiling that accumulates as context-dependent flags move into the hot request path). Five ADR sections: flag system selection and flag type taxonomy; flag lifecycle policy and retirement criteria; targeting model and configuration surface definition; evaluation performance budget and hot-path flag policy; flag ownership, audit cadence, and dead-flag detection mechanism.
2026-07-12 · ~20 min read
A fintech startup documents twenty-eight runbooks in Confluence with a post-incident update policy that works — twenty-six runbooks not involved in recent incidents accumulate seventeen to twenty-two months of drift from routine system changes (a Kubernetes migration, a UI path consolidation, an org restructuring) that trigger no documentation update signal; a junior on-call engineer executing the database connection pool runbook at 2am follows a path that returns 404, a config file that no longer exists, and an escalation role that was eliminated eight months ago; forty-seven of ninety-three minutes of P0 duration are caused by stale procedures. A developer-tools SaaS assigns API documentation ownership to the endpoint author and discovers two years later that enterprise customers built CSV-to-JSON workaround pipelines around an output format parameter that existed in the code for eighteen months but was never added to the Notion documentation page because the sprint retrospective item was never assigned. Covers the three structural properties the documentation strategy decision determines: the ownership gap rate and the staleness detection mechanism (ownership assignment produces accurate documentation at creation; the detection mechanism — CI spec diff, periodic runbook dry-run, architecture review — determines the useful life of documentation, not the creation date), the docs-as-code versus wiki divergence rate (co-location with code creates a natural enforcement checkpoint at PR review time; wiki-based documentation requires a social norm for updates with no enforcement mechanism), and the undiscoverable surface accumulation rate (closed Slack threads, PR descriptions, and stale wiki pages that appear authoritative but describe a system state that no longer exists). Five ADR sections: documentation platform selection and docs-as-code versus wiki decision rationale; ownership model, assignment mechanism, and ownership transfer protocol; staleness detection mechanism and documentation currency policy; canonical location policy and undiscoverable surface control protocol; API documentation contract enforcement gate and runbook test cadence.
2026-07-12 · ~22 min read
A developer productivity tool company builds a Wasm plugin system to let enterprise customers extend the product with custom logic — fourteen enterprise plugins ship over two years; a wasmtime 12-to-19 upgrade breaks all fourteen simultaneously because the WASI preview 2 interface definitions changed between RC0 and RC2, seven plugins use affected interfaces, and two plugins cannot be recompiled because the plugin authors left and the source was not escrowed. A data transformation startup runs CPU-DoS exposed for fourteen months because wasmtime's fuel metering API — the instruction-counting mechanism that terminates infinite loops — was never wired into the execution harness; the gap between "Wasm provides sandbox isolation" (true) and "Wasm protects against algorithmic complexity attacks" (false without explicit fuel metering configuration) was discovered in a security audit commissioned for an enterprise procurement review. Covers the three structural properties the WebAssembly execution model decision determines: the sandbox security boundary and the capability provisioning model (Wasm provides memory isolation and import-controlled host access by design; it does not protect against algorithmic complexity attacks by default — fuel metering must be explicitly configured), the performance ceiling and the execution model selection (AOT-compiled Wasm within 10-20% of native for CPU-bound workloads; cold-start instantiation overhead 0.5-3ms per invocation — on the critical path for serverless-style execution patterns; module instance pooling required for latency-sensitive workloads), and the component model evolution and the plugin ecosystem lock-in surface (WASI preview 1 and preview 2 are binary-incompatible; a preview 1 plugin system migration to the component model is a breaking change to every published plugin API). Five ADR sections: execution model selection and use case rationale; sandbox security model, threat level, and capability provisioning policy; runtime selection and WASI version target; language support matrix and component model adoption plan; performance budget, resource limits, and cold-start constraint.
2026-07-12 · ~22 min read
A SaaS company grows from 12 to 65 engineers over 26 months without restructuring its founding three-squad model — an onboarding flow change that should take two weeks requires coordination across four squads with misaligned sprint cadences, a notification API that does not support per-customer templates, a platform team three sprints backlogged, and an undocumented billing-change freeze; eleven weeks later, the feature is live. A developer tools company forms a four-engineer platform team at 22 people to eliminate redundant infrastructure work — eighteen months and 94 open tickets later, every product squad describes the platform team in retrospectives using the same phrase: "we're waiting on platform." Covers the three structural properties the topology decision determines: the cognitive load ceiling and the service ownership surface (the maximum domain a stream-aligned team can hold in working memory — measured by onboarding time, cross-team coordination fraction, and service surface breadth — and what happens when a new engineer takes four months to ship independently rather than two weeks), the team interaction mode and the collaboration overhead accumulation rate (collaboration vs X-as-a-service vs facilitating, and why persistent collaboration mode grows coordination overhead with the square of team-pair relationships rather than linearly), and the organizational boundary and the ownership gap rate (how components enter the system without documented owners and why ownership gaps are the most reliable leading indicator of future incident response failures). Five ADR sections: team topology model selection and organizational stage rationale; domain boundary definition and cognitive load assessment per team; platform team scope and self-service threshold; team interaction mode and collaboration protocol per team-pair; topology review cadence and restructuring criteria.
2026-07-12 · ~20 min read
A team's auth layer grows to 14 services each using a different JWT validation library because the "clean it up later" debt decision from the founding sprint was never written as a policy — 'later' never arrived, and when a security audit finds five distinct validation code paths with three different expiry policies and one CVE-vulnerable implementation, the remediation takes 11 weeks instead of the 4 that were scoped. A CTO mandates 20% sprint capacity for technical debt but provides no prioritization criteria — engineers improve rarely-touched code with clean abstractions while high-traffic hot paths remain untouched and velocity stays flat. Covers the three structural properties the debt tolerance decision determines: the classification model and accumulation trigger (deliberate deferral vs knowledge debt vs architectural drift vs entropy — each requires a different discovery mechanism), the visibility mechanism and the recency bias problem (point-in-time discovery captures only code currently being touched; systematic discovery requires periodic processes that examine stable subsystems), and the repayment ownership model and the priority displacement mechanism (who decides what gets scheduled, what the prioritization criteria are, and what completion looks like). Five ADR sections: debt classification model and tolerance threshold; debt registry structure and discovery cadence; repayment ownership model and prioritization criteria; refactoring scope definition and completion standard; architectural debt review cadence and escalation triggers.
2026-07-11 · ~22 min read
A 35-person data team pins Airflow 2.1.4 in the founding sprint and discovers 14 months later that the dynamic task mapping feature they need requires Airflow 2.3 — the migration takes six weeks, not one, because 23 production DAGs use the deprecated API, six break on 2.3's hook changes, and the Slack notification plugin they rely on has no maintained 2.3 release. A 20-person analytics team uses dbt Cloud scheduling alongside existing Airflow DAGs without documenting whether dbt is the transformation layer or the orchestrator — when an incremental model is added to both scheduling paths, the 9 AM dbt Cloud run advances the watermark before midnight Airflow processes its batch, and a finance dashboard shows "no new data" for three days before a two-day investigation traces the cause. Covers the three structural properties the orchestration choice determines: the task dependency model and its expressiveness ceiling (static DAGs vs dynamic task mapping vs functional pipelines — the ceiling is set at tool selection time), the backfill model per pipeline type (date-partitioned batch vs incremental watermark vs event-triggered — each requires a different recovery procedure that must be documented at design time), and the orchestration boundary and the two-path failure mode. Five ADR sections: orchestration framework selection and version governance; transformation layer boundary and orchestrator responsibility; pipeline type classification and backfill model; failure notification model and on-call integration; pipeline dependency management and shared table ownership.
2026-07-11 · ~22 min read
A 29-person SaaS applies a rolling update while a NOT NULL column migration is in-flight — half the fleet crashes before rollback completes because rolling deployment runs old and new code simultaneously and old code has no null-handling for the new column. A 41-person platform uses blue/green for instant rollback, but smoke tests don't cover production credentials — a deploy that passes all 47 smoke scenarios fails when the green environment's SMTP credential is 11 months stale. Covers the four structural properties the deployment strategy choice determines: rollback speed and mechanism (rolling re-deploy takes minutes; blue/green load-balancer flip takes seconds; canary abort depends on automated signal), database migration compatibility constraint (rolling requires backward-compatible expand-migrate-contract for every schema change; blue/green requires only pre-flip/post-flip migration sequencing), production validation surface (smoke tests vs canary real-traffic sampling vs feature-flag-gated release), and infrastructure cost and complexity overhead. Five ADR sections: deployment model selection and rationale; backward compatibility constraint and migration discipline; pre-deploy validation surface and smoke test requirements; rollback procedure and stability window; multi-region deployment sequencing and promotion criteria.
2026-07-11 · ~18 min read
A 45-person SaaS loses a senior engineer after she handles 23 weekend pages in a single quarter — 8x the median — because alert routing was configured at launch and never audited as the team grew. A 35-person API platform's follow-the-sun rotation fails its first complex cross-timezone incident because the handoff model is a 5-line Slack message and the incoming engineer spends 90 minutes re-doing investigation already completed. Covers the five decisions the on-call rotation design record should contain: rotation schedule and participant breadth (rotation length, participant selection, compensation model), alert routing model and toil classification baseline (actionable vs automatable vs noise split, per-engineer distribution, reduction target), handoff model and incident state transfer protocol (five-section template, overlap window design, follow-the-sun handoff quality), primary/secondary coverage model and escalation chain (single-tier vs two-tier vs domain-tiered, secondary availability expectation, cross-domain incident command), and on-call health review cadence and rotation modification criteria (the metrics reviewed quarterly, the conditions that trigger a rotation redesign, why attrition is the most expensive feedback signal).
2026-07-11 · ~26 min read
A 55-person HR tech company's Vanta dashboard shows 39 of 47 controls as green for eight months. The auditor requests evidence packages. Six manual controls have never had evidence collected — the compliance lead assumed the monitoring dashboard showed audit-ready evidence, but "monitored" means "no current deviation," not "auditor-acceptable evidence artifact exists." A 28-person fintech passes SOC 2 Type I cleanly, then adds a mobile app, payment reimbursement API, and analytics service over 18 months without a compliance scope review. When pursuing Type II, the auditor extends the system boundary to include all three new products and finds 14 control gaps — mobile JWT storage, unmonitored analytics pipeline, unreviewed bank account data in the privacy notice. Covers the five decisions the compliance automation decision record should contain: compliance framework and tool selection (criterion scope, connector coverage inventory), evidence coverage model and manual evidence cadence (per-control classification, collection cadence, owner, reminder mechanism), system scope boundary and expansion trigger criteria (event-driven scope review embedded in the service launch checklist), auditor relationship and audit type progression (Type I to Type II milestones, system description commitments), and control gap remediation process and pre-audit dry run cadence.
2026-07-11 · ~22 min read
A B2B SaaS celebrates eleven months of 99.97% uptime while their background job processor has been silently failing for 34 cumulative hours — the SLI measured the API tier's HTTP health check, not the notification delivery behavior users paid for. A consumer fintech sets 99.95% SLO without calculating that their 4-deploy-per-day cadence consumes 32 minutes of error budget per month from deployments alone against a 22-minute monthly allowance; the freeze that follows is discovered through velocity collapse, not through the budget dashboard. Covers the five decisions the SLO decision record should contain: SLI selection and user-experience validity (the blind spots that health-check SLIs cannot detect), SLO target and error budget model at projected scale (the per-deploy budget calculation that should precede the target choice), error budget policy and velocity trade-off thresholds (default freeze behavior, override process, and approver roles), burn rate alerting thresholds and incident response tiers (fast-burn and slow-burn windows, the dead zone below 6× burn rate, and traffic volume minimums), and the SLO review cadence and target evolution protocol (when targets should change and what analysis the change requires).
2026-07-11 · ~25 min read
A startup's acquisition due diligence reveals that four years of incremental AWS managed service adoption — DynamoDB single-table design, SQS FIFO, EventBridge, Cognito — has created a replatforming estimate of 28–40 engineering weeks that nobody had ever calculated. Each individual service choice made sense; the cumulative proprietary coupling was never inventoried. A 25-person SaaS team cannot evaluate a proposal to migrate from Kubernetes to App Runner because nobody documented why Kubernetes was chosen over ECS or Fargate two years ago, and the three Kubernetes-specific features in the cluster turn out to be incidental adoption rather than deliberate architectural choices. Covers the five decisions the infrastructure decision record should contain: cloud provider selection and proprietary service inventory, account and organizational structure and blast radius boundary, network topology and data transfer cost model, compute abstraction model selection and workload constraints, and the provider coupling review cadence and replatforming cost estimate.
2026-07-10 · ~25 min read
A VP Engineering's first formal cost audit after three years of growth surfaces $14,200/month of unnecessary AWS spend: 73% of EC2 cost is on-demand despite 22 months of stable utilization, cross-region S3 replication runs to a bucket that no application accesses (the DR plan that justified it was replaced 14 months earlier), and 35-day backup retention is set on all 12 development and staging databases because that was the Terraform module default. None of the configurations were wrong when set — each had a justification. But the justification was never written down, so when it evaporated it left behind cost. Covers the five decisions the cost optimization decision record should contain: commitment model selection per workload tier, resource tagging policy and cost attribution hierarchy, waste detection policy and drift trigger criteria, reserved capacity commitment sizing and renewal cadence, and the review cadence that produces an audit trail instead of a recurring investigation.
2026-07-10 · ~28 min read
A missing read routing policy lets stale replica reads oversell 47 products during a Black Friday sale: all SELECT queries route to the replica, no distinction is made between browse reads (tolerate lag) and inventory reads (must be current), and during peak load the replica falls 42 seconds behind the primary. An undocumented replication lag ceiling produces a 247-second data loss window during a disaster recovery failover that the DR plan documented as zero RPO — the plan correctly cited Multi-AZ synchronous replication for the us-east-1 HA scenario but applied the same guarantee to a cross-region async read replica used as the tier-2 DR target. Covers the five decisions the read replica decision record should contain: replication topology and lag target, read routing policy and consistency requirements per query type, read-after-write consistency strategy, failover procedure and RPO/RTO documentation, and observability instrumentation for replication health.
2026-07-10 · ~30 min read
A Databricks Delta Lake deployment that adopts Change Data Feed, liquid clustering, and column mapping across 12 tables creates a managed-service coupling surface that costs three months and $47,000 to unwind when an acquisition forces a migration to Snowflake — a cost that was invisible at platform selection time because no one inventoried which Delta Lake features were Databricks-proprietary extensions versus open-specification. An undocumented schema evolution policy on an Iceberg lakehouse lets a data scientist delete two struct fields, then a retraining pipeline time-travels to historical snapshots where those fields existed, applies the post-deletion schema, silently drops the values from the feature array, and produces a model with degraded precision that runs in production for two months before anyone detects the corruption. Covers the five decisions the data lakehouse decision record should contain: open table format selection and managed-service dependency, file layout policy and compaction strategy, schema evolution policy and reader compatibility contract, concurrent write model and orchestration requirements, and observability instrumentation for data quality and performance.
2026-07-10 · ~30 min read
A render-level assignment call (Math.random() in the React component) mismatches the unit of analysis (task completion at the user level), inflating variance so severely that an 8-week experiment with 180,000 tasks fails to detect a real 14% improvement in completion rate. A team that peeks at p-values daily and ships at p<0.05 operates at a 26% false positive rate rather than 5%, accumulating 24 shipped "winners" over 18 months while the holdout group shows identical metrics — zero aggregate effect. Both outcomes trace to the same missing decision record: the assignment model, statistical framework, and stopping rule policy were not documented when the experimentation infrastructure was built. Covers the five decisions the A/B testing infrastructure decision record should contain: assignment model and randomization unit selection, statistical framework and stopping rule policy, concurrent experiment model and interaction management, metric hierarchy and guardrail policy, and observability instrumentation for experiment validity including Sample Ratio Mismatch detection.
2026-07-10 · ~30 min read
A checkout service with no circuit breaker hammers a degraded payment API until thread pool exhaustion takes down checkout, order management, and the product catalog — a 40-minute payment degradation that became a full platform outage. A retry storm that fires immediately on failure doubles the load on a recovering database, sustaining the I/O spike for 90 minutes after the triggering VACUUM ANALYZE completes in 11. Both outcomes trace to the same missing decision record: the failure handling strategy, timeout hierarchy, and retry policy for the service fleet were never written as a consistent set of decisions. Covers the five decisions the resilience pattern decision record should contain: service dependency graph and failure mode inventory, circuit breaker configuration and threshold rationale, retry policy and backoff strategy (including idempotency surface), timeout hierarchy and deadline propagation, and observability instrumentation for resilience patterns.
2026-07-09 · ~28 min read
Runtime configuration starts with environment variables because they are universally supported, and the strategy is never documented with a secret rotation policy or environment parity requirement. A 60-person e-commerce company discovers during their first quarterly credential rotation that a 90-second maintenance window is unavoidable because the application reads the database password once at startup with no hot-reload mechanism. A 35-person SaaS company traces an elevated database load investigation to a staging–production environment variable name mismatch (REDIS_URL vs REDIS_HOST/REDIS_PORT) that silently bypassed Redis caching in production for eight months. Both outcomes trace to the same missing decision record: the access model, rotation procedure, and environment parity enforcement mechanism were not documented at strategy selection time. Covers the five decisions the configuration management decision record should contain: storage and access model selection, secret lifecycle management and rotation policy, environment parity enforcement and configuration drift prevention, configuration validation and startup gate policy, and observability instrumentation for configuration failures.
2026-07-09 · ~30 min read
A Protobuf field number is removed and reused for a new field type; consumers running the old schema silently interpret payment reference IDs as idempotency keys, corrupting a reconciliation report without a single deserialization error. An Avro schema change passes the schema registry's BACKWARD compatibility check; the Java consumer crashes in production because fastavro 1.4.7 encodes nullable nested union fields incorrectly and the Java reference implementation correctly rejects the malformed bytes. Both outcomes trace to the same missing decision record: the field lifecycle policy was not documented at format selection time. Covers the five decisions the serialization decision record should contain: format selection and wire format rationale, schema registry strategy and compatibility enforcement, schema evolution compatibility policy and field lifecycle management (including Protobuf field number reservation and Avro migration timeline), language ecosystem requirements and client library governance, and debugging tooling and observability instrumentation for serialization failures.
2026-07-09 · ~28 min read
A team adopts Apollo Federation without a subgraph ownership model and discovers three weeks into a type rename that entity type names are shared across all subgraph extensions and cannot be renamed in a single-subgraph operation. A second team inherits an undocumented Schema Stitching configuration built by a consulting firm and spends four months rebuilding cross-boundary queries when migrating to federation because stitching delegation logic must be rewritten, not translated, for the federation entity resolution model. Both outcomes trace to the same missing decision record: the federation gateway architecture, subgraph ownership model, and breaking change policy were not documented at adoption time. Covers the five decisions the federation decision record should contain: gateway approach selection and version, subgraph ownership and type boundary policy, breaking change detection and deprecation process, authentication and authorization propagation model across subgraph boundaries, and query plan caching strategy and resolver blast radius policy.
A 60-person B2B SaaS adds a usage-based tier and discovers the application has no metering infrastructure — usage events go to the app log with no customer attribution, a field rename six months earlier created an identifier gap in the historical data, and the in-product usage dashboard the enterprise customers were promised cannot be satisfied by Stripe's metered billing API. A 4-month rebuild replaces what engineering scoped as a 2-week sprint. A 25-person developer tools startup hard-codes Stripe calls across seven services, then spends six weeks migrating to a billing abstraction when a $280k ARR enterprise deal requires purchase-order invoicing. Both outcomes trace to the same undocumented decision: the billing architecture was chosen without documenting the metering data requirements, the payment abstraction boundary, or the pricing iteration mechanism. Documents the five decisions that belong in a billing architecture ADR: revenue model selection and ceiling documentation, payment processor selection and abstraction boundary, metering infrastructure and usage data model, revenue recognition model and audit trail requirements, and pricing iteration mechanism and plan lifecycle management.
2026-07-09 · ~38 min read
A 45-person startup builds Backstage with a 4-person platform team. Two years later 9 of 15 plugins are unmaintained, the Backstage version is 14 months behind, the service catalog is 40% stale, and the platform team spends 60% of its capacity on maintenance rather than new capabilities — time-to-first-PR has drifted back from 1 day to 3 days. A 20-person startup picks Railway for its zero-ops deployment model. At 55 people, a SOC 2 enterprise deal requires VPC isolation the PaaS cannot satisfy without an enterprise plan, and a 3-week migration uncovers hardcoded Railway internal DNS references in three services that required migrating all dependent services simultaneously. Both outcomes trace to the same undocumented decision: the platform was chosen without written reasoning about self-service scope, maintenance cost model, or scale thresholds. Documents the five decisions that belong in a platform engineering ADR: platform scope and self-service ceiling, build vs. buy vs. PaaS decision and maintenance cost model, onboarding sequence and time-to-first-PR target, platform maintenance model and contribution governance, and migration trigger criteria and scale thresholds.
2026-07-09 · ~35 min read
A 35-person SaaS company builds RBAC with three roles — Viewer, Editor, Admin — in the multi-tenant sprint and never writes down the model. Four years later the role table has 47 rows, the exception_grants table has 312 entries with no expiry dates, and SOC 2 audit preparation takes three weeks because nobody can enumerate who can access a given record without tracing through four tables. A 20-person developer-tools company chooses ABAC with OPA/Rego at founding, accumulates 120 policies, loses the engineer who understood Rego at month nine, and freezes all permission model changes for six months while evaluating a rewrite. Both outcomes trace to the same undocumented decision: the access control model was chosen without writing down the expressiveness ceiling, the exception grant policy, the audit trail requirements, or the migration path. Documents the five decisions that belong in an access control model ADR: model selection and expressiveness rationale, role or policy inventory and growth cap, permission enforcement point architecture, audit trail requirements and the "who can access this resource" query, and migration path documentation and trigger criteria.
2026-07-08 · ~30 min read
A 45-person B2B SaaS company launches /api/v1/ with URL versioning, ships /api/v2/ with breaking changes and a 6-month migration deadline, and watches 30% of integration partners ignore the deadline. Eighteen months later /api/v1/ still gets 12% of traffic from clients nobody can identify because the client identification query was never built. A 20-person developer tools company chooses header versioning (X-API-Version: 2) for internal services, then opens the API to external developers and discovers that generated SDK clients and log aggregation both require explicit header parsing that URL versioning makes unnecessary. Both outcomes trace to the same undocumented decision: the versioning scheme was chosen without written reasoning about log segment visibility, SDK distribution surface, routing infrastructure complexity, or the client identification model needed to enforce deprecation. Documents the five decisions that belong in an API versioning ADR: versioning scheme selection and routing infrastructure rationale, backward compatibility commitment and breaking change classification, client identification and usage tracking model, deprecation timeline and enforcement triggers, and SDK versioning alignment and distribution strategy.
2026-07-04 · ~28 min read
A 40-person fintech adds Semgrep to CI after a SOC 2 audit and inherits 312 findings on day one. Nobody documents what the false-positive threshold is, what the remediation SLA is, or who triages new findings. Twelve months later the queue has 847 open items — 60-70% estimated phantom results from ORM parameterization that the SAST rule cannot model — and nobody can determine which findings represent genuine vulnerabilities. A 25-person developer tools company integrates Snyk and gates on medium-severity findings; within a week, 80% of pull requests are blocked by three transitive CVEs in code paths the application never calls; the CI gate is disabled and the scanner is demoted to a nightly Slack message nobody reads. Both outcomes share the same undocumented decision: scanner selection, rule set configuration, gate threshold, and remediation ownership were all set in the same afternoon under compliance pressure, without reasoning about false-positive rate, triage overhead, or the escalation model for contested findings. Documents the five decisions that belong in a security scanning ADR: scanner selection and scanning approach rationale, rule set configuration and false-positive management policy, severity gate threshold and delivery velocity trade-off, dependency vulnerability scanning scope and SCA configuration, and scanning cadence, finding queue ownership, and remediation SLA.
2026-07-04 · ~32 min read
A 28-person SaaS company uses Docker Compose for local development. Nobody wrote down why Docker Compose over Dev Containers, what the acceptable production-parity divergence was, or who owns the configuration when it drifts. New engineer onboarding takes four days — three of them finding undocumented manual steps and a Postgres 13 vs 15 version mismatch that produces one production incident every five weeks. A 15-person developer tools company chose the highest-fidelity local setup: kind with Telepresence. When M-series MacBook Pros arrived, two of nine container images had no arm64 variants and the setup stopped working. No documentation existed of the original tooling decision, so five engineers spent ten days diagnosing a failure mode that would have taken an afternoon if the original architecture choice had been recorded. Documents the five decisions that belong in a developer experience ADR: local environment strategy and technology selection, production-parity surface and acceptable divergence policy, onboarding sequence and time-to-first-PR target, service dependency management and local overrides, and environment maintenance ownership and version drift policy.
2026-07-04 · ~35 min read
A 22-person TypeScript startup sets up Turborepo at 12 packages — fast, obvious, never written down. At 64 packages, the free remote cache tier fills, LRU eviction begins purging the shared design system outputs, and 37 downstream packages rebuild cold every Tuesday morning: $400/month in paid cache tier fees that were discoverable at selection time from the free tier's 10 GB storage model. A 32-person fintech team migrates six Java microservices to a Gradle monorepo, then adds Go services built with Makefiles. A Protobuf schema change is tracked correctly by Gradle through the Java dependency graph but is invisible to the Makefile's timestamp-based change detection — the Go services ship to staging with the stale schema and the cross-language dependency gap takes 90 minutes to diagnose, because nobody documented that the Go build was outside the build system's dependency graph. Both failures trace to the same undocumented decision: the build system was chosen during the first sprint as an obvious configuration default, without documenting its incremental build granularity, remote cache storage model, or cross-language dependency visibility surface. Documents the five decisions that belong in a build system ADR: build system selection and polyglot requirements, incremental build strategy and change detection granularity, remote cache architecture and eviction policy, dependency graph enforcement and cross-project visibility, and CI integration and affected-package detection.
2026-07-03 · ~30 min read
A consumer fintech app ships a weekly release cadence decided during the first sprint and never written down. A critical payment race condition is discovered eighteen months later: the hotfix takes 54 hours to reach iOS users via App Store review — but 39% of users still do not update, and the team has no forced upgrade mechanism and no server-side version gate, because the server API never documented which headers old client versions send. A B2B field service app deprecates its authentication endpoint 60 days after shipping the replacement; 22% of infrequent users are still on the old client and encounter a complete login outage the day the endpoint is removed. Both failures trace to the same undocumented decision: the release cadence was set as a scheduling choice without documenting its implications for hotfix window latency, forced upgrade surface, or server-side API backward compatibility window. Documents the five decisions that belong in a mobile deployment ADR: release cadence and emergency release procedure, forced upgrade policy and minimum version enforcement, server-side API backward compatibility window, feature flag strategy for undecoupled mobile releases, and staged rollout strategy with rollback procedure.
2026-07-03 · ~30 min read
A fintech company designs payment.completed with six flat fields. Twelve microservices build against it. Two years later, adding a merchantId field requires eleven weeks, six synchronized service releases, and two maintenance windows — because nobody documented what the contract was, which consumers depended on specific fields, or whether the schema guaranteed backward compatibility. An e-commerce platform embeds a full product snapshot in every inventory.updated event. The catalog team restructures the product image schema. Three consumer services silently start producing incorrect alt texts. The failure is discovered six weeks later during a mobile app audit. The root cause: an undocumented event boundary decision that embedded a copy of the product schema rather than referencing the canonical product identifier. Documents the five decisions that belong in an event-driven architecture ADR: event schema structure and field ownership, schema versioning strategy, consumer coupling surface registry, event payload boundary policy, and breaking-change migration procedure.
2026-07-03 · ~28 min read
A multi-tenant SaaS shards by organization_id. Tenant-scoped queries run in under 100ms. The analytics dashboard that aggregates across all tenants requires scatter-gather across 16 shards and loads in 45 seconds. A marketplace shards orders by created_at date. Archiving old data is easy. Black Friday puts 100% of new writes on one shard, which saturates at 2.8× normal peak, and 14,000 order transactions fail before the team can add a second writer. Both failures trace to the same undocumented decision: the shard key was chosen to solve the scaling problem of the moment, not the cross-shard query cost and rebalancing surface it would impose for the life of the system. Documents the five decisions that belong in a database sharding ADR: shard key selection and query pattern analysis, shard count policy, rebalancing strategy, cross-shard query handling, and shard key migration path.
2026-07-03 · ~26 min read
A startup runs on m5.4xlarge instances for three years because the founding engineer chose them on the first AWS deploy and nobody questioned them. A growth-stage company copies auto-scaling parameters from a blog post and discovers on a flash sale day that their scale-out latency is 4 minutes — 3 minutes longer than their traffic spike's rise time. Both outcomes trace to the same undocumented decision: the provisioning model was chosen once under the constraints of the moment and never written down with explicit reasoning. Documents the five decisions that belong in a capacity planning ADR: provisioning model, instance sizing, horizontal vs. vertical scaling policy, auto-scaling configuration, and capacity buffer policy.
2026-07-03 · ~25 min read
A junior engineer receives a PagerDuty alert at 2am while the senior on-call is on vacation. The runbook says to check the db_connections panel — the panel was renamed eight months ago in a Grafana migration. The runbook's next step links to runbook.internal/background-jobs — the domain was decommissioned four months ago when the company moved from a self-hosted wiki to Confluence. By the time the CTO is reached at 3:17am, the database has exhausted all connections and the API is returning 500 for every request. A follow-the-sun team has a payment webhook incident start two minutes before the rotation handoff; two engineers open parallel investigations, one restarts the webhook service and masks the root cause, and the other — who had already identified the issue as a mismatched Stripe signing secret after a key rotation — applies the fix 22 minutes later. Both outcomes trace to the same undocumented decision: the incident response process was designed once during a production crisis and allowed to drift without maintenance obligations tied to the infrastructure changes that made it incorrect.
2026-07-03 · ~24 min read
A devtools company deprecates a webhook API with a 12-month notice and no forcing function — eighteen months later, 37% of webhooks still hit the v1 endpoint, two timeline extensions later the endpoint is finally removed, and three enterprise customers' CI pipelines break in production because the deprecation email was never read by the engineers who owned the consuming code. A platform team with 17 known consumer services sends one email to the engineering distribution list; 60 days into a 90-day migration window, 12 of 17 services are still on the old endpoint because the email was read by managers who forwarded it to teams who never put the migration in a sprint; the sunset takes 8 months. Both failures trace to the same undocumented decision: there was a sunset date but no forcing function and no communication protocol that reached the people with code to change.
2026-07-02 · ~23 min read
A B2B SaaS team adds PII to 23 tables over 18 months without a data map — when the first GDPR right-to-erasure request arrives, their DELETE on the users table misses 8 tables containing user references, including session logs, billing addresses, and activity events that contain embedded email addresses; they discover this 14 days later when the user sends a second request, and the correct erasure takes 6 engineering-hours plus 2 additional hours to build the data map that should have been maintained incrementally. A healthcare-adjacent SaaS cannot determine within GDPR Article 33's 72-hour breach notification window whether a leaked read-replica credential exposed health data, because no one documented whether the 47 intake survey questions constitute special-category health data under Article 9; the legal review takes 12 days; the supervisory authority issues a €15,000 fine for the late notification. Both outcomes are determined when the first PII field enters the database without a classification schema.
2026-07-02 · ~22 min read
A B2B SaaS with Elasticsearch default BM25 grows a synonym dictionary to 400 entries over three years while users still get zero results for queries that mean the same thing as indexed content — because closing the vocabulary alignment gap requires semantic embeddings, and migrating from a BM25-only index to hybrid dense retrieval after the fact requires a full index migration that takes four months. A marketplace with an LTR model built on engagement features launches a new vertical where all items have no purchase history and all LTR features are zero — within-category ranking collapses to BM25, which the LTR model suppressed as its weakest feature, and the cold-start fallback takes six weeks to implement. Both failures are set when the first ranking model is chosen.
2026-07-02 · ~22 min read
A recommendation team deploys a gradient-boosted tree behind a Flask endpoint on CPU; fourteen months later they train a two-tower neural network that achieves 22% better offline CTR but takes 180ms p99 on CPU while the product page SLA requires sub-50ms — and migrating to GPU TorchServe takes six weeks because the Flask/pickle/CPU choice was never written down as a choice with a latency-floor consequence. A fintech credit risk model runs as a nightly batch scoring job for eight months before the business adds instant loan approvals with a three-second SLA, requiring a three-month migration to build an online inference endpoint and a feature caching layer that nobody planned to build. The inference serving architecture you choose determines whether real-time predictions are possible, whether a new model architecture can be deployed on the existing fleet, and whether a bad model deployment can be rolled back in minutes.
2026-07-02 · ~23 min read
A recommendation model trains with natural log normalization in a Jupyter notebook; the backend engineer reimplements the same feature in Go using log base 10 — both values are numerically plausible, neither raises an error, and the model delivers 11% CTR in production against 18% offline evaluation for three months. A fintech team computes fraud velocity features in Redis and trains the fraud model from a database export; eight months later they need to retrain but Redis has no historical state — the recomputed training features differ from the production-served values in 12% of examples due to different event timestamp semantics, producing a retrained model with 7% lower recall. The feature computation architecture you choose determines whether training and serving share a transformation contract or accumulate divergences that surface as unexplained model degradation.
2026-07-02 · ~22 min read
A customer success SaaS builds a nightly Airflow batch pipeline in the founding month; four months later a product manager requests a 4-hour-freshness dashboard for the CS team — investigation reveals the batch cadence is an architectural ceiling, not a configuration option, and a two-month incremental rewrite produces no new features. A fintech uses a Flink streaming pipeline for real-time fraud scoring with a 10-second watermark copied from a tutorial; the mobile SDK's P99 event arrival latency is 4.2 hours; 29% of transactions are scored with incomplete velocity context for two months before anyone measures the coverage rate. The processing model you choose sets the freshness floor for every downstream data product and determines whether late events are handled by a documented watermark policy or silently discarded without a count, an error, or an alert.
2026-07-01 · ~20 min read
Microservices are adopted in the founding sprint because a previous employer used them or a conference talk praised their independence — and the service boundaries are drawn without asking which business operations cross them. A field service management team splits Orders, Inventory, Scheduling, and Billing into separate services; the "complete job" workflow requires all four and produces a 480-line saga with compensation logic that two engineers spend three weeks building. A marketplace startup starts with four services and grows to eleven; two engineers spend 30% of their time on deployment coordination and docker-compose maintenance before merging seven services back into a modular monolith. The deployment coupling surface and the distributed transaction surface area are determined when the first boundary is drawn — before the first saga is needed.
2026-06-27 · ~20 min read
Event sourcing is adopted in the founding sprint when an audit trail requirement appears — and never documented as a deliberate architecture choice. A compliance SaaS team chose it for regulatory reasons; three years later their upcaster file is 740 lines, touched most frequently by bugs and avoided most frequently during refactors. A marketplace startup adopted it after a blog post; six months later a new analytics projection requires reading 2.3 million events from the beginning and takes 14 hours to rebuild. The projection rebuild SLA, the event schema migration policy, and the snapshot checkpoint frequency are structural decisions set when the first event row is appended.
2026-06-27 · ~20 min read
State management is decided in the first sprint when a component needs data from a sibling — and never documented. A team stores every API response in Redux; after fourteen months they have 31 request-lifecycle reducers and spend a week building a custom staleness middleware that reimplements TanStack Query's stale-while-revalidate loop. A startup uses useState for everything; five Contexts and a re-render cascade later, two engineers spend two days tracing why the notification badge re-renders on every cart update. The client state vs server state classification, the memoization policy, and the optimistic update strategy are established before the first reducer is written — and invisible after.
2026-06-27 · ~19 min read
GraphQL versus REST is decided in the founding sprint when the first API endpoint is needed and never documented. A new GraphQL resolver fetches customer records once per order in a list query — 340 orders produce 681 database queries and a 47-second admin dashboard timeout. A REST team adds a combo endpoint for each new mobile view; after eighteen months the API has fourteen view-specific aggregation endpoints that must be cloned for every new client layout. The DataLoader convention, the caching model, the schema governance process, and the authorization boundary are established when the paradigm is chosen — and surface as incidents when they are not.
2026-06-27 · ~18 min read
Cron versus job queue versus event-driven trigger is decided in the first sprint where an engineer asks "how do I run this on a schedule?" and never documented. A second server provisioned from the same machine image carries the same crontab entry — and a monthly billing job fires simultaneously on both servers, charging 318 customers twice. A nightly aggregation job is killed by the OS at 3am; the cron daemon records nothing; the team discovers the gap 72 hours later when a customer calls about missing data. The distributed lock policy and missed job detection mechanism are structural decisions, not implementation details.
2026-06-27 · ~20 min read
React versus Vue versus Svelte is decided in the first sprint and never documented. The SSR requirement determines whether a client-side-rendering-only framework selection requires a full routing and data-fetching rewrite when SEO becomes necessary. State management coupling determines how expensive major version upgrades become. A CRA-to-Next.js migration took fourteen weeks; a Vue 2 EOL migration took eleven months. Neither was anticipated.
2026-06-27 · ~20 min read
S3 versus GCS versus Azure Blob is decided in the first sprint where an engineer asks "how do I store user-uploaded files?" and never documented. Presigned URL expiry determines whether download links are bearer tokens that can be forwarded outside the product. CDN integration determines whether deleting a file makes it inaccessible or leaves it cacheable for hours. Virus scanning integration point — pre-upload versus post-upload — determines whether the scanning layer catches novel exploits or only known signatures.
2026-06-26 · ~18 min read
In-app versus email versus push versus SMS versus webhook is decided in the first AI session on notifications and never documented. APNs device tokens expire when users reinstall the app; 410 Gone must be processed to remove stale tokens or push budget delivers to addresses that receive nothing. Email deliverability depends on SPF, DKIM, and DMARC decisions made before the first send. The channel decision determines what your users see, when, and whether the message arrives or silently fails.
2026-06-26 · ~20 min read
WebSocket versus SSE versus long-polling is chosen in the first sprint AI session and never documented. Sticky sessions are enabled at the load balancer because WebSockets require them, disabling the auto-scaling policy. The client reconnection policy is not implemented because the developer testing with thirty concurrent connections never observed a thundering herd. The transport decision determines your horizontal scaling ceiling and what your users experience the first time a rolling deploy drops connections at scale.
2026-06-26 · ~19 min read
RPO and RTO targets are decided in the initial architecture session, written on a whiteboard, and never documented. A 340 GB database restore from S3 takes 7 hours when the assumed RTO was 2 hours. A cross-region replication lag of 47 minutes invalidates a 5-minute RPO that was set as a target without being measured under production load. The disaster recovery decision determines what data you lose and how long you are down when the incident you did not expect happens.
2026-06-26 · ~19 min read
Response shape, enum extensibility, nested object embedding, and field deprecation policy are decided in the first API design session and never documented. When the product needs to add a new subscription state six months later, the iOS mobile app crashes because the enum was treated as exhaustive. When billing addresses need to become their own resource, 61 consumer integrations each reference the embedded object directly, and the migration cost is proportional to every consumer's update cycle and app store review timeline.
2026-06-26 · ~19 min read
Log format, log level policy, correlation IDs, sampling rate, and retention tiers are decided by default — console.log strings in development that ship to production, DEBUG enabled because nobody wrote a policy, no traceId contract because OpenTelemetry wasn't in the initial stack — and never documented. Three years later, the log format is the primary reason a production incident takes four hours to diagnose instead of fifteen minutes, and the absence of a log level policy is why $108 a month in unexpected CloudWatch charges appeared in the cost review.
2026-06-25 · ~20 min read
Authorization is treated as an implementation detail of the authentication system — a few role checks added alongside the login flow, a roles table with Admin and Viewer, a middleware that checks the field. The permission model, the tenant isolation boundary, and the delegation policy are chosen during the first access control ticket and never documented. Two years later, the tenant isolation model set in that first ticket is the root cause of a cross-tenant data exposure requiring a permission schema migration across three million rows, or the RBAC structure adopted for five internal roles cannot express the custom combinations that enterprise customers require without a second authorization system bolted on with different enforcement semantics.
2026-06-25 · ~19 min read
Monorepo versus polyrepo looks like a file organization preference until a cross-service type rename requires eleven coordinated pull requests across eleven repositories to land in the correct sequence because any intermediate state breaks the build, or a monorepo CI pipeline grows from six minutes to forty-seven because every commit rebuilds all twenty-three packages and nobody set up a task graph — because the cross-service change atomicity model, the CI pipeline blast radius, and the dependency version alignment policy were never documented when the first service was created.
2026-06-25 · ~21 min read
Email infrastructure setup looks like a 30-minute Mailgun or SendGrid API key until a shared IP pool reputation event drops your transactional open rate 11 points, or a provider migration re-sends to 1,847 addresses who had previously unsubscribed and pushes your Google domain reputation above the enforcement threshold — because the IP allocation model, the suppression list portability architecture, and the bounce and complaint webhook model were never documented when the first email arrived in the inbox.
2026-06-25 · ~22 min read
Data warehouse selection looks like a one-afternoon BigQuery setup until the BI tool's dashboard refresh schedule produces a $1,200 scan bill, or a Redshift cluster running 20 concurrent analyst queries during a nightly ETL load degrades every query to 30–90 seconds — because the query compute model, the workload isolation policy, and the data freshness architecture were never documented when the first SELECT returned results.
2026-06-25 · ~20 min read
Payment processor selection looks like a one-session Stripe setup until a European user's card is declined because 3DS isn't configured, a dispute rate spike from unauthenticated transactions threatens card acceptance rights, or a SaaS product discovers it owes VAT in 34 jurisdictions — because the SCA liability model, the cross-border fee structure, and the tax compliance scope were never documented when the first test charge returned status: succeeded.
2026-06-24 · ~21 min read
Database vendor selection looks like a technical default until a MySQL 5.7 application discovers window functions were unavailable for three years, or a PlanetScale application discovers that foreign key constraints were never enforced — because the SQL feature ceiling, the referential integrity model, and the scaling architecture were never documented when the first query returned results.
2026-06-22 · ~21 min read
Search engine selection looks like a configuration detail until a relevance improvement triggers a 4.5-hour index rebuild that freezes document creation, or a managed SaaS bill grows from $420 to $1,240 a month — because the relevance model ceiling, the schema evolution constraint, and the per-operation cost at scale were never modeled when the session closed.
2026-06-22 · ~21 min read
CDN selection looks like a routing detail until a pricing bug is patched in four minutes of deploy time but cached API responses keep serving the wrong entitlements for an hour — because the team cached API responses without documenting the invalidation procedure, the propagation latency, or the maximum accepted staleness for pricing endpoints.
2026-06-22 · ~22 min read
API gateway selection looks like an infrastructure detail until a security researcher finds four unprotected admin routes — because the authentication model was per-route, the default was open, and the policy that said "all state-modifying routes must have the JWT authorizer" had never been written down.
2026-06-21 · ~23 min read
Observability platform selection looks like a tooling detail until a $47k Datadog invoice arrives in month 38 and the team discovers APM was enabled on every Lambda function, custom metrics grew to 12,000 unique time series past the 10k default limit, and per-host charges had been scaling with every microservice extraction for three years. The platform you chose — Datadog, Grafana OSS stack, New Relic, or Honeycomb — determines your cardinality ceiling, your trace sampling model, your cost scaling formula, and whether the engineer paged at 2am can find the root cause before the SLA expires.
2026-06-21 · ~22 min read
Message broker selection looks like an infrastructure detail until a payment reconciliation service needs 30 days of event replay and you discover the topic retention policy was set to 24 hours at initial setup. The broker you chose — Kafka, RabbitMQ, SQS, or Redis Streams — determines your delivery guarantee, your replay window, your consumer group isolation model, and whether adding the sixth downstream consumer is a configuration change or a six-week engineering project.
2026-06-21 · ~22 min read
Feature flag service selection looks like a tooling choice until a bad rollout burns through Black Friday traffic and you discover the homegrown flag system uses per-request random bucketing — so 10% of users is actually 10% of requests, and the same user sees both versions on alternate page loads. The flag service you chose — LaunchDarkly, Unleash, Flipt, or a homegrown Redis-backed system — determines your rollback propagation latency, your user-consistency guarantee, your A/B testing trustworthiness, and how expensive a future migration will be.
2026-06-21 · ~20 min read
Secrets management looks like a security detail until a leaked credential triggers an incident response at 3am and your team discovers the rotation procedure in the runbook assumes the secrets store you replaced six months ago. The store you chose — environment variables, AWS Secrets Manager, HashiCorp Vault, or a managed platform — determines your rotation automation model, your audit trail granularity, and how many minutes elapse between detecting a compromised credential and denying access to every consumer of it.
2026-06-21 · ~20 min read
Database connection pooling looks like a performance tuning detail until your application hits FATAL: sorry, too many clients already at 2am and your on-call engineer doesn’t know whether the pooler is PgBouncer in transaction mode or the ORM’s built-in pool. The pooler you chose — PgBouncer, RDS Proxy, or application-level pooling — determines your connection exhaustion ceiling, your server RAM commitment per idle connection, and how fast your application recovers when the database restarts.
2026-06-21 · ~22 min read
Container orchestration looks like an infrastructure choice until a traffic spike reveals that your autoscaling model adds four minutes of latency to scale-out and your on-call engineer doesn’t know why Kubernetes was chosen over ECS in the first place. The orchestration platform you chose in year one sets the operational complexity floor — the minimum ongoing maintenance work your team carries — and determines the autoscaling behavior your users experience when demand changes suddenly.
2026-06-20 · ~20 min read
CI/CD pipelines look like plumbing until a production incident requires an emergency rollback and the pipeline that ships code in 12 minutes takes 47 minutes to roll back because nobody designed the rollback path. The deployment pipeline decisions made in year one — build tool, artifact model, deployment strategy, rollback mechanism — determine whether your team can respond to production incidents with a single command or with a multi-step manual procedure under pressure.
2026-06-20 · ~20 min read
Infrastructure-as-code looks like a solved problem until a manually-applied security group fix from a late-night incident sits in production for eight months, invisible to your Terraform state, and a junior engineer's next plan would delete it. The IaC tool and module structure you chose — Terraform, Pulumi, CloudFormation, or direct CLI — determines whether drift is detectable, whether your compliance audit has a change trail, and whether the engineer running apply has enough context to understand what they're about to destroy.
2026-06-20 · ~20 min read
API versioning looks like a URL prefix until a mobile client running version 1.2 refuses to update and your paying enterprise customer's integration breaks when you remove a deprecated field. The versioning strategy embedded in your API — URL path versioning, header-based versioning, date-based versioning, or no versioning at all — determines how many simultaneous API versions you can maintain, what migration friction your customers experience, and how long deprecated endpoints consume your engineering capacity.
2026-06-20 · ~20 min read
Background jobs look like a simple work-queue pattern until a critical billing job silently exhausts its retries, disappears into a dead-letter queue no one monitors, and the revenue leak runs for three days before anyone notices the accounts receivable balance is wrong. The retry policy, backoff strategy, dead-letter handling procedure, deduplication approach, and concurrency model embedded in the job infrastructure determine what happens to failed work and whether the team can diagnose the failure without reading application logs.
2026-06-20 · ~20 min read
Multi-region deployment looks like infrastructure configuration — pick a cloud provider, select additional regions, enable database replication, deploy identical application stacks. This framing conceals the failover model that determines RPO and RTO during regional outages; the data residency policy that determines which data can leave which jurisdiction; the cross-region consistency model that determines the tradeoff between write latency and data loss risk; and the traffic routing approach that determines the actual latency floor users experience.
2026-06-19 · ~20 min read
Search feels like a database query — a WHERE clause, a LIKE operator, maybe a full-text index. This framing obscures the architectural decisions embedded in the implementation: the indexing strategy that determines the performance ceiling under corpus growth; the relevance model that determines whether the first result is actually the most relevant result; the synchronization approach that determines how stale the index can be; the schema evolution policy that determines whether adding a new field requires downtime.
2026-06-19 · ~20 min read
Message queues look like infrastructure configuration — choose a broker, publish a message, consume it in a worker. This framing hides the architectural decisions embedded in the choice: the delivery semantic that determines what your consumer must do when a message arrives twice; the dead-letter strategy that determines whether a malformed message is isolated for inspection or loops forever; the schema evolution policy that determines whether producer and consumer can be deployed independently.
2026-06-19 · ~20 min read
Rate limiting is added reactively — after the first abuse incident or the first traffic spike that takes the service down. The approach chosen at that moment, under pressure, determines whether legitimate traffic degrades gracefully when spikes arrive, whether the abuse surface can be narrowed without breaking existing integrations, and whether the on-call engineer can change a limit value at 2am without triggering a deployment.
2026-06-19 · ~20 min read
Authentication libraries are chosen from quickstart tutorials and rarely revisited as decisions. Two years later, the session management approach you chose determines what an SSO integration costs, whether a compromised token can be revoked before it expires, and whether your SOC 2 audit finds undocumented session data retention policies.
2026-06-19 · ~19 min read
Database migration tooling is chosen when the project starts and rarely revisited. Two years later, the migration approach you chose determines whether adding a column to a 200-million-row table takes 3 seconds or 40 minutes, whether a failed deploy can be rolled back, and whether the team can apply schema changes during a rolling deploy without breaking the running version.
2026-06-18 · ~18 min read
Observability platform adoption is treated as a DevOps improvement, not an architecture decision. The metrics backend is chosen when the first dashboard is needed. The distributed tracing library is chosen when the first latency mystery appears. Two years later, the cardinality limits of the metrics backend determine whether you can answer "which customer is experiencing the slowest API response?" during an incident. The tracing format you chose determines whether you can switch backends without re-instrumenting every service. Three backend families: self-hosted Prometheus (active series in memory, ~30M series per 32 GB instance — high-cardinality labels like customer_id produce memory exhaustion rather than cost spikes, cardinality explosions are the most common Prometheus failure mode); managed SaaS (Datadog, Grafana Cloud — no operational burden, cost scales with active series; a customer_id label on a metric with 50,000 active customers multiplies the metric's cost by 50,000); high-cardinality backends (Honeycomb, ClickHouse — column-store, per-event pricing, filter-by-any-dimension without cardinality limits). The prohibited label set: customer_id, user_id, request_id, session_id, tenant_id are labels that produce cardinality explosions on high-traffic metrics — the most consequential undocumented constraint of any Prometheus deployment. The distributed tracing decision has three independently load-bearing components: (1) instrumentation format — OpenTelemetry SDK (vendor-neutral, backend migration requires only OTEL Collector exporter reconfiguration) vs Jaeger native or Zipkin B3 (backend-specific, migration requires re-instrumenting every service); (2) sampling strategy — head-based probabilistic (independent draw per request, 1% sample misses a 0.1% error rate most of the time during a 5-minute incident window) vs tail-based (sampling decision made after the full trace is assembled, every error trace retained, memory cost for buffering); (3) trace context propagation — W3C traceparent headers for HTTP, Kafka message header serialization for async boundaries; missing propagation at async boundaries produces orphaned root spans disconnected from the originating request trace. The observability contract: explicit statements of which incident questions the platform can answer ("which service is responsible for elevated latency?") and which it cannot ("which specific customers are affected? — customer_id label not permitted due to cardinality constraints; manual database query required"). Writing the observability strategy ADR: the observability contract with explicit capability and incapability statements; the metrics backend decision with cardinality limit and prohibited labels; the distributed tracing decision with format, sampling strategy, and propagation policy; the log aggregation integration with cross-pillar correlation policy; the revisitation conditions naming cost thresholds, cardinality thresholds, compliance retention requirements. Finding observability decisions in AI chat: four session types — initial instrumentation sessions (backend selection, tracing library choice, sampling strategy); incident response sessions (capability gaps discovered under pressure: "how do I filter Prometheus metrics by customer ID?"); cardinality incident sessions (the event that surfaces the cardinality limit); platform migration sessions (format lock-in consequences discovered at migration time).
2026-06-18 · ~18 min read
Repository structure is chosen once — at project inception — and rarely revisited. Four years later, the build graph determines whether a one-line change to a shared utility takes 4 minutes or 40 minutes in CI. The CODEOWNERS model determines whether a cross-team PR review takes one day or one week. The dependency sharing model determines whether a breaking change in a shared package can be deployed atomically or requires coordinating five separate service teams. Four structural patterns with distinct architectural consequences: flat monorepo (single CI pipeline, no package boundaries, CI time grows linearly with codebase size, no ownership model — the starting point most teams outgrow without documenting the transition); modular monorepo with workspace tooling (Turborepo, Nx, Bazel — affected-package computation drives CI efficiency, accuracy depends entirely on the correctness of dependency declarations in package.json, phantom dependencies produce CI green results for artifacts built against outdated transitive dependencies); polyrepo (one repo per service, independent release cadence, shared code versioned via an internal registry, version skew accumulates without a maximum skew policy, cross-service refactoring requires coordinated multi-repo PRs); hybrid structure (monorepo per domain, polyrepo across domains — inherits both affected computation complexity and cross-repository coordination cost). The affected computation accuracy invariant: a phantom dependency — a package that a module imports directly without declaring it in package.json — is resolved at runtime because the transitive dependency is installed in the workspace, but the build tool has no record of the relationship; when the phantom dependency's package changes, the importing package's build cache is not invalidated, CI shows green, and the deployed artifact is built against the outdated version; phantom dependency prevention (pnpm strict hoisting mode, dependency-cruiser pre-merge lint) converts the phantom dependency from a silent production failure mode to a development-time error. The CODEOWNERS ownership model: in a monorepo, ownership is a CODEOWNERS file that must be maintained as the team structure changes; stale CODEOWNERS entries gate PRs on reviewers who left the team, producing review bottlenecks; ungated packages (no CODEOWNERS entry) must be documented choices, not accidental gaps. The dependency sharing model: workspace:* references mean all packages always use the current version of all internal dependencies — a breaking change to a shared package must update all consumers in the same PR; in a polyrepo, consumers upgrade on independent schedules, enabling staged rollouts but accumulating version skew without a documented maximum skew policy. Writing the repository structure ADR: structure decision with alternatives evaluated and rejection reasons; CI build model and caching policy naming how affected computation works and what phantom dependency prevention mechanism is in place; code ownership model with CODEOWNERS maintenance policy and cross-team contribution process; dependency sharing model with breaking change deployment policy; revisitation conditions naming checkable thresholds for CI time, review turnaround, and team count. Finding repository structure decisions in AI chat: four session types — initial setup sessions (structure choice, build tool selection); CI speed sessions (affected computation configuration, remote cache backend); phantom dependency sessions (accuracy policy discovery through incident); cross-team contribution sessions (CODEOWNERS design and review process).
2026-06-18 · ~18 min read
Caching adoption is treated as a performance optimization, not an architecture decision. The invalidation mechanism is chosen quickly when the first slow endpoint appears and rarely documented. Two years later, the TTL policy determines the staleness window for every cached resource, the cache stampede behavior determines what users experience when the cache is flushed, and the failure mode determines whether the application degrades gracefully or breaks entirely when the cache node restarts. Five caching patterns: cache-aside (lazy loading — application checks cache first, on miss reads from source and populates cache; write path decoupled from cache, staleness bugs emerge when write paths lack invalidation calls); write-through (updates cache synchronously on every write, always current, write latency cost, cold start problem after flush); write-behind / write-back (acknowledges writes to cache immediately, DB write async — minimizes write latency, durability risk if cache node fails before DB write completes); read-through (cache layer handles cache miss automatically by calling a configured data loader, consistent population logic); CDN/edge caching (HTTP response caching via Cache-Control headers, tag-based surrogate key invalidation for purge-on-update). The invalidation mechanism decision: TTL-based invalidation is the default but is simultaneously the staleness window that determines how long a stale billing address, expired permission, or outdated price persists; event-driven invalidation supplements TTL by explicitly deleting keys on write, but every new write path that lacks the corresponding invalidation call is a staleness bug; tag-based invalidation groups cached responses under named tags and purges by tag, requiring a maintained tag design that determines the granularity of invalidation. The cache stampede problem: when a high-traffic cached value expires, multiple concurrent requests simultaneously execute the cache miss handler, saturating the database; single-flight / mutex pattern deduplicates concurrent miss executions; probabilistic early expiration spreads regeneration load across the TTL window before expiry. The consistency guarantee: cache-aside with TTL gives eventual consistency at the TTL boundary — account tier, feature entitlements, and prices silently serve stale data for the entire TTL window when the write path lacks invalidation. Writing the caching ADR: caching mechanism and cache provider decision with alternatives evaluated; TTL policy and invalidation mechanism by data class; consistency guarantee naming the staleness window for each data class; cache key design and namespace policy; failure behavior and cache stampede mitigation strategy. Finding caching decisions in AI chat: four session types — initial adoption sessions (mechanism choice, provider selection, TTL reasoning); staleness incident sessions (invalidation gap discovery); stampede sessions (thundering herd discovery and mitigation); and cache failure sessions (failure behavior and fallback policy).
2026-06-18 · ~18 min read
Feature flag adoption is treated as a developer experience enhancement, not an architecture decision. The evaluation mechanism is chosen during the first dark launch emergency and rarely documented. Two years later, the flag evaluation model determines whether gradual rollouts are safe under concurrent deployment versions, the flag store design determines whether A/B testing produces reliable impression data, and the absence of a lifecycle policy produces a codebase with 200 flags where 30 are actively used and none can be safely deleted. Five evaluation pattern categories: boolean environment variable flags (requires restart/redeploy to change — the zero-infrastructure entry point that many teams outgrow without documenting the transition); database-backed server-side evaluation (DB as flag store, synchronous request evaluation, caching policy determines propagation delay); SDK with local evaluation (LaunchDarkly, Unleash, Flipt — in-memory flag config synced via streaming, sub-millisecond evaluation latency, bootstrap state behavior is the undocumented constraint on Lambda cold starts); remote evaluation via vendor API call (flag vendor evaluates per-request, each evaluation adds a network round-trip on the critical path, vendor outage blocks flag evaluation); client-side evaluation (flag config in browser, PII in evaluation context exposed in developer tools, personalization without server round-trip). The gradual rollout safety constraint: per-request random selection vs user-ID-consistent hashing — a checkout flow that writes state in one format under treatment and reads in another format under control requires consistent hashing to be safe; with per-request random assignment, a user has a 10% chance of encountering a format mismatch on every request after the first. The rolling deployment window interaction: two application versions running simultaneously during a Kubernetes rolling update may evaluate the same flag differently if the flag was added after the old version was built. The A/B testing impression recording gap: SDK-based local evaluation tools do not automatically generate impression events — the application must explicitly call the SDK impression API; teams that adopt a flag tool for dark launches and repurpose it for experiments discover the missing impression data at first analysis. Assignment persistence across sessions: user-ID-consistent hashing provides cross-session consistency for authenticated users; cookie-based session UUIDs break when users clear cookies or switch devices. Writing the feature flag ADR: evaluation mechanism with alternatives evaluated and bootstrap behavior documented; user targeting and assignment model with consistent hashing policy and PII restrictions; A/B testing and impression recording policy naming how impressions are generated and where they land; flag lifecycle policy with type definitions, expected lifetimes, ownership assignment, and removal sequence; critical path policy naming which flag types are permitted in the synchronous user-request path. Finding decisions in AI chat: initial adoption sessions (mechanism choice), gradual rollout incident sessions (assignment model constraints), A/B testing sessions (impression recording gap discovery), and flag removal sessions (lifecycle policy constructed reactively).
2026-06-18 · ~17 min read
Log aggregation tool selection is treated as infrastructure configuration, not architecture. The tool is chosen once during initial cluster setup and rarely revisited. Two years later, it determines which incident response questions the on-call engineer can answer without a multi-minute scan: field-indexed tools (Elasticsearch, Datadog Logs) allow filter-by-any-structured-field regardless of cardinality; label-indexed tools (Grafana Loki, CloudWatch Logs) index only a fixed set of low-cardinality deployment labels and require full-text content scans for application-level fields like customerId or tenantId. The 3am constraint is discovered in production, not at evaluation time. Five pattern categories: print-to-stdout with external aggregation (Kubernetes-native default; the aggregation tool determines what downstream); field-indexed self-hosted aggregation (EFK/ELK — rich query capability, high operational overhead); label-indexed self-hosted aggregation (Loki — low ingestion cost, label schema determines the query surface); SaaS aggregation (Datadog, New Relic — low operational cost, super-linear per-GB cost growth at traffic scale); no centralized aggregation as a deliberate documented choice. The retention cost constraint: log volume grows with traffic; self-hosted Elasticsearch at 100 GB/day with 30-day hot retention costs ~$390/month in NVMe storage alone; Loki at the same volume costs ~$69/month in S3; SaaS tools have a cost crossover point that most teams hit three years after adoption. Structured logging as the aggregation tool's access key: a field-indexed tool paired with unstructured log output is equivalent to full-text search; field naming divergence across services (user_id vs userId vs customerId) produces incident query errors without a documented convention. The log level contract as the missing policy: WARN misuse (logging routine high-frequency events at WARN to make services visible in production filters) produces alert signal that cannot be trusted; the on-call engineer cannot distinguish a genuine WARN spike from a busy cache layer without a written contract. Writing the logging ADR: aggregation mechanism with alternatives evaluated; structured logging policy with required fields and field naming convention; log level contract with per-level definitions and on-call trigger; retention policy with cost projection at current volume; revisitation conditions naming cost and query latency thresholds. Finding logging decisions in AI chat: four session types — aggregation tool selection (mechanism choice), structured logging adoption (field decisions), incident debugging (revealed query constraints), and cost investigation (retention policy decision under budget pressure).
2026-06-18 · ~16 min read
Service mesh adoption is rarely treated as an architecture decision — it is treated as a platform upgrade. Three years later, the mesh is load-bearing for observability, security posture, and Kubernetes upgrade compatibility, and no one can reconstruct what is intentional configuration and what is accumulated default. Four pattern categories to document: sidecar-proxy meshes (Istio, Linkerd — every pod carries an Envoy sidecar that intercepts traffic via iptables); sidecar-less eBPF-based meshes (Cilium, Istio ambient mode — node-level eBPF programs eliminate per-pod sidecar overhead but require kernel ≥5.10); application-layer meshes (mTLS and circuit breaking embedded in each service's SDK); and the "no mesh" posture as a deliberate documented choice. The observability constraint: sidecar proxies emit distributed tracing spans at the network layer but cannot propagate trace context through the application — trace header propagation remains an application responsibility that the mesh does not eliminate; teams that believe the mesh "handles tracing" discover disconnected root spans rather than end-to-end traces. The zero-trust constraint: permissive mTLS (Istio's default) accepts both mTLS and plaintext — authentication theater for any SOC 2 requirement that requires enforcement; strict mTLS migration requires auditing the sidecar exclusion list before flipping the mode or it breaks batch jobs and CronJobs that were intentionally excluded. The sidecar exclusion policy as the most commonly missing section: every production mesh has workloads outside the mTLS perimeter (CronJob pods whose Envoy proxy prevents termination; DaemonSets with host networking); undocumented exclusions are indistinguishable from accidental gaps, producing audit findings regardless of actual coverage. The Kubernetes version coupling: Istio is explicitly versioned against Kubernetes minor versions; an undocumented version coupling converts a cluster upgrade into an unplanned mesh version assessment. Writing the service mesh ADR: mesh selection with alternatives evaluated and rejected; sidecar exclusion policy naming each excluded workload type with the technical reason and security implication; mTLS mode and authorization policy model with the migration plan from permissive to strict; Kubernetes version compatibility with the upgrade trigger. Finding service mesh decisions in AI chat: four session types — initial evaluation (mechanism selection), CronJob incident sessions (sidecar exclusion policy), mTLS migration sessions (authorization policy model), and upgrade planning sessions (version coupling).
2026-06-15 · ~15 min read
Every codebase has a dependency injection pattern. Most teams didn't choose it — it came with the framework, or emerged from whoever wrote the first service class. Once 200 classes are structured around it, the DI mechanism is no longer a preference. It determines which things the test suite can substitute (the testability consequence: constructor injection makes every dependency an explicit, mockable parameter; IoC containers make dependencies implicit and resolution rules load-bearing); how fast the application starts (the startup cost consequence: eager container initialization that is invisible in long-lived servers becomes a Lambda cold-start penalty that can exceed 6 seconds); and how much context a new engineer needs before they can make a confident change (the cognitive overhead consequence: a constructor signature tells you exactly what a class depends on; a container annotation tells you the type and leaves the resolution rules to be discovered). The scope and lifetime decision as the most common source of concurrency bugs: singleton services holding per-request state (current user ID, transaction context) produce intermittent cross-request data leaks under concurrent load — the bug caused by a DI lifetime mismatch, not application logic. The injectable boundary decision: making everything injectable produces a container that knows about hundreds of classes that are never substituted, while drawing the boundary tightly around layer-crossing interfaces (database repositories, HTTP clients, message producers) produces a test suite with a clear substitution contract. How the DI mechanism decision becomes irrecoverable: embedded in every class constructor or annotation in the codebase, requiring a full codebase migration to change — which means the initial choice needs a revisitation condition before the accumulation makes it permanent. Writing the DI ADR: mechanism decision with specific alternatives evaluated and rejected; lifetime policy naming the rule for each service category; injectable boundary naming what crosses a layer and what doesn't; composition root policy naming where registrations live. Finding DI decisions in AI chat: framework evaluation sessions (the mechanism choice); testability problem sessions (the first time the mechanism's test consequences become visible); startup time sessions (the Lambda cold-start discovery); and onboarding sessions where a new engineer's questions reveal the cognitive overhead the mechanism creates.
2026-06-15 · ~16 min read
Every application has an error handling strategy. Most teams chose theirs by not choosing it — "we'll add proper error handling later" became the policy. The implicit default (what happens when you don't handle an error: framework returns 500, exceptions are swallowed, empty arrays are returned on database failure) is the de facto strategy. Four decisions that accumulate into policy: the error surface decision (what users see vs. what engineers see from the same error — and why the boundary is also a security boundary that prevents stack traces from leaking internal paths); the failure mode decision (which operations fail-fast vs. degrade gracefully vs. fail silently — and why "silently" is almost always wrong); the retry and idempotency policy (which operations are safe to retry, what backoff strategy, how idempotency keys prevent duplicate payments and duplicate emails); and the observability contract (what gets logged at what level, what triggers an alert, what is explicitly ignored — and why alert fatigue is a consequence of the undocumented observability contract, not a tooling problem). The error taxonomy problem: without a named taxonomy, teams invent local HTTP status code conventions that conflict — one endpoint returns 400, another 422, a third 200 with an error field — producing inconsistency that frontend engineers must special-case per endpoint. The user-facing error as a product decision: what users see when something fails determines whether they retry, fix their input, or abandon; this copy is almost always written by engineers at implementation time rather than by product at design time. Writing the error handling strategy ADR: error taxonomy with consistent handling per category, failure mode policy with the list of ancillary features exempt from fail-fast, retry and idempotency policy with backoff parameters and retry budget, observability contract with log level trigger conditions and alert thresholds. Finding error handling decisions in AI chat: four session shapes — design question sessions where individual choices are first made, production incident sessions where error handling costs become visible, retry and idempotency sessions where the duplicate-action problem surfaces, and observability frustration sessions where alert fatigue is explicitly named.
2026-06-15 · ~15 min read
Every team has a testing pyramid. Most teams didn't choose it — it emerged from whoever set up the CI pipeline first, from the framework the team happened to adopt, from the testing philosophy of the engineer who wrote the first test. The ratio of unit to integration to end-to-end tests then constrains refactoring safety, deployment speed, CI cost, and what the team can honestly promise about production behavior. Three test strategy archetypes worth documenting: the unit-test-dominant pyramid (fast CI, strong protection for internal logic, but structurally incapable of catching mock drift bugs — the real implementation changed, the mock wasn't updated, the tests went green, the production deployment failed); the integration-test-dominant strategy (hits a real database or real service infrastructure, catches the class of bugs that mock-based tests cannot — query plan regressions, constraint violations, transaction isolation failures — but requires container provisioning and longer CI runs); and the E2E-primary strategy (highest-fidelity deployment signal, slowest CI, highest flakiness tax). The mock boundary decision as the most consequential undocumented testing decision: where in the stack do tests switch from real implementations to fakes? The mock boundary determines what the test suite can and cannot tell you about production behavior — and it cascades into CI infrastructure, test data management, and what refactoring is safe without updating tests. The test strategy as a hiring constraint: engineers arriving from TDD backgrounds extend mock-heavy suites further; engineers from integration-test backgrounds add real-database tests; engineers from E2E backgrounds add browser tests; without a documented strategy, three overlapping strategies accumulate in one suite rather than one coherent approach. Writing the test strategy ADR: pyramid ratio decision with reasoning, mock boundary decision naming exactly where real implementations give way to fakes, deployment confidence threshold, refactoring safety guarantee, and revisitation conditions. Finding test strategy decisions in AI chat: four session shapes — setup sessions in project week one where framework choices locked in the strategy before it was consciously evaluated; mock boundary debates where the team argued about database mocking; CI speed sessions where the strategy's cost first became explicit; and production escape sessions where the test suite's coverage gap was named directly.
2026-06-15 · ~14 min read
A library that was rejected still leaves a trace — it isn't in package.json. A pattern that was rejected leaves nothing. The codebase that doesn't use decorators for dependency injection looks identical whether the team spent two weeks evaluating and rejecting them or whether decorators were never considered. Three pattern rejection categories worth documenting: the evaluated-and-explicitly-rejected pattern (the team looked at decorator-based DI, ORMs, or event-driven state management and chose not to adopt them, with specific reasoning that disappears when the evaluating engineers leave); the deliberately excluded pattern as a categorical standards decision ("no ORM," "no global state," "no magic" — a documented constraint that can be cited in code review, versus an undocumented preference that erodes as each new engineer adds one exception that seems obviously justified); and the adopted-and-then-de-standardized pattern (partial migration where old and new patterns coexist without documentation explaining the direction of travel — looks like inconsistency rather than an in-progress architectural shift). The consistency compulsion: new engineers import patterns from prior codebases, and absent a record, the current codebase's pattern absence looks like a gap to close rather than a decision to respect. The technical debt misread: a new technical leader who sees raw SQL everywhere in 2026 reads it as a legacy choice if there is no record, and reads it as a deliberate stance if there is. The anti-pattern ADR: categorical rejections produce a document type distinct from the standard ADR — not documenting what was chosen but what is not acceptable and what the team uses instead. The de-standardization migration record: the most urgent pattern documentation because the mixed state actively misleads new engineers about which pattern is acceptable for new code. Writing the rejected pattern ADR: Context names the pattern and why it was under consideration, Alternatives Considered names the evaluated pattern with specific concerns, Decision names what was chosen instead with the constraint that drove it, Revisitation Condition names concrete triggers for re-evaluation. Finding pattern rejections in AI chat: evaluation sessions that conclude negatively are harder to anchor than dependency rejections because the pattern has no canonical search term — the quarterly review pass is the most reliable extraction mechanism.
2026-06-14 · ~16 min read
Every SaaS product has a multi-tenancy isolation model. Most chose it implicitly: a tenant_id column added because it was the simplest way to separate customer data when there was only one paying customer and the question of tenant isolation felt theoretical. Three categories of multi-tenancy decisions worth documenting: the isolation model (row-level security vs. schema-per-tenant vs. database-per-tenant — the mechanism determines the blast radius of a cross-tenant leak, the per-tenant backup and restore capability, and what the enterprise sales team can promise a prospect); the data residency decision (which regions tenant data can live in — single-region deployment is a data residency policy with GDPR Article 44 consequences for European customers; the isolation model constrains whether per-tenant regional routing is achievable without a migration); and the compliance certification scope (what certifications the isolation model supports — SOC 2 with RLS, HIPAA BAA, FedRAMP — and what the dedicated-instance path looks like for certifications the current model cannot satisfy). The "default" pathology: the team that added tenant_id without documenting the trigger for reconsidering, added row-level security under audit pressure without evaluating database-per-tenant, then lost an enterprise deal when the security questionnaire asked "do you offer dedicated database instances?" Finding multi-tenancy decisions in AI chat: three session types — early database design ("how do I separate customer data in Postgres?"), security review sessions ("is there any risk of cross-tenant data leaks?"), and enterprise deal sessions ("the customer wants their own database — how hard would that migration be?") — the enterprise deal sessions are the highest-value extraction targets because they contain the first honest gap analysis between the current isolation model and enterprise requirements.
2026-06-14 · ~14 min read
Most engineering teams discover their data retention policy during a GDPR audit, a legal hold request, or the quarter when infrastructure costs spike 40% and someone asks why the user_events table is 800 gigabytes. Three categories of retention decisions worth documenting: the deletion policy (what gets deleted, when, by what mechanism — hard delete, soft delete, or anonymization — and who triggers it; the mechanism choice determines whether a GDPR right-to-erasure request is a 20-minute operation or a multi-day archaeological project); the backup and archival retention period (how long backups live and whether archives are subject to the same deletion policy as the primary store); and the compliance scope decision (which tables contain personal data, which regulation applies, what the minimum and maximum retention periods are). The "default" pathology: the audit log created for SOC 2's 90-day minimum that becomes permanent by default; the user activity table kept because "storage is cheap" that grows 5 GB per month and becomes a GDPR compliance surface nobody classified as personal data; the chat transcript archive retained "for the life of the account" that nobody defined. Writing the retention ADR: data classification inventory as prerequisite, compliance scope statement naming which regulations apply to which tables, legal hold exception policy, and a revisitation condition tied to infrastructure cost thresholds or GDPR territory expansion. Finding retention decisions in AI chat: three session types — early database design ("should we soft-delete or hard-delete users?"), GDPR compliance preparation ("what personal data do we store?"), and infrastructure cost review ("our S3 costs are growing 20% per month, can we archive older data?").
2026-06-14 · ~15 min read
Every API has a versioning strategy — even "no versioning" is a strategy. The decision that determines the cost of every future breaking change is usually made by default, undocumented, and invisible until the first consumer breaks. Three categories of API versioning decisions worth documenting: the strategy choice (URL versioning vs. header versioning vs. no versioning — each encodes a different set of consumer commitments at a different level of visibility); the breaking change definition (what counts as breaking versus backwards-compatible — the missing definition that produces inconsistent API evolution and surprised consumers when different engineers apply different standards to the same type of change); and the deprecation and migration timeline (how long old versions live, how consumers learn about deprecation, what migration support the team provides). The "just in case v1" problem: most teams add /v1/ because it seems like good practice without documenting what condition would trigger a /v2/ — the trigger condition is the entire decision. The breaking change definition gap: a field type change from string to uppercase enum is breaking or not depending on the consumer's comparison logic, and "probably backwards-compatible" is not a versioning policy. The consumer landscape as context: the same versioning strategy that is appropriate for an internal API serving two frontend applications is inappropriate for a public API serving hundreds of third-party developers — and the ADR must name the consumer landscape so a future team can evaluate whether the strategy still fits. Writing the API versioning ADR: Context must include the consumer landscape; Alternatives Considered names each versioning mechanism evaluated; the breaking change definition section is the section most commonly omitted; the deprecation policy sets the terms of the contract. Finding API versioning deliberations in AI chat: they appear at two predictable points — the API design session that precedes the first external endpoint (the alternatives evaluation) and the first breaking change deliberation (the implicit breaking change definition produced through deliberation).
2026-06-14 · ~14 min read
The Redis cache your team added three years ago has no ADR. The commit says "add caching for user profile queries." What it doesn't say: what the query latency was before the cache, which load condition made it necessary, what TTL was chosen and why, or whether the cache is still needed now that the query has been rewritten. Three categories of performance decisions worth documenting: caching decisions (Redis, Memcached, in-process — the cache is visible in the stack but the decision that put it there is not); index decisions (accumulated reactively for queries that may have since changed; each unused index adds write overhead on every insert); and query optimization decisions (the N+1 fix, the denormalized column, the materialized view — each looks like ordinary implementation but represents a performance trade-off with ongoing maintenance obligations). The baseline problem: "we added a cache because the query was slow" — slow at what traffic, with what data volume? The "is this still needed?" question: the cache added for a query pattern that has since been rewritten, the index added for a report that was deprecated, the denormalized column added for a product feature that no longer exists — each accumulates maintenance cost without a record that would allow it to be reconsidered. Writing the performance optimization ADR: Context must include the baseline measurement and load condition; Alternatives Considered must name the alternatives and the specific reason each was not chosen; Revisitation Condition must name checkable triggers for when the optimization should be removed. Finding performance deliberations in AI chat: performance debugging sessions are among the most structured patterns — symptom, query plan, root cause, alternatives, recommendation, verification — and the verification session contains the post-optimization measurement that completes the decision record.
2026-06-14 · ~14 min read
The build vs. buy decision is the one engineering teams most reliably document with the wrong reason. The ADR says "vendor lock-in concerns and need for customizability." What it doesn't say: Auth0's multi-tenant tier was $1,800 a month at projected MAU; the lead engineer had built JWT systems before; the CEO didn't want to pay a vendor for something two engineers could build. Three categories of build-vs-buy decisions worth documenting: feature-level (authentication, search, payment processing — made early, become load-bearing, and inherited by every new CTO who asks "why aren't we using Auth0?"); capability-level ("we'll use Datadog when we're bigger" — the "not yet" condition disappears from the ADR even though it was the entire decision); and platform-level (the vendor market that moves fastest while the internal implementation accumulates switching cost). The vendor evaluation spreadsheet problem: three AI chat sessions, a four-tab comparison matrix, and an engineer who left eighteen months later — the spreadsheet is gone but the build decision it produced is load-bearing infrastructure. The build decision cascade: the first ADR opens fifty implicit downstream decisions about token format, session storage, and multi-tenant isolation, each of which accumulates switching cost independently. The lock-in rationalization in detail: the test for whether vendor lock-in was the actual constraint, why pricing and team familiarity are the real drivers in most early-stage builds, and why the honest record is harder to write but more useful for the re-evaluation in year three. The re-evaluation asymmetry: the original decision was made at zero switching cost; the re-evaluation must include the switching cost accumulated by three years of downstream implementation decisions — and that calculation is only possible if the build ADR was written honestly enough to name the cascade it opened. Writing the build-vs-buy ADR with specific Alternatives Considered (named vendors, actual pricing, specific capability gaps) and a Revisitation Condition that is checkable rather than general. Finding build-vs-buy deliberations in AI chat: they are multi-session, multi-round discussions — the final "we'll build it" is rarely in one session — and the extractor identifies them through the vendor-name comparison pattern combined with elimination markers across the temporal cluster.
2026-06-13 · ~14 min read
Every public interface is a promise. A REST endpoint promises a JSON response shape. A gRPC service promises a Protobuf contract. A message queue event schema promises a payload format. The promise constrains every consumer, and the choice of what to promise — which protocol, which fields, which versioning strategy — is a decision made against alternatives that almost never gets documented. Three types of interface decisions worth documenting: transport and protocol decisions (REST vs. gRPC vs. GraphQL — the consumer profile is the decisive constraint and is completely invisible from the interface itself); schema and shape decisions (envelope vs. root-level, cursor vs. offset pagination, timestamp format — incremental extensions accumulate as a palimpsest of individual choices nobody made explicitly); versioning strategy decisions (URL versioning vs. header versioning vs. no strategy — the decision that determines the cost of every future breaking change, often made by default). The consumer-producer asymmetry: the producer knows which behaviors are intentional promises and which are implementation details; consumers infer intent from observed behavior; without a record distinguishing them, producers cannot change implementation details without breaking consumers who treated them as promises. The incremental extension problem: each backward-compatible addition felt like a non-decision, but collectively they represent structural choices nobody made explicitly. The REST/gRPC/GraphQL decision: the most common technical architecture discussion in AI chat and the most consistently undocumented — the consumer profile, tooling constraints, and capability requirements that drove the selection are fully present in the session and exist nowhere else.
2026-06-13 · ~13 min read
The evaluated-but-rejected library is the most invisible entry in your dependency graph. It isn't in package.json, it doesn't appear in any audit, and the deliberation that produced the rejection exists only in someone's AI chat history from fourteen months ago — until a new engineer joins and proposes the same library again. The dependency re-proposal cycle: proposers have done fresh research; current team members reconstruct reasoning from memory; the asymmetry consistently favors the proposal. Five rejection reasons worth documenting: the security-surface reason (what the library brings in — a specific transitive dependency concern or supply chain risk that may change when the library drops it); the maintenance-burden reason (release cadence, maintainer responsiveness, bus factor, evaluated and found to be a risk at a specific point in time); the native capability sufficiency reason (the native API covered the use cases — the absence of the popular library looks like an oversight to a new engineer); the deliberate dependency diet (explicit architectural policy of minimizing dependency count for auditing, deployment constraints, or supply chain surface); and the internal implementation decision (library rejected because the team built the same capability — without a record, the custom implementation looks like accidental technical debt and a new engineer proposes replacing it with the library that was originally rejected). The rejection record format: title in decision-statement form ("Chose native Temporal API over date-fns for date handling"), Context naming the problem and the evaluated library, Alternatives Considered naming the library and what was chosen instead with the specific rejection constraint, Consequences naming the trade-offs accepted, Revisitation Condition naming the specific trigger for re-evaluation. How to find rejected dependency decisions in AI chat history: sessions that start as installation or comparison questions and conclude with a non-installation have a characteristic shape detectable through negative decision markers; the quarterly extraction pass surfaces them systematically.
2026-06-13 · ~13 min read
Dependency upgrades are almost universally treated as technical tasks, not decisions. What gets committed is "upgrade react to 18.3.1." What disappears is the reasoning — why this quarter rather than last or next, whether pinning and patching was considered, what the gap cost calculation showed, which adjacent decisions the migration forced about rendering patterns or module system boundaries. Three distinct decisions are embedded in every breaking upgrade: the timing decision (the trigger that made this quarter right), the alternatives decision (pin-and-patch, fork, replace, or wrap-and-decouple), and the forced adjacent decisions made inline during migration. The 'why not defer?' question is the dependency management equivalent of the "not building this" record — a deferral without a named revisitation condition becomes permanent by default. The upgrade-vs-replace threshold: the comparison becomes live when migration cost exceeds replacement cost for the same capability, and this evaluation almost never gets written down. How AI chat migration deliberations are among the highest-confidence extraction targets: engineers frame upgrade evaluations as explicit comparison questions from the first message, producing sessions that contain the timing trigger, alternatives, and adjacent decisions in a single recoverable artifact.
2026-06-13 · ~13 min read
Platform teams make decisions for dozens of product teams who weren't in the room. Standard ADR templates assume the reader was present for the decision. Platform ADRs need four additional sections that product ADRs rarely require: a blast radius (which teams and services does this constrain, and what does it explicitly not constrain), an interface contract (what does compliance actually look like from the outside — not "we chose Kafka" but the bootstrap address, the topic provisioning process, the consumer group naming convention, and the responsibility split), a consulted vs. informed record (the political legitimacy signal — which teams had input before the decision, which were notified after), and a revisitation condition (named triggers under which the platform team will re-evaluate the constraint). The constraint propagation problem: an undocumented naming convention propagates through inference across fourteen services over two years; a blast radius section at decision time converts the implicit constraint into an explicit one before the inference chain forms. The temporal mismatch: platform ADRs are written at maximum context and read at minimum context, months later by engineers encountering a constraint they didn't author. Why platform deliberation sessions in AI chat are particularly valuable for extraction: the blast radius section comes from which teams were explicitly named in the original deliberation, and that organizational context is the first thing lost from memory.
2026-06-13 · ~12 min read
Most ADR titles are topics, not decision statements. "PostgreSQL as the primary database" is a topic. "Chose PostgreSQL over MongoDB for primary persistence" is a decision statement. The difference determines whether the record is findable at the list level — whether a new engineer scanning the decisions directory can answer "has this comparison been made?" without opening each file. The anatomy of a decision statement title (verb + subject + rejected alternative + context), how the verb encodes the decision type (chose / rejected / deferred / kept / adopted), why the rejected alternative belongs in the title and is the part most frequently omitted, the five anti-patterns that produce topic titles, the filename constraint, the MADR convention trade-off (noun-phrase titles for supersession stability vs. decision statement titles for retrieval), and why the title is the right diagnostic for whether the body is done.
2026-06-12 · ~12 min read
Post-mortems and ADRs are two separate processes that belong together. The constraint that broke in production was almost always implicit before the incident — the post-mortem surfaces it at peak clarity, but most teams file the learning in an incident ticket where future architectural decisions can't cite it. Three ADR types that post-mortems produce: the original decision ADR (documenting the architectural choice that created the failure mode), the deferral ADR (making explicit the implicit "not yet" decision that allowed the vulnerability to accumulate), and the incident response ADR (the rollback vs. hotfix decision with the actual constraint stated). The 48-hour window: the specific rate at which post-mortem clarity degrades from precise constraint understanding to abstracted lesson. How to use the post-mortem timeline as a date range for the WhyChose extractor — running it on AI chat history from when the original decision was made surfaces the deliberation that was live at the time, which is more accurate than memory. How to structure the post-mortem template to make ADR production a named deliverable, not an oversight.
2026-06-12 · ~13 min read
Writing an ADR doesn't just document a decision — it changes the quality of the decision itself. Three mechanisms: the Alternatives Considered section forces you to name options you dismissed without full evaluation; the constraint statement forces you to distinguish the constraint that was actually decisive from the ones that felt decisive; the Consequences section forces you to name what you gave up. The blank template is the most useful thinking tool in the room, and most teams never use it that way because they open it only after the decision is made. Covers the blank ADR as a pre-decision agenda, the ADR-first meeting pattern, what retrospective ADRs reveal as a decision quality audit, how AI-chat-based decisions change the forcing-function dynamic, and the second-order effect: teams that write ADRs consistently report making better decisions, not just better-documented ones.
2026-06-12 · ~14 min read
Year one is when the most consequential decisions get made and the least documentation happens. Twelve categories ordered by the cost of not having them: tech stack foundational choices, ICP decision (including who you decided NOT to build for), first pricing structure, database and persistence, auth/identity approach, founding-team authority structure, first hire prioritization, infrastructure and deployment, open source vs. closed, monetization model, key deferral decisions (deliberate "not yet" choices with implicit trigger conditions), and the first "no" to a paying customer. For each: what the record looks like, why the absence is expensive, and what AI chat history from year one is likely to contain. Plus the three-sentence minimum viable ADR format, the extraction pass for recovering decisions that were never written down, and the quarterly founding-team review ritual that keeps the starter pack current.
2026-06-12 · ~13 min read
The constraint field is the most load-bearing element of any ADR — every downstream decision that references it is subtly wrong when the original constraint was misidentified. Three patterns that produce false constraints: the misremembered constraint (clean precision that came from a different project), the rationalized constraint (team familiarity reframed as an operational complexity argument), and the absent constraint (the real driver left implicit because it felt too informal to document). A worked case — the 5ms latency requirement that was actually a team familiarity constraint — and how it propagated through three subsequent decisions over eighteen months. The detection test: run the WhyChose extractor on the original AI chat history and check whether the documented constraint appears in the actual deliberation. The correction protocol: a dated Notes amendment, not an in-place edit, so the historical record stays intact while the informational status of the constraint becomes accurate.
2026-06-12 · ~14 min read
Most teams that try to adopt architecture decision records stall not on the format question but on the tooling question. A practical comparison of the three main approaches — adr-tools CLI (fast write, no browser view, bash-only), Log4Brains documentation site (searchable static site, non-engineer-readable, Next.js maintenance burden), and the MADR directory approach (zero dependencies, survives any build-chain change, findability caps at ~80 records). Where each breaks, the choice framework by team type (small/single-repo vs. non-engineer readership vs. multi-repo vs. wiki-first), and the shared gap: all three tools assume the reasoning is already available when the record is written — but the deliberation that produced most decisions happened in AI chat sessions that none of these tools can read.
2026-06-12 · ~16 min read
The decisions most worth documenting are the ones your team made last month — not the ones from two years ago. Recent AI chat history degrades fastest because the three types of context that make a decision record interpretable (explicit reasoning, implicit background constraints, interpersonal context) are all still recoverable in the 90-day window and rapidly inaccessible after it. The team knowledge gap isn't about personnel turnover — it's about divergent recollection: each team member participated in a different part of the AI-assisted deliberation (engineer evaluated technical feasibility, PM evaluated user impact, EM evaluated cross-team dependencies) and none has the complete picture. Four decision types that concentrate in the 90-day window: scope decisions made under time pressure, technical approach decisions made during implementation, constraint decisions made when requirements changed, and deferral decisions with implicit conditions. How to run the 90-day extraction pass for a 5-person team (2–3 hours, 15–25 records), why to check for divergent recollections before writing records, and why the quarterly rhythm — not a one-time project — is what closes the knowledge gap permanently.
2026-06-11 · ~15 min read
Solo founders make 3–5 consequential decisions a week in AI chat sessions. After one year, those decisions are invisible — scattered across hundreds of tabs that were closed the moment the thinking was done. A WhyChose extraction pass on twelve months of exports reveals what categories dominate (stack decisions 30–40%, ICP definition 20–25%, pricing iterations 15–20%, scope deferrals 15–20%), which decisions you thought were temporary are now load-bearing, and why the first-hire onboarding problem starts the day you open your first ChatGPT tab. Includes a practical guide to running the extraction pass: how to export both platforms, how to anchor dates from git history, how to triage by category and permanence, and why the ICP records should be written before anything else.
2026-06-11 · ~14 min read
Engineering managers inherit decisions they didn't make and answer for them as if they did. A decision log changes how the "why did we build it this way?" question gets answered in 1:1s. Four EM use cases: the pre-1:1 brief (pull the two or three decisions most relevant to what each engineer is currently working on before the meeting — turns reactive hedging into prepared context), the onboarding fast-path (a curated reading list of the 8–12 most consequential decisions in a new engineer's domain, shared before their first day), the performance conversation (decision records that give specific feedback grounded in artifacts rather than impressions — essential for engineers doing infrastructure or platform work whose output isn't user-facing features), and the promotion case (every claim in the brief maps to a specific record — the difference between "Alice showed good architectural judgment" and a structured argument with citations). Why EMs should also run the WhyChose extractor on their own AI chat exports, which typically surface a different category of decisions than IC exports: technical debt triage, process adoption, resource allocation reasoning. The private-vs-shared log boundary, and the three curator responsibilities that make ADRs a living practice rather than a documentation backlog.
2026-06-11 · ~13 min read
Hiring calls are the most politically sensitive category of decision record. The candidate comparison doesn't go in git. The role design, capability gap, and team structure decision do. Three categories worth documenting: role prioritization decisions (why this hire now instead of alternatives — these encode product thesis and execution priorities), role design decisions (full-time vs. contractor, senior vs. two mid-levels, specialist vs. generalist — these expire as the team grows and need review triggers tied to team size thresholds), and team structure decisions (org topology, reporting lines, decision authority distribution — organizational architecture that outlasts individuals). The candidate-anonymous format that makes hiring ADRs safe for a technical git repository, what goes in the record vs. what stays private, review triggers for hiring decisions, and how to recover hiring deliberation from AI chat history using offer letter dates to narrow the export window.
2026-06-11 · ~13 min read
IaC config tells you what the infrastructure looks like. It doesn't tell you why prevent_destroy is set, why workspaces were chosen over folder-per-environment separation, or why Kyverno won the admission controller evaluation. Five categories that consistently need documentation: state management decisions, module boundary decisions, security and lifecycle decisions, provider and service decisions, and naming conventions. Where ADRs live relative to .tf files and Helm charts, Terraform-specific patterns (prevent_destroy audits, provider version pinning, module DRY vs. explicit trade-offs), Kubernetes-specific patterns (CRD adoption, namespace architecture, admission controller decisions, Helm vs. Kustomize), how to extract the deliberation-before-migration from AI chat history using git blame to narrow the export window, and review triggers for IaC decisions that expire with provider pricing changes, API deprecations, and team growth thresholds.
2026-06-11 · ~11 min read
Most ADR reviews are format reviews — checking that sections exist and the title is descriptive. Format reviews produce correctly structured records with thin reasoning. A decision review checks whether the ADR captures the three things that make it useful in 18 months: named alternatives with concrete rejection reasons, the constraint that drove the choice, and honest consequences that name a real trade-off. Five checks an ADR author runs before opening the PR, three checks a reviewer runs, and why Alternatives Considered is the most consistently under-populated field in any decision log — plus how running the WhyChose extractor on the AI chat session that preceded the ADR produces better alternatives sections with less reconstruction effort.
2026-06-10 · ~12 min read
A monorepo collapses the team-boundary signal that polyrepos provide for free. In a polyrepo, the repo boundary tells you whose decision it was. In a monorepo, three services' code coexists in the same commit history, and the decision about which service owns a shared concern may predate any of the affected code. Three categories of monorepo decision with different scoping, governance, and folder placement: service-local (one team, one service folder), cross-service (peer teams, downstream stakeholders field), and platform-wide (platform team, root decisions/ folder, all service teams downstream). How the unified commit history narrows the extraction window for AI chat recovery, why shared library decisions are the highest-priority documentation gap in any monorepo, and the ADR folder structure that makes decision scope visible from the file path.
2026-06-10 · ~12 min read
Distributed teams write more than co-located teams and document decisions worse. The artifacts — Slack threads, PR comments, RFC docs, AI chat sessions — are process artifacts that capture conversation, not decision artifacts that capture named alternatives, the constraint that drove the selection, and the reasoning a new team member needs. Three conventions that fix this for async-first teams: the async RFC pattern (RFC document with 48–72 hour comment window, decision summary appended at close — RFC and ADR live in the same document), the decision channel (a dedicated registry where every closed decision gets a one-paragraph announcement with a link to the full record), and the multi-engineer quarterly extraction pass (every team member exports AI chat, output is pooled and triaged async).
2026-06-10 · ~12 min read
Architecture decision records got their name from software architecture, but the decisions that cause the most expensive confusion at early-stage companies are product bets, pricing calls, hiring decisions, and process choices. They share the same three properties that make technical decisions worth documenting. Why the standard ADR format works without modification, the five categories worth logging (product scope bets, pricing and packaging, hiring and team structure, process adoption, market and ICP), and why AI chat exports make non-technical decision recovery more tractable than for technical decisions — because founders frame product deliberations as decisions from the first message.
2026-06-10 · ~11 min read
The most expensive class of undocumented decision is the cross-cutting one — API contracts, shared schemas, event formats — where one team's architecture choice silently creates obligations for a different team. What makes a decision cross-team rather than service-local, the three fields that standard ADR templates omit (downstream stakeholders, notification record, migration obligation), the governance ceremony that prevents constraint collisions (RFC before ADR, not ADR then notification), and how to recover cross-team decision deliberation from AI chat history when the original design review happened in a Zoom call that wasn't recorded.
2026-06-09 · ~11 min read
The new-CTO onboarding problem is the most vivid version of the gap, but the everyday version affects every engineer who joins a team: the "why is it built this way?" questions that nobody can answer confidently because the reasoning happened in AI chat. What the first week looks like when a decision log exists vs. when it doesn't, the four categories of questions new engineers always ask that wikis and READMEs never answer, how to structure the log so it's navigable without prior codebase knowledge, and the onboarding reading list — a curated ten-record subset that turns the first week from inference to reference.
2026-06-06 · ~11 min read
Most teams have either an overcrowded decision log or an empty one. The benchmarks: 3–6 per quarter for very early teams, 5–10 for small teams in active development, 8–15 per squad at scaling stage. "Too few" is a diagnostic signal — almost always a dormant log, not a quiet team. The difference between a quiet quarter (low activity, low decisions) and a dormant log (high activity, near-empty log), what decision debt costs and how it compounds, and how to use extraction data to measure the gap between decisions made and decisions documented.
2026-06-06 · ~10 min read
An ADR written in year one doesn't stay accurate forever. The library gets replaced. The team structure changes. The product pivots. The lifecycle states (Proposed, Accepted, Deprecated, Superseded), when to use each, how to write the links correctly so both records stay navigable, why you should never delete, and what AI chat extraction reveals about how fast decisions actually go stale — technology choices last about eight months before engineers start questioning them; architectural invariants last eighteen months or more.
2026-06-06 · ~10 min read
The most valuable and most commonly missing entry in a decision log is the deliberate choice not to build something. "Yes" decisions leave artifacts — code, PRs, deploys. "No" decisions leave nothing except the reasoning, which lives in a chat session that's impossible to search eight months later. Why AI chat captures rejection reasoning better than ADR tooling, what a "not building this" record looks like, the deferred-vs-permanent distinction that determines whether to write a revisit condition, and how to find these records in the quarterly triage pass.
2026-06-06 · ~10 min read
You extracted your first batch of decisions from AI chat. Now what? The exact 30-minute workflow: request your export the day before, run the extractor, triage the output into four buckets (Promote / Link / Park / Dismiss), write up the 2–5 records that rise to ADR level, archive and set the next quarter's reminder. Plus: which records actually warrant the full ADR treatment, three anti-patterns that kill decision logs, and what the second quarterly review looks like once you have an existing log to check against.
2026-06-05 · ~10 min read
The engineering story behind the WhyChose extractor: five heuristics that went in the bin (sentence-length thresholds, named-entity recognition, first-person verbs in isolation, message-count filtering, Q&A adjacency), four patterns that survived (question shapes, user commit phrases, trade-off markers, reversal markers), and what each failure mode revealed about how engineers actually think with AI. The two-pass architecture that makes it work — and why 3.1% is the right hit rate.
2026-06-05 · ~7 min read
MADR (Markdown Any Decision Record) is the ADR format the ecosystem has converged on — YAML frontmatter, a formal Considered Options section, and a structured Decision Outcome with an explicit "because" clause. What it adds over Nygard's original five-section format, what changed in 4.0 vs 3.x, when to use MADR vs Nygard, how AI chat output maps naturally onto MADR structure, and the tooling stack (Log4Brains, GitHub Action CI validation) that makes the format pay off at scale.
2026-06-05 · ~9 min read
18 months of ChatGPT history, 1,214 conversations, 47MB of export. The extractor surfaced 154 candidates and 38 durable decisions after the durability filter — a 3.1% hit rate. Here's the breakdown by category, the three findings that changed how we work, and what the extractor missed. Pure proof, no pitch.
2026-06-05 · ~8 min read
Three terms for capturing decisions, three different jobs. RFC is the pre-decision proposal (you want input before committing); ADR is the post-decision permanent record (immutable, captures the rejected options, answers "why" at staff turnover); decision log is the broader collection (ADRs plus the lighter product and operations calls). The practical disambiguation — which artifact for which situation — and what to do about the 90% of decisions that happened in AI chat without any of the three.
2026-04-29 · ~9 min read
Almost every team that adopts an ADR practice abandons it within two months. Not because the team is lazy — because the ceremony costs more per decision than people are willing to pay, and the cost compounds every time a record is skipped. The round-trip math, the broken-window effect, and the two-tier shape that actually sticks: extracted records for the 80%, hand-written ADRs only for the load-bearing 20%.
2026-04-25 · ~8 min read
You join a Series A as the new CTO. Six weeks in, someone asks why the stack was picked. The answer isn't in the wiki, isn't in the repo, isn't on the calendar. The reasoning happened — but it happened inside someone else's ChatGPT history, and now they're gone. Why this is the modal new-CTO experience in 2026, why ADR ceremony and Notion wikis don't fix it, and the three-line first-90-days plan that actually does.
2026-06-10 · ~12 min read
Security decisions are the category where undocumented trade-offs are most expensive. SOC 2 auditors ask not just "what are your controls?" but "what was the decision process for each control, and who reviewed it?" The five fields a security ADR adds beyond the standard template (threat model scope, data classification, compliance scope, security reviewers, review triggers), why the standard Consequences section fails for security decisions — it conflates accepted risks with desired outcomes and has no reviewer field or review cadence — the classification problem and how to handle security ADR content that reveals attack surface, how compliance documentation makes security ADRs uniquely valuable at audit time, and why security decisions need review triggers when most ADRs don't require them.
