Why does certificate renewal automation fail silently and what monitoring detects the failure before expiry?

Certificate renewal automation — whether implemented via certbot cron jobs, cert-manager Kubernetes controllers, ACME webhooks, or CA-managed auto-renewal — fails silently when the automation mechanism breaks without producing a user-visible error. The failure mode is silent because the certificate continues serving HTTPS requests normally until the moment it expires: a certbot cron job that fails to execute does not cause any TLS errors during the failure period, it simply does not renew the certificate, and the certificate continues to be served by the web server until its expiry date passes. Let's Encrypt issues certificates with a 90-day lifetime and recommends renewing at 60 days remaining, meaning the window between a renewal failure and a certificate expiry is at most 60 days for a certificate that was just renewed when automation broke, and as little as 30 days for a certificate midway through its validity window. Common silent failure modes include: (1) certbot binary path changes after an OS upgrade or package manager update (the cron job runs as root with a hardcoded path like /usr/bin/certbot that no longer exists after certbot was reinstalled to /usr/local/bin/certbot); (2) certbot cron job writing renewal errors to stdout or stderr with no associated alerting (the error is logged to /var/log/letsencrypt/letsencrypt.log but no monitoring reads this file); (3) cert-manager Kubernetes controller running but unable to solve the ACME challenge (the HTTP-01 challenge fails because an ingress rule change blocked /.well-known/acme-challenge/ requests, or because the DNS-01 challenge fails because a DNS API credential was rotated without updating the cert-manager secret); (4) commercial CA auto-renewal failing because the auto-renewal email went to a departing employee's address. The monitoring strategy that detects renewal failure before expiry requires checking the certificate's NotAfter field directly rather than monitoring the renewal process. Certificate expiry monitoring sends an alert when the certificate's expiry is within 30 days — providing a 30-day window to diagnose and fix the renewal automation failure before users are affected. The monitoring must check the certificate at the TLS layer (connecting to the domain and reading the certificate presented by the server) rather than checking the file on disk (because the file may be renewed but the web server may not have reloaded to pick up the new certificate, a different silent failure mode). Services for certificate expiry monitoring include UptimeRobot's SSL monitoring, Datadog's synthetic TLS check, or a simple cron job running openssl s_client -connect domain:443 -servername domain and parsing the NotAfter date. The monitoring interval should be daily, and alerts should be sent to a shared team channel rather than an individual engineer's email.

What is the compatibility risk of restricting TLS protocol versions and cipher suites for compliance?

Restricting TLS protocol versions and cipher suites for compliance creates a client compatibility risk that varies by the client population connecting to the service. TLS 1.3, finalized in 2018, removed the cipher suite negotiation mechanism used by TLS 1.2 and earlier — TLS 1.3 mandates three specific cipher suites (TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_AES_128_GCM_SHA256) and does not support the cipher suite negotiation that TLS 1.2 used. Clients that do not support TLS 1.3 — specifically Java runtimes before Java 11, Android versions before 10, older versions of OpenSSL (before 1.1.1), embedded devices running lightweight TLS stacks, some enterprise middleware, and some payment processing terminal firmware — cannot connect to a server that has disabled TLS 1.2. PCI DSS 3.2 and later versions require disabling TLS 1.0 and TLS 1.1 (which are cryptographically weak) but explicitly permit TLS 1.2 alongside TLS 1.3. A compliance implementation that disables TLS 1.2 based on a security hardening checklist item that reads 'enable TLS 1.3 only' has gone beyond the compliance requirement and created a client connectivity restriction with no compliance benefit over supporting both TLS 1.2 (with strong cipher suites) and TLS 1.3 simultaneously. Cipher suite restriction within TLS 1.2 carries a similar compatibility risk: disabling non-forward-secret cipher suites (RSA key exchange without Diffie-Hellman) improves security (recorded TLS traffic cannot be decrypted retroactively if the server's private key is later compromised) but breaks compatibility with clients that do not support ECDHE or DHE key exchange. The standard practice for TLS 1.2 cipher suite restriction is to support ECDHE and DHE suites with AES-GCM and CHACHA20-POLY1305, which provides forward secrecy for all clients that support these suites (effectively all modern clients), while disabling the RSA key exchange suites (TLS_RSA_WITH_AES_128_GCM_SHA256, etc.) that do not provide forward secrecy. This cipher suite profile is supported by Mozilla's 'Intermediate' configuration recommendation, which represents the best trade-off between security and compatibility for services with a general-purpose user base. Services with a controlled client population (all clients are under the operator's control and can be upgraded) can use the 'Modern' profile (TLS 1.3 only) safely. Services with a heterogeneous client population including enterprise integration partners, embedded devices, or legacy systems require the 'Intermediate' profile at a minimum, and the decision record must document why the profile was chosen and which client segments were audited for compatibility.

What is OCSP stapling and how does a missing OCSP staple affect TLS handshake latency?

OCSP (Online Certificate Status Protocol) is the mechanism by which a TLS client can verify that a certificate has not been revoked by the issuing certificate authority before it expires. Without OCSP stapling, the client must make a separate HTTP request to the CA's OCSP responder during the TLS handshake — the client extracts the OCSP responder URL from the certificate's Authority Information Access extension, makes a GET or POST request to that URL with the certificate's serial number, and waits for a signed response indicating whether the certificate is valid, revoked, or unknown. This OCSP request adds one full round-trip to the TLS handshake latency: a client connecting to an API endpoint in a different geographic region may experience 50 to 200 milliseconds of additional handshake latency for the OCSP request to the CA's responder, which may itself be geographically distant. More critically, if the CA's OCSP responder is unreachable — because of a CA infrastructure outage, a network partition, or a firewall rule on the client's network that blocks outbound HTTPS to the OCSP responder URL — the client's behavior depends on its OCSP hard-fail policy: clients with hard-fail enabled (uncommon in browser implementations but common in some enterprise security configurations and some programming language TLS stacks) will refuse to complete the TLS handshake, treating an unreachable OCSP responder as a certificate revocation; clients with soft-fail enabled (the default in most browser implementations) will proceed with the handshake after a timeout, accepting the certificate without revocation verification. OCSP stapling solves both the latency and availability problems: the server periodically fetches the OCSP response from the CA's responder (every few hours, before the cached response expires), stores it in memory, and staples it inline in the TLS handshake as part of the Certificate Status Request TLS extension. The client receives the OCSP response as part of the TLS handshake itself, eliminating the separate OCSP request entirely. OCSP stapling requires explicit server configuration: in nginx, 'ssl_stapling on; ssl_stapling_verify on; ssl_trusted_certificate /path/to/chain.pem;'; in Apache, 'SSLUseStapling on; SSLStaplingCache shmcb:/tmp/stapling_cache(128000)'; in Caddy, OCSP stapling is enabled automatically for all managed certificates. The initial HTTPS setup session that runs certbot --nginx frequently does not verify whether OCSP stapling is enabled and functioning, because the TLS handshake succeeds and the green padlock appears regardless of whether the OCSP staple is present — the staple is invisible to the engineer performing the setup verification, visible only to clients that check the Certificate Status Request extension.

When should service-to-service communication use mutual TLS rather than API key authentication?

Mutual TLS (mTLS) is a variant of the standard TLS handshake where both the server and the client present certificates, so that both parties cryptographically authenticate each other before exchanging data. In standard TLS, only the server presents a certificate (the client verifies the server's identity) and the client authenticates to the server using application-layer credentials (an API key, a bearer token, a session cookie). In mTLS, the server also verifies the client's certificate, establishing cryptographic identity for both parties at the transport layer rather than the application layer. Service-to-service communication should use mTLS in place of API key authentication when: (1) the service is handling sensitive data and the threat model includes an attacker who has gained access to the internal network (lateral movement) — API keys can be exfiltrated from environment variables, configuration files, or memory, while mTLS client certificates require access to the private key file, which can be stored in a hardware security module or Kubernetes secret with stricter access controls than environment variables; (2) the organization is implementing a zero trust network model where network-layer trust (being on the internal network) is explicitly not sufficient for service access, and each service must present a cryptographic identity for every connection; (3) the service mesh infrastructure (Istio, Linkerd, Consul Connect) is already present and provides automatic mTLS for all service-to-service communication in the mesh as a default behavior — in this case, disabling mTLS for a specific service requires an explicit policy decision, and the default is to use mTLS; (4) regulatory requirements (PCI DSS for cardholder data environments, HIPAA for healthcare data, FedRAMP for government data) mandate mutual authentication for service-to-service communication within the security boundary. Service-to-service communication does not require mTLS when: the services are in a fully trusted network with no path for lateral movement (a Kubernetes namespace with NetworkPolicy restricting all ingress except from the specific calling services), the operational overhead of certificate rotation for service identities (which should be much shorter than 90 days — Istio issues workload certificates with 24-hour lifetimes by default) cannot be justified by the threat model, or the services use a secrets management system (HashiCorp Vault, AWS Secrets Manager) that provides API key rotation with short-lived tokens, which provides a similar security property to mTLS certificate rotation. The decision record must specify which services are in scope for mTLS, the certificate authority for service identity certificates (separate from the public-facing CA), the certificate lifetime and rotation mechanism, and the enforcement mode (permissive mode that logs but does not enforce while mTLS is being rolled out, versus strict mode that rejects non-mTLS connections).

The TLS and certificate management decision record: why the certificate provisioning model you chose determines your TLS handshake failure surface and your certificate expiry incident ceiling

Published 2026-07-18 · WhyChose

TLS certificate decisions are made when a server is first deployed, when a security hardening checklist is run before a compliance audit, and when a service mesh is introduced to manage service-to-service authentication. The AI session that handles initial HTTPS configuration is practically oriented and terminates at the visible result: it installs certbot or configures Caddy's automatic certificate management, runs the certificate request against Let's Encrypt, verifies that the browser shows a green padlock and that curl returns a valid 200 response over HTTPS, and notes the certificate expiry date in passing. The session succeeds when HTTPS works. The certificate is valid. The session closes.

What the AI session does not produce is the second half of the TLS decision. The session answers "how do I get HTTPS working?" and delivers a valid certificate. It does not ask: what is the renewal automation model and what is the monitoring strategy for detecting silent renewal failures — Let's Encrypt certificates expire after 90 days, certbot cron jobs fail silently when a package manager upgrade moves the certbot binary to a different path, and a cert-manager ACME challenge fails silently when an ingress controller change blocks the challenge validation path, and in all three cases the certificate continues serving requests normally until the expiry date arrives; what is the cipher suite and TLS protocol version selection rationale — TLS 1.3 only eliminates forward-secret-less cipher suites and reduces the TLS negotiation surface but breaks connectivity for Java runtimes before version 11, embedded payment terminals, and enterprise middleware whose upgrade cadence is measured in years, while supporting both TLS 1.2 with strong cipher suites and TLS 1.3 provides forward secrecy for all modern clients without the connectivity restriction; what is the OCSP stapling configuration — a server that presents a valid certificate but does not staple the OCSP response forces each connecting client to make a separate HTTP request to the CA's OCSP responder during the TLS handshake, adding 50 to 200 milliseconds of latency for clients connecting from geographic regions distant from the CA's infrastructure, and creating a dependency on the CA's OCSP responder availability during every TLS connection; what is the certificate chain configuration — a certificate served without its intermediate certificate causes TLS handshake failures for clients that do not have the intermediate in their local trust store, a failure mode that affects mobile clients and some server-side TLS stacks but is invisible to development-environment testing; what is the mTLS specification for service-to-service authentication — whether internal services authenticate to each other using client certificates rather than shared API keys, which services are in scope, what the certificate lifetime is, and what happens when the internal certificate authority certificate itself expires. Each of these questions has an answer that is not derivable from "HTTPS is working and the padlock is green" as the session completion criterion. Each answer determines whether a certificate expiry at 3am is discovered by a customer error report or by a monitoring alert 30 days in advance, whether a compliance audit that mandates TLS 1.3 causes 8% of enterprise integration partners to stop being able to connect to the API, whether a CA infrastructure incident produces 200 milliseconds of additional TLS handshake latency for every user connection or produces a transparent zero-latency handshake because OCSP stapling was configured, and whether a service mesh introduction that enforces mTLS in permissive mode for 60 days before switching to strict mode breaks four internal services that connect to the payment service using API keys that were not in scope for the mTLS rollout plan. The answers are in the AI sessions. They are almost never written down.

Two ways TLS certificate decisions produce the wrong outcome in production

The silent certbot renewal failure and the 89-day expiry window

A developer-tools startup deploys its API on a single Ubuntu 22.04 VPS. The founding engineer runs a ChatGPT session to set up HTTPS: installs nginx, installs certbot, runs certbot --nginx -d api.example.com, follows the prompts, confirms the certificate is issued, checks that https://api.example.com returns 200. The session also notes that certbot --nginx has installed a systemd timer for automatic renewal — on Ubuntu 22.04, certbot's Debian package creates a systemd service (certbot.service) and timer (certbot.timer) rather than a cron job. The timer runs twice daily and renews certificates that will expire within 30 days. The session ends. HTTPS is working. The renewal automation is in place via the systemd timer. The founding engineer considers certificate management complete.

Eight months later, the team upgrades the VPS from Ubuntu 22.04 to Ubuntu 24.04. The upgrade is performed in-place using do-release-upgrade, which is the recommended upgrade path for Ubuntu LTS versions. The upgrade completes successfully. The application is verified working post-upgrade. During the upgrade, certbot is also upgraded from version 1.21.0 (packaged with Ubuntu 22.04's apt repositories) to version 2.7.4 (packaged with Ubuntu 24.04). The upgrade changes the certbot executable path from /usr/bin/certbot to /usr/bin/certbot (same path in this case), but the systemd timer unit file is replaced by the new package version. The new timer file has a different ExecStart path that references a wrapper script in /usr/lib/certbot/ that no longer exists because the 24.04 package uses a different internal structure. The systemd timer is enabled and running — systemctl status certbot.timer shows "active (waiting)" — but when the timer fires and calls the wrapper script, the renewal attempt fails with "No such file or directory" and the failure is recorded in journald but generates no alert. The engineering team does not monitor journald for certbot failures. The certificate renewal failures begin appearing in journald on the day of the Ubuntu upgrade, twice daily, silently.

Eighty-nine days after the Ubuntu upgrade, the Let's Encrypt certificate expires at 11:47pm. At 11:51pm, the first customer reports in the shared Slack channel that the API is returning an SSL error. The on-call engineer wakes up, checks the monitoring dashboard — all application-layer metrics are green, because the issue is not in the application layer. The engineer opens a browser to api.example.com and sees "NET::ERR_CERT_DATE_INVALID." The certificate has expired. The engineer runs certbot renew manually, which succeeds in 30 seconds (the ACME challenge has not changed), and reloads nginx. The service returns to normal at 12:04am. The incident duration is 17 minutes. The post-mortem identifies the systemd timer wrapper script failure introduced by the Ubuntu upgrade. The action item is to add certificate expiry monitoring — a daily check that alerts when the certificate's NotAfter date is within 30 days. The action item is assigned, added to the backlog, and not completed for three months because the team's immediate priority is the Q3 roadmap. A second certificate expiry incident occurs four months after the post-mortem. The certificate expiry monitoring that would have prevented it was never implemented because the decision record for the TLS setup session had not specified a monitoring requirement — only "certbot installed, renewal automation via systemd timer, certificate valid until [date]."

The TLS 1.3-only migration and the enterprise client connectivity failure

A B2B SaaS company processes financial data for enterprise customers. Before a SOC 2 Type II audit, the company engages a security consultant who reviews the TLS configuration of the company's API endpoints. The consultant's report notes that the API endpoints support TLS 1.2 with non-forward-secret RSA key exchange cipher suites alongside ECDHE cipher suites. The report recommends: "Disable TLS 1.2 and enable TLS 1.3 only, with cipher suites restricted to TLS_AES_256_GCM_SHA384 and TLS_CHACHA20_POLY1305_SHA256. This aligns with NIST SP 800-52 Rev 2 guidance and demonstrates a strong cryptographic posture to the auditors." The engineering team implements the recommendation in a Claude session: updates the nginx ssl_protocols directive to TLSv1.3 only, removes the ssl_ciphers directive (TLS 1.3 cipher suites are not configurable in nginx; the protocol version restriction is sufficient), deploys the configuration change to the staging environment, tests from the engineer's laptop using curl — TLS 1.3 handshake confirmed. The change is deployed to production. The SOC 2 audit passes. The TLS configuration finding is marked resolved.

Three days after the production deployment, the customer success team receives the first escalation: an enterprise customer reports that their integration with the API has stopped working. Their middleware system — a Java 8-based ETL platform that was last updated in 2019 — cannot establish a TLS connection to the API endpoint. Java 8 supports TLS 1.2 by default; TLS 1.3 was added in Java 11 and backported to Java 8 in update 251 (released April 2020), but the customer's ETL platform vendor has not released an update incorporating the backport and the customer is running a version from 2019. The ETL platform's TLS stack does not support TLS 1.3 and cannot connect to a TLS 1.3-only endpoint. The customer's integration returns "javax.net.ssl.SSLHandshakeException: No appropriate protocol" at connection time. The customer's engineering team spends two days diagnosing the failure before escalating to the company. A second escalation arrives the following week: a different customer's payment terminal integration uses an embedded TLS stack based on OpenSSL 1.0.2 that predates TLS 1.3. A third escalation: a customer's business intelligence tool uses a Windows-based ODBC connector built against an older OpenSSL version that does not include TLS 1.3.

Investigation reveals that 8% of the company's enterprise API integrations use TLS stacks that do not support TLS 1.3. The security consultant's recommendation was technically correct — TLS 1.3 only does provide a stronger cryptographic posture — but was not conditioned on an audit of the client population. SOC 2 does not require TLS 1.3 only; it requires strong cryptography, which TLS 1.2 with ECDHE cipher suites and AES-GCM satisfies. The compliance requirement and the security recommendation were conflated into a single configuration change that eliminated the RSA key exchange cipher suites (a genuine security improvement) alongside TLS 1.2 support (not required by the compliance framework and incompatible with 8% of enterprise clients). Re-enabling TLS 1.2 with strong cipher suites (TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256) while keeping TLS 1.3 also enabled would have satisfied both the SOC 2 requirement and maintained compatibility with the enterprise client base. The TLS configuration session had not documented the rationale for disabling TLS 1.2, had not audited the enterprise client population for TLS 1.2 dependencies, and had not specified whether TLS 1.2 was disabled because the compliance requirement demanded it or because the security hardening recommendation went beyond what the compliance framework required.

Three structural properties that TLS certificate decisions determine

The certificate provisioning and renewal model — the expiry ceiling

The certificate provisioning model specifies the certificate authority, the issuance mechanism, the certificate lifetime, the renewal automation mechanism, and the monitoring strategy for detecting renewal failures. Without a documented provisioning model, the team's mental model is "certbot is installed and handles renewal automatically" — a model that is accurate until it isn't, and which provides no specification of what "automatic" means in terms of the renewal mechanism, the failure modes, or the monitoring that detects when automatic renewal has stopped working. The provisioning model must specify five properties. First, the certificate authority and its issuance constraints: Let's Encrypt issues Domain Validation certificates with a 90-day lifetime and a 5 certificates per domain per week rate limit; commercial CAs (DigiCert, Sectigo, GlobalSign) issue certificates with lifetimes of 90 days to 398 days (398 days is the maximum allowed by browser root programs since September 2020) and provide organization validation and extended validation tiers for use cases where the certificate's organization field is visible to users; internal PKI (HashiCorp Vault PKI, AWS Private CA, cfssl) issues certificates signed by an internal CA trusted only within the organization, used for service-to-service authentication and development environments. Second, the renewal automation mechanism and its failure modes: certbot systemd timer or cron job (failure mode: timer/cron is disabled or the certbot binary path changes; monitoring: check journald or cron output and certificate expiry date); cert-manager Kubernetes controller (failure mode: ACME challenge HTTP-01 path is blocked by ingress configuration change, DNS-01 challenge credentials are rotated without updating the cert-manager secret, or the certificate custom resource is deleted without the cert-manager deployment being aware; monitoring: Certificate resource status condition and expiry date in cert-manager's metrics); Caddy automatic certificate management (failure mode: Caddy's internal ACME client fails to renew because the Caddy process is running under a user that cannot write to the certificate storage path, or the domain's DNS record changes without Caddy's configuration being updated; monitoring: Caddy's admin endpoint certificate status and expiry monitoring). Third, the certificate expiry monitoring strategy: a monitoring system that checks the certificate presented over TLS at the domain and alerts when the expiry date is within 30 days provides a 30-day window to diagnose and fix the renewal automation failure before users are affected; the alert should go to a shared team channel, not an individual engineer's email; the monitoring should check the externally presented certificate (not the file on disk, which may be renewed but not yet loaded into the server process). Fourth, the incident response procedure for an expiry: who is responsible for manual certificate renewal if automation fails, what command is run (certbot renew --force-renewal for Let's Encrypt; acme.sh --renew for acme.sh; kubectl delete certificate in cert-manager to trigger re-issuance), and what server reload is required after the certificate file is updated (nginx -s reload, systemctl reload caddy). Fifth, the certificate storage and access model: where the certificate and private key files are stored on disk, what Unix permissions protect the private key file, whether the private key is stored in a secrets management system alongside the certificate, and what the procedure is if the private key is leaked — which requires immediately revoking the certificate and re-issuing (the secrets management decision record specifies the vault or secrets store in use; TLS private keys are high-sensitivity secrets whose storage and rotation policy must be specified at the same level as database credentials and API keys).

The cipher suite and protocol version configuration — the compatibility floor

The cipher suite and protocol version configuration is the specification of which TLS versions and cipher suites the server accepts, the rationale for the selection, the client populations audited for compatibility, and the compliance framework that the selection satisfies. Without this specification, the configuration that is in place is whatever the initial HTTPS setup session produced — often the web server's default, which varies by web server version and operating system, and which may include weak cipher suites (TLS_RSA_WITH_AES_128_CBC_SHA256, which does not provide forward secrecy), may or may not include TLS 1.3, and may not match the organization's compliance requirements. The specification must document four properties. First, the TLS version policy: whether TLS 1.0 and 1.1 are disabled (both are deprecated by RFC 8996 and prohibited by PCI DSS 3.2 for cardholder data environments and must be disabled for any payment-processing or financially regulated service), whether TLS 1.2 is enabled alongside TLS 1.3 (the "Intermediate" profile supporting both versions, covering essentially 100% of clients released after 2013), or whether only TLS 1.3 is enabled (the "Modern" profile, covering clients released after 2018, appropriate for services with a fully controlled client population). Second, the cipher suite selection within TLS 1.2: TLS 1.2 cipher suite selection is meaningful (TLS 1.3 cipher suites are fixed by the protocol and not configurable); the selection must disable non-forward-secret RSA key exchange cipher suites (TLS_RSA_WITH_AES_*) while enabling ECDHE suites (TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 as the minimum forward-secret suite list), and must disable 3DES, RC4, NULL cipher suites, and export cipher suites if the web server version includes them by default. Third, the client compatibility audit: which client populations connect to the service, whether any client population uses Java runtimes before version 11 (TLS 1.3 not supported in older versions), embedded device firmware, legacy enterprise middleware, or other TLS stacks with restricted version support; the audit result must be documented and referenced as the rationale for the version policy choice. Fourth, the compliance framework citation: which specific control in PCI DSS, SOC 2, FIPS 140, NIST SP 800-52, or other applicable framework the cipher suite configuration satisfies, and whether the configuration goes beyond the compliance requirement (as in the TLS 1.3-only recommendation from a security consultant that was not required by the SOC 2 framework) — because going beyond the compliance requirement creates a compatibility constraint that must be justified by a threat model, not just a security posture preference. The security threat model and compliance decision record specifies the applicable compliance frameworks and their cipher suite requirements; the TLS configuration must be revisited whenever the compliance framework changes (PCI DSS version 4.0 cipher suite requirements differ from version 3.2) or when the client population composition changes (a new enterprise integration partner is onboarded using a legacy Java runtime).

The OCSP stapling and certificate chain configuration — the handshake reliability surface

The OCSP stapling and certificate chain configuration determines the TLS handshake latency, the reliability of the TLS handshake under CA infrastructure failures, and the ability of clients without intermediate certificates in their trust store to validate the certificate chain. These properties are set at the time of the initial HTTPS configuration session and are invisible during normal operation — they only surface as performance differences (50 to 200 milliseconds of additional TLS handshake latency for clients connecting without a stapled OCSP response), availability incidents (CA OCSP responder outage causes TLS handshake failures for clients with OCSP hard-fail policies), or sporadic certificate validation failures (clients that do not have the intermediate CA certificate in their trust store receive a chain validation failure). The configuration specification must document three properties. First, the OCSP stapling configuration: whether the server is configured to fetch and staple OCSP responses inline in the TLS handshake, the OCSP response cache lifetime (Let's Encrypt OCSP responses are valid for 7 days; the server should refresh the staple daily to avoid serving a stale staple), the failure behavior when the OCSP responder is unreachable (the server should continue serving the certificate without a staple rather than refusing TLS connections, while logging the OCSP fetch failure for monitoring), and the verification of the stapling configuration post-deployment (openssl s_client -connect domain:443 -status shows the OCSP Response Status in the TLS handshake output; a status of "successful" with a valid response means stapling is working; "no response sent" means it is not). Second, the certificate chain configuration: whether the certificate file served by the server includes the full chain (end-entity certificate plus all intermediate CA certificates up to but not including the root CA), or only the end-entity certificate; Let's Encrypt's certbot populates /etc/letsencrypt/live/domain/fullchain.pem with the full chain and /etc/letsencrypt/live/domain/cert.pem with only the end-entity certificate — the nginx ssl_certificate directive should reference fullchain.pem, not cert.pem, and a misconfiguration that references cert.pem serves only the end-entity certificate, which causes validation failures for clients that do not have the Let's Encrypt intermediate CA in their trust store (primarily server-side TLS clients and some mobile environments); the certificate chain configuration must be verified post-deployment by checking that openssl s_client shows all expected chain certificates in the output and that testssl.sh reports "Chain of trust: OK." Third, the certificate transparency monitoring: Let's Encrypt logs all issued certificates to the Certificate Transparency (CT) log network, which means any certificate issued for the domain is publicly visible within minutes; CT monitoring (via crt.sh, Facebook CT Monitor, or Cloudflare's CT monitoring) provides visibility into unexpected certificate issuance — a certificate issued for the domain by an unexpected CA indicates either a misconfiguration (a staging or development environment is requesting certificates from Let's Encrypt for production domains) or a certificate misdirection attack; the CT monitoring policy must specify who receives alerts for unexpected certificate issuance and what the response procedure is. The CDN decision record specifies whether a CDN is terminating TLS in front of the origin server; if TLS termination is at the CDN layer, the OCSP stapling and cipher suite configuration is controlled by the CDN provider's settings panel rather than the origin server configuration, and the TLS decision record must specify which TLS properties are managed at the CDN layer versus the origin layer, and what the procedure is to update the CDN layer TLS configuration when the compliance requirements change.

Three AI session types that embed TLS decisions without documenting them

The initial HTTPS setup session is where the certificate provisioning model is established without being named as a decision. The session is task-oriented and terminates at the green padlock: install certbot or configure Caddy, run the ACME challenge, verify HTTPS, note the certificate expiry date, and close. The renewal automation mechanism that is configured during this session — whether it is a systemd timer installed by the certbot Debian package, a cron job added manually, a cert-manager deployment in Kubernetes, or Caddy's built-in certificate management — is treated as infrastructure plumbing rather than an architectural decision. The session does not specify what happens when the renewal automation fails, because the automation is expected to be reliable and the question of failure modes is not the session's immediate concern. The certificate expiry date is noted in a project management comment or in a Notion page, not as a monitoring requirement that triggers an alert when the expiry is within 30 days. The OCSP stapling and certificate chain configuration are not verified post-deployment, because the TLS handshake works and the padlock is green — the absence of an OCSP staple is invisible to the browser, and the full-chain vs end-entity certificate distinction is only visible if the engineer runs openssl s_client and reads the output carefully. The cipher suite configuration is whatever the web server default provides, unless the session was specifically focused on TLS hardening; even then, the rationale for the cipher suite selection (which compliance framework it satisfies, which client populations were audited for compatibility) is not documented in the session output. The session result is "HTTPS is working" — not "certificate provisioning model is Let's Encrypt via certbot systemd timer, expiry monitoring is configured at 30-day threshold, cipher suite profile is Intermediate (TLS 1.2 + TLS 1.3 with forward-secret suites only), OCSP stapling is enabled and verified, full chain is served." The second result requires treating the TLS setup as a decision with consequences rather than as a task with a binary completion criterion.

The security hardening or compliance preparation session embeds TLS protocol decisions as configuration changes without a client compatibility audit. The session is driven by an external requirement — a penetration test finding, a compliance framework control, a security consultant recommendation, or an internal security review checklist — and terminates when the finding is marked resolved or the control is satisfied. The TLS hardening session updates the ssl_protocols directive to remove TLS 1.0 and 1.1 (which is universally appropriate and not compatibility-risk-bearing for any modern client), but may also disable TLS 1.2 in response to a recommendation that TLS 1.3 provides stronger security. The session does not audit the enterprise client population connecting to the API for TLS 1.2 dependencies, because the session's scope is the server configuration, not the client base. The cipher suite hardening step disables RSA key exchange cipher suites to enforce forward secrecy, which is a genuine security improvement, but the session may also inadvertently disable ECDSA-based cipher suites if the server's private key is RSA (the ECDHE_ECDSA suites require an ECDSA key pair, not RSA; a server with an RSA certificate and key can only use ECDHE_RSA suites). The session result is logged in the security tracking system as "TLS hardening complete — TLS 1.3 enabled, deprecated cipher suites removed, PCI DSS control 4.2.1 satisfied." The client compatibility consequence of the TLS 1.2 removal is not visible in the security tracking system and is not detected until enterprise integration partners report connection failures. The DNS architecture decision record specifies the DNSSEC configuration; a security hardening session that enables both DNSSEC and TLS 1.3-only creates two independent compatibility constraints — DNSSEC-validating resolver incompatibility for CDN migrations and TLS 1.3 incompatibility for legacy enterprise clients — neither of which is documented as a constraint that future infrastructure changes must account for.

The service mesh or mTLS configuration session establishes service-to-service authentication without specifying the rollout model or the scope of enforcement. The session is focused on a specific technical goal — deploying Istio, enabling cert-manager workload certificates, or adding client certificate authentication to the internal API gateway — and terminates when the immediate configuration is working. The mTLS rollout for a Kubernetes cluster proceeds in permissive mode (requests without client certificates are accepted, enabling gradual rollout), and the session switches the cluster policy to strict mode (client certificates required for all service-to-service communication) without auditing whether all services in the cluster have been configured with valid workload certificates. The session may not identify which services connect to which other services using application-layer credentials (API keys, bearer tokens) that will break when strict mTLS enforcement rejects connections from services that have not been issued workload certificates. The internal certificate authority's root certificate has a 10-year lifetime that is never documented in the decision record; when it expires, the entire service mesh's certificate chain becomes invalid simultaneously, and the team's first encounter with the root CA expiry is when all service-to-service connections in the cluster fail at the same moment. The internal CA certificate rotation procedure — which requires issuing a new root CA certificate, distributing it to all trust stores in the cluster before the old root expires, and transitioning workload certificate issuance to the new root — is not documented because the decision record for the mTLS session captured "mTLS enabled in strict mode, Istio 1.17, cert-manager issuing workload certificates with 24-hour lifetime" but not "internal CA root certificate expires 2036-03-14, rotation procedure is [procedure]." The container orchestration decision record specifies the Kubernetes version and the service mesh in use; the TLS decision record must specify the internal CA configuration alongside the public-facing TLS configuration, because the internal CA has a separate expiry timeline, a separate renewal procedure, and different operational failure modes from the Let's Encrypt certificates used for external traffic.

The five sections of a TLS and certificate management decision record

The first section documents the certificate authority selection and provisioning model. The CA choice determines the certificate lifetime, the issuance mechanism, the organizational validation tier, and the renewal automation options available. Let's Encrypt is appropriate for domain validation certificates on public-facing services where 90-day certificate lifetimes are acceptable and ACME automation is possible; the rationale for choosing Let's Encrypt over a commercial CA must include acknowledgment of the 90-day lifetime (shorter than the 398-day maximum for commercial CAs) and the implication that renewal automation is not optional — a manual renewal process for Let's Encrypt certificates cannot realistically be maintained across hundreds of domains or across team member turnover. Commercial CAs are appropriate when the certificate needs to carry organization validation information visible in the certificate's Subject field (required by some enterprise security policies for internal tooling), when the certificate lifetime must be longer than 90 days for operational reasons (a certificate installed on a device that cannot be easily updated remotely), or when the CA's customer support and SLA are required for the organization's risk model. Internal PKI is appropriate for service identity certificates (mTLS client certificates issued to internal services), development environment HTTPS, and internal tooling that does not receive traffic from the public internet. The provisioning model section must also specify the certificate renewal lead time (how many days before expiry renewal is attempted — 30 days for Let's Encrypt's recommended practice, 60 days for commercial CAs with longer lead times for reissuance), the renewal automation mechanism and its dependencies, and the certificate expiry monitoring requirement. The infrastructure decision record specifies the servers and load balancers in use; TLS certificates must be issued and configured for each public-facing hostname, and the provisioning model must enumerate all hostnames requiring certificates and the CA and automation mechanism for each.

The second section documents the renewal automation and monitoring model. The specification must include the complete renewal automation configuration — not just "certbot installed" but "certbot 2.7.4 installed via Ubuntu 24.04 apt repository, renewal managed by systemd timer certbot.timer configured to run twice daily at 12:00 UTC and 00:00 UTC, renewal attempted when certificate expiry is within 30 days, renewal command is certbot renew --quiet, successful renewal followed by nginx reload via post-hook /etc/letsencrypt/renewal-hooks/post/reload-nginx.sh" — because this level of specificity is what allows a new team member to verify that the automation is working, diagnose a failure, and reproduce the automation setup on a new server. The monitoring requirement must specify the monitoring mechanism (OpenSSL s_client check via cron, UptimeRobot SSL monitoring, Datadog synthetic TLS check, or Prometheus blackbox exporter with TLS certificate expiry metric), the alert threshold (30 days before expiry provides a 30-day window; 14 days provides a tighter window but leaves less time for diagnosis and resolution), the alert routing (team channel rather than individual email), and the escalation procedure if the alert is not acknowledged within 24 hours. The monitoring requirement must also specify what constitutes a failed renewal: certbot's exit code is non-zero (check via systemd unit status or cron mail), cert-manager's Certificate resource condition is not "Ready: True" for more than 24 hours, or the certificate's NotAfter date has not advanced within the expected renewal window after the renewal automation ran. The monitoring specification must include the procedure for manual certificate renewal when automation fails — the command to run, the expected output, the server reload required — so that the 3am engineer responding to a certificate expiry alert has a runbook and does not need to reconstruct the renewal procedure from memory. The secrets management decision record specifies where API keys and credentials are stored; TLS private keys must be stored with the same access controls as other high-sensitivity secrets, and the secrets management specification must include the private key backup and recovery procedure in the event of key loss or server replacement.

The third section documents the cipher suite and protocol version configuration. The specification must state the TLS configuration profile by name (Mozilla Intermediate, Mozilla Modern, or a custom profile with explicit justification), the specific TLS versions enabled (TLS 1.2 and TLS 1.3 for the Intermediate profile; TLS 1.3 only for the Modern profile), the cipher suites enabled within TLS 1.2 (the ECDHE suites with AES-GCM and CHACHA20 providing forward secrecy), the cipher suites explicitly disabled (RSA key exchange suites, 3DES, RC4, NULL, export suites), the compliance framework control that the configuration satisfies (PCI DSS 4.0 requirement 4.2.1, NIST SP 800-52 Rev 2 Section 3.3, or equivalent), and the client compatibility audit results (which client populations were verified to support the configured TLS version and cipher suites). The specification must also document what is not in scope: if TLS 1.3 only was considered and rejected because the enterprise client population includes Java 8 integrations, the rejection rationale must be recorded explicitly, because a future security hardening session may re-recommend TLS 1.3 only without knowing that the enterprise client compatibility constraint was already evaluated and rejected. The web server configuration directive must be specified explicitly (the nginx ssl_protocols and ssl_ciphers values, the Apache SSLProtocol and SSLCipherSuite values, or the Caddy tls block with custom min_version and cipher_suites) so that the deployed configuration can be verified against the decision record. The forward secrecy guarantee must be stated: TLS 1.3 provides forward secrecy for all connections because all TLS 1.3 cipher suites use ephemeral key exchange; TLS 1.2 provides forward secrecy only when ECDHE or DHE cipher suites are used, not when RSA key exchange is used. The load balancer decision record specifies whether TLS termination occurs at the load balancer; if TLS is terminated at the load balancer rather than the origin server, the cipher suite configuration is applied at the load balancer and must be specified in both the TLS decision record and the load balancer configuration, with a note that changes to the cipher suite configuration require updating the load balancer's TLS policy, not the origin server's web server configuration.

The fourth section documents the OCSP stapling and certificate chain configuration. The specification must state whether OCSP stapling is enabled on each TLS endpoint, the OCSP staple refresh interval, the server behavior when the OCSP responder is unreachable (serve without staple and log the failure, rather than refusing connections), and the verification procedure used to confirm that stapling is working post-deployment. The certificate chain configuration must specify which certificate file is served by each web server configuration directive — fullchain.pem for nginx ssl_certificate, the chain file path for Apache SSLCertificateChainFile, or Caddy's automatic chain management — and the verification procedure (openssl s_client -connect domain:443 shows all expected intermediate certificates in the chain output, testssl.sh reports "Chain of trust: OK"). The certificate transparency monitoring policy must specify whether CT log monitoring is in scope (for high-value domains where unauthorized certificate issuance is a material security risk), the monitoring service used, and the alert procedure for unexpected certificate issuance. If TLS termination is at a CDN layer, the section must specify whether the CDN provider enables OCSP stapling by default (Cloudflare enables OCSP stapling by default for all proxied domains; Fastly and Akamai require explicit configuration), what the CDN provider's cipher suite and TLS version defaults are, and whether there is a mechanism to customize the CDN provider's TLS configuration for compliance requirements. The HTTP Strict Transport Security (HSTS) header configuration must also be specified: the max-age value (31536000 seconds is the standard one-year value required for HSTS preloading), whether the includeSubDomains flag is set (which applies HSTS to all subdomains, preventing HTTPS downgrade attacks on any subdomain), and whether the domain is submitted to the HSTS preload list (which causes browsers to refuse HTTP connections before the first HTTPS connection is established, providing HSTS protection for first-time visitors). The security threat model and compliance decision record specifies whether HSTS preloading is in scope; HSTS preloading is difficult to reverse (removal from the preload list takes months to propagate through browser updates), so the decision to submit to the preload list must be documented and the implications for future domain changes (renaming the domain, moving content to a new subdomain structure) must be acknowledged.

The fifth section documents the mTLS specification for service-to-service authentication. The specification must state whether mTLS is in scope for the deployment (whether service-to-service communication uses client certificates for authentication), which services are in scope for mTLS enforcement, the certificate authority for service identity certificates (the internal CA, specified separately from the public-facing CA), the certificate lifetime for workload certificates (Istio default is 24 hours; cert-manager can issue certificates with any lifetime; shorter lifetimes reduce the blast radius of a compromised certificate but increase the operational overhead of managing rotation), the rotation mechanism (automatic rotation by the service mesh sidecar, cert-manager CertificateRequest renewal, or manual rotation via a deployment restart), and the enforcement mode rollout plan (permissive mode duration, strict mode cutover date, and the procedure for auditing all services before strict mode enforcement). The internal CA root certificate lifetime and rotation procedure must be documented explicitly: the root CA certificate lifetime (commonly 10 years for internal CAs), the expiry date, the rotation procedure (issuing a new root CA, distributing it to all trust stores before the old root expires, transitioning workload certificate issuance to the new root), and the monitoring strategy for the root CA expiry (an alert at 1 year before expiry provides sufficient lead time for the rotation procedure). If mTLS is not in scope, the rationale must state why — whether the services are in a network environment where lateral movement is not a modeled threat (fully isolated Kubernetes namespace with NetworkPolicy), whether the secrets management system provides API key rotation with security properties equivalent to short-lived mTLS certificates, or whether the operational overhead of managing mTLS certificates is not justified by the current threat model — and the condition under which mTLS would be reconsidered (when the team size exceeds N engineers and the probability of a lateral movement attack becomes relevant, when a compliance framework mandates mutual authentication). The new CTO onboarding problem with TLS is that every endpoint presents a valid padlock regardless of whether the certificate is renewed manually or automatically, whether OCSP is stapled or forces each client to make a separate CA request, whether TLS 1.2 was disabled because compliance required it or because a security consultant recommended it without auditing the enterprise client base, and whether the service mesh's internal CA root certificate expires in 1 year or 9 years. The incoming technical leader inherits a TLS configuration they cannot audit from the browser or from a curl command. WhyChose's open-source extractor surfaces the HTTPS setup sessions, the compliance hardening sessions, and the service mesh configuration sessions from AI chat history before the certbot systemd timer failure that followed the Ubuntu upgrade becomes the team's primary documentation of why certificate expiry monitoring is not optional, before the TLS 1.3-only security hardening session that disabled TLS 1.2 without a client compatibility audit becomes the primary documentation of why cipher suite decisions require an enterprise client base audit, and before the internal CA root certificate expiry — the one that was never monitored because it was not in the public-facing certificate namespace — becomes the primary documentation of why the service mesh certificate authority has the same operational requirements as the public-facing certificate authority.

The HTTPS setup session that ran certbot --nginx and verified the padlock, the security hardening session that disabled TLS 1.2 because the security consultant's report said TLS 1.3 provides stronger security, and the service mesh session that enabled mTLS in strict mode without auditing whether all services had been issued workload certificates — each produced TLS decisions whose operational consequences (a certificate expiry incident because the systemd timer broke silently during an OS upgrade with no monitoring to detect it, a three-week customer success escalation because 8% of enterprise integration partners cannot connect after TLS 1.2 was disabled, a cascading service mesh failure because the internal CA root certificate expired without a documented rotation procedure) exceed what a provisioning model specifying renewal automation failure modes and a 30-day expiry monitoring threshold, a cipher suite selection documenting the enterprise client audit and the compliance framework control, an OCSP stapling configuration verified post-deployment, and a service mesh mTLS specification with a documented internal CA rotation procedure would have cost to specify at the time the founding TLS decisions were made. The decisions are in the AI sessions: the renewal automation mechanism accepted as infrastructure plumbing rather than a decision with a documented failure mode, the TLS version restriction applied because the security recommendation was implemented without asking whether the compliance framework required it, the OCSP stapling configuration skipped because the padlock was green and the absence of a staple was invisible, the internal CA root certificate with a 10-year lifetime and no expiry monitoring because the session that deployed cert-manager documented the workload certificate lifetime but not the root CA lifetime. WhyChose's open-source extractor surfaces these TLS setup sessions, security hardening sessions, and service mesh configuration sessions as structured decision records before an unplanned certificate expiry reveals that the renewal automation broke silently six weeks ago and no monitoring detected it, before a compliance-driven TLS hardening breaks enterprise integrations because the cipher suite selection was implemented without an enterprise client audit, and before a service mesh root CA expiry takes down all service-to-service communication in the cluster because the 10-year certificate lifetime made monitoring feel unnecessary. The decisions never written down in a TLS deployment are the renewal automation failure mode specification with the expiry monitoring threshold that turns a 3am customer-reported certificate error into a 30-day-early engineering alert, the cipher suite selection rationale with the enterprise client compatibility audit that prevents a compliance hardening session from breaking 8% of integrations, the OCSP stapling verification that turns every TLS handshake from a two-round-trip operation dependent on the CA's infrastructure availability into a single-round-trip operation with the staple included, and the internal CA root certificate rotation procedure with the expiry monitoring that ensures the service mesh's certificate authority does not become the most impactful undocumented expiry in the infrastructure.