Skip to content

Beyond Post-Quantum Bloat: Engineering Strategies to Contain TLS Handshakes

Post-quantum cryptography makes TLS handshakes bigger. Here's how to shrink the payload and cut down on full handshakes, starting today.

Ivan Ristic·Chief Scientist
Published: November 9, 2026·5 min read

In my previous post, we looked at how post-quantum key establishment impacts performance. The short answer? CPU usage doesn't change much, but TLS handshake size increases significantly. Upgrading from classical X25519 to hybrid X25519MLKEM768 adds about 1.2 KB to the TLS handshake in the critical direction. When we inevitably add post-quantum signatures (like ML-DSA) on top, certificate chains will easily exceed TCP’s initial congestion window (14.5 KB), forcing extra round trips (RTTs) and introducing real network latency.

Web PKI is already moving towards Merkle Tree Certificates (MTCs), which are primarily designed to address this problem of very long post-quantum signatures. However, not all systems will be able to use MTCs, so, in this blog post, I wanted to explore what else we can do. There are a bunch of things, actually.

We have two primary engineering levers: one is to reduce the physical size of the TLS handshake, and the other is to reduce how often a full handshake happens in the first place.

Here is a practical look at how we can optimize both—and where the industry stands on standardizing these fixes. The best part is that you will benefit from these optimizations today, even with traditional cryptography (RSA and/or Elliptic Curve).

Shrinking the Wire Payload

When it comes to TLS handshake options, we have several options:

  • Remove root certificates from the chains: One of the most common configuration errors is transmitting the root CA certificate alongside intermediate certificates. Clients already store trust roots locally; sending them over the wire serves zero functional purpose and wastes precious bandwidth.
  • Deploy certificate compression (RFC 8879): RFC 8879 allows endpoints to negotiate standard compression algorithms (Brotli, zlib, or zstd) for certificate payloads. In practice, this compresses bulky certificate chains by 30% to 50%. Although support is fragmented in modern browsers (Chrome supports Brotli; Firefox supports Brotli, zlib, and zstd; and Safari supports zlib), OpenSSL supports all three, so this technique should yield some savings. Sadly, compression works only for the text contained within certificates. By design, signatures and public keys don't compress well.
  • Modernize Public Keys (ECC over RSA): If you haven't already migrated off legacy RSA certificates, do so. Moving from RSA-2048 to ECDSA immediately reduces public key and signature sizes, yielding smaller certificate payloads before PQC arrives.
  • Raw Public Keys (RFC 7250): For machine-to-machine communication, APIs, or constrained environments where endpoints already trust or out-of-band cache the server's public key, RFC 7250 lets you skip transmitting intermediate CAs and certificate envelopes entirely.

What about experimental IETF proposals?

The IETF has explored several other compression and reduction mechanics, though adoption varies wildly:

  • TLS Cached Info (RFC 7924): Designed to let clients signal cached certificates to avoid re-transmission. This RFC was published in May 2026, but it's not yet widely supported by libraries and user agents. Perhaps it will become better supported as we start to migrate to post-quantum cryptography.
  • Abridged Compression for WebPKI Certificates (draft-ietf-tls-cert-abridge): Uses pre-distributed static dictionaries to compress certificate structures. Although adopted by the TLS Working Group, it's since expired and looks dead.

Reducing Full TLS Handshake Frequency

In TLS, we have two types of handshakes: full and abbreviated (resumed). The latter don't carry any certificates, which means they represent a significant saving:

  • Session Resumption: Utilizing session resumption allows returning clients to authenticate via a symmetric pre-shared key, entirely skipping the heavy asymmetric key exchange and certificate transfer.
  • TLS 1.3 0-RTT (Early Data): For repeat visitors, 0-RTT allows application data to travel in the very first flight of packets alongside the resumption handshake, removing handshake round-trip delays altogether. This technique doesn't provide full forward secrecy and should only be used when the first user-agent flight doesn't contain sensitive information.

Avoiding Handshakes Altogether

The fastest handshake is the one you never have to make. Eliminating asymmetric exchanges avoids the growing size penalty of post-quantum keys.

  • Tune Keep-Alives: Don't let idle TCP or TLS connections die prematurely. Extending application-level and TCP Keep-Alive timeouts on edge proxies (e.g., NGINX, HAProxy, Envoy) keeps sockets warm for bursty or sequential client traffic.
  • Deploy HTTP/2 and HTTP/3 (QUIC): By multiplexing multiple HTTP requests over a single underlying connection, modern protocols eliminate the wasteful habit of opening parallel TLS sessions for assets living on the same origin.

Don't Forget Outbound Connections: The Role of Forward Proxies

When optimizing TLS performance, the conversation almost universally focuses on inbound (frontend) traffic—how clients connect to your edge servers. But for modern microservice architectures, enterprise networks, and cloud-native applications, outbound (egress) traffic represents a massive hidden latency penalty.

Every time an internal service initiates an outbound API call, fetches a third-party dependency, or communicates across multi-region clusters, it triggers a fresh TLS handshake.

When post-quantum overhead is factored into these internal microservice meshes, the accumulated latency grows rapidly. To mitigate this on the egress side:

  • Deploy forward proxies with connection pooling: Placing an outbound forward proxy (or service mesh sidecar like Envoy) between your internal microservices and external endpoints allows you to consolidate and pool connections. Instead of every ephemeral application thread spinning up its own post-quantum TLS handshake to an external endpoint, the forward proxy maintains a pool of persistent, long-lived TLS connections.
  • Egress keep-Alives & resumption: Similar to frontend proxies, tuning aggressive session resumption and keep-alive settings on outbound proxy infrastructure ensures internal applications benefit from low-latency, warm connections.

By optimizing both frontend delivery and egress infrastructure, you shield your end users and internal architecture from the inescapable byte inflation of post-quantum cryptography.

Ivan Ristic
Ivan Ristic
Chief Scientist

Ivan Ristic is the Chief Scientist for Red Sift and former founder of Hardenize. Learn more about how Red Sift helps organizations with their Certificate Monitoring.