IPsec combined with L2TP is a widely used VPN solution for remote access and site-to-site connectivity. One of the least visible yet most critical parts of maintaining a secure IPsec tunnel is rekeying — the process of periodically replacing cryptographic keys used to protect traffic. This article digs into what happens during L2TP/IPsec rekeying, what mechanisms drive it, how the protocol exchanges proceed, and practical considerations for sysadmins, developers, and network architects who operate or integrate these tunnels.

Fundamentals: Which Keys Are Replaced and Why

When we say “L2TP/IPsec rekeying” we are actually referring to two layers of cryptographic context:

  • IKE (ISAKMP) SA — the control channel that negotiates and protects keying material and management messages (IKE Phase 1 in IKEv1, IKE SA in IKEv2).
  • IPsec Child SAs (ESP or AH) — the data plane SAs that encrypt and authenticate payload traffic (what actually protects L2TP tunnels).

Keys are rotated for several reasons: to limit exposure if keys are compromised, to limit the amount of traffic encrypted under a single key (mitigates some cryptanalytic attacks), and to comply with policy or regulatory lifetimes. Rekey events are triggered by one or more of:

  • Time-based lifetimes (seconds)
  • Traffic-volume lifetimes (kilobytes/megabytes)
  • Manual triggers or administrative reconfiguration
  • Reauth events (e.g., re-entered credentials for L2TP sessions)

Where L2TP Fits in the Stack

L2TP provides the tunneling of PPP sessions. In the common deployment “L2TP over IPsec,” IPsec/ESP provides confidentiality and integrity for the L2TP packets. L2TP itself has its own control & session management, but it relies on IPsec for secure transport. Thus, IPsec rekeying must be transparent to the L2TP/PPP session if you want uninterrupted connectivity.

Typical Packet Flow

Application → PPP → L2TP → (IPsec ESP) → IP

During a rekey, the ESP layer replaces keys beneath L2TP; properly implemented systems perform rekey without breaking PPP sessions, by installing a new SA and switching traffic over while the old SA remains valid for a grace period.

IKEv1 vs IKEv2 Rekeying: Key Exchanges at a Glance

Rekey procedures differ between IKEv1 and IKEv2. Most modern deployments should prefer IKEv2 (RFC 7296) for robustness and clearer child SA handling, but many legacy L2TP/IPsec setups still use IKEv1.

IKEv1 (Main Mode + Quick Mode)

– Phase 1 (Main Mode): Establishes the IKE SA (ISAKMP SA). Rekey of Phase 1 entails creating a new ISAKMP SA which then provides fresh keying material for child SAs. A typical rekey uses Main Mode to negotiate new Diffie-Hellman and then establishes new keys.

– Phase 2 (Quick Mode): Negotiates the IPsec SAs (ESP). Rekeying a child SA is performed by a new Quick Mode exchange that performs a new Diffie-Hellman (if PFS is used) and derives new SK_ei/SK_er (encryption keys), SK_ai/SK_ar (integrity keys), and updates lifetimes.

IKEv2 (CREATE_CHILD_SA)

IKEv2 simplifies rekeying: both creation and rekey of child SAs are done via the CREATE_CHILD_SA exchange. The IKE SA itself can be rekeyed via an IKE_SA_REKEY exchange which creates a new IKE SA while preserving continuity. The CREATE_CHILD_SA supports rekey with or without PFS and cleanly handles SA lifetimes, sequence numbers, and the interplay between old and new child SAs.

Key Derivation and Material

At the heart of rekeying are the pseudo-random functions (PRFs), Diffie-Hellman (DH) operations, and sequential key derivation steps. A high-level summary:

  • DH exchange (if performed) yields a shared secret (g^xy).
  • A PRF (often HMAC-SHA1, HMAC-SHA256) derives SKEYSEED and then multiple keys: encryption keys, integrity keys, and keys for generating further keying material.
  • IKEv2 uses a single KDF (PRF+) to expand the seed into child SA keys (SK_d, SK_ai, SK_ar, SK_ei, SK_er, SK_pi, SK_pr depending on direction and role).

Perfect Forward Secrecy (PFS) means child SA rekey typically performs a fresh DH and new keys are independent of prior keys. If PFS is disabled, new keys may be derived without an extra DH, but PFS is recommended for security.

Practical Rekey Sequence (Typical IKEv2 CREATE_CHILD_SA)

1. Rekey timer or byte threshold reaches configured lifetime.

2. The initiator sends CREATE_CHILD_SA (with new SA payload proposing new encryption/integrity algorithms, new nonces, and optionally a new DH public value if PFS is used).

3. The responder authenticates the request and responds with chosen transforms and its own nonces and DH public value.

4. Both sides compute new keying material, install the new CHILD SA, and start using it for outbound traffic. The old SA is kept inbound for a short overlap window to drain in-flight packets.

5. Optionally, a DELETE(SA) message is issued to remove the old SA after the overlap period.

This overlap is crucial: packets in flight that were encrypted under the old SA can still be decrypted during the overlap, preventing session interruptions.

Stateful Details: Sequence Numbers, Anti-Replay, and NAT

Every ESP SA uses sequence numbers and anti-replay windows. Rekeying resets sequence numbers for the new SA. To prevent packet drops or replay-window confusion, implementations use overlapping lifetimes and carefully manage direction-specific sequence spaces.

NAT Traversal (NAT-T) complicates rekey because IKE messages can be encapsulated in UDP/4500 and peer NAT mappings may time out. NAT-T keeps an established UDP mapping by sending keepalives, and rekey exchanges may revalidate NAT detection and re-establish UDP encapsulation parameters. If a NAT mapping expires mid-rekey, the exchange can fail; robust implementations retry and support rapid re-establishment.

Common Problems During Rekey and How to Mitigate Them

  • Traffic interruption: Occurs if new SA isn’t installed atomically or overlap window is zero. Mitigation: enable overlap/time-based soft rekey and ensure correct behavior in both endpoints.
  • Packet loss in high-latency or lossy links causing rekey messages dropped. Mitigation: retransmit timers, retry policy, and larger retransmit windows.
  • MTU/fragmentation: IKE rekey messages can be large (DH public values + certificates). If UDP fragmentation occurs through middleboxes, rekey may fail. Mitigation: lower DH group size (carefully), enable jumbo MTU path MTU discovery or use certificate-less PSK setups where feasible.
  • Sequence number wrap: High-volume tunnels may hit byte or sequence thresholds. Configure lifetimes based on expected throughput and use PFS to limit exposure.
  • NAT timeouts: NAT translation may drop the UDP mapping used for IKE. Ensure keepalive frequency keeps NATs alive during long sessions.

Operational Recommendations

For operators managing L2TP/IPsec for business or multi-tenant scenarios, consider the following best practices:

  • Use IKEv2 where possible — simpler state machine, better resilience, and more explicit rekey semantics.
  • Configure sane SA lifetimes — common defaults: IKE SA = 86400s (24h), Child SA = 3600s (1h), but adjust based on traffic patterns; shorter lifetimes increase rekey frequency and CPU cost.
  • Enable PFS for sensitive tunnels; note it adds CPU costs due to extra DH computations.
  • Plan for overlap — ensure your endpoints keep old SAs alive long enough to drain in-flight traffic.
  • Monitor rekey events — collect logs of CREATE_CHILD_SA/Quick Mode messages, failed rekeys, and retransmits to detect issues early.
  • Test under load — simulate heavy flows and NAT scenarios to observe rekey behavior before deploying to production.

Development and Integration Considerations

Developers building VPN clients or integrating L2TP/IPsec into applications should be aware of these points:

  • Expose configurable lifetimes and PFS toggles in client UI or configuration APIs so administrators can tune for their environment.
  • Handle rekey transparently: queue outgoing packets until new SA is installed, continue accepting inbound packets on old SA during overlap, and avoid resetting L2TP/PPP sessions on rekey.
  • Provide comprehensive logging for the key lifecycle: proposals, nonces, DH groups, chosen transforms, and rekey failures.
  • Implement robust retransmission and backoff policies for rekey exchanges; don’t assume a single attempt will succeed across diverse networks.

Summary and Key Takeaways

Rekeying in L2TP/IPsec is a coordinated dance between control plane (IKE) and data plane (ESP) that replaces cryptographic keys without disrupting tunneled sessions. The principal mechanisms include timed or volume-based lifetimes, Diffie-Hellman and PRF-based key derivation, and graceful overlap of old and new SAs. Practical concerns such as NAT traversal, fragmentation, and CPU costs for PFS need to be balanced against security requirements.

By aligning SA lifetimes, enabling overlap windows, monitoring rekey events, and preferring modern protocol implementations (IKEv2), administrators and developers can ensure secure, reliable rekey behavior that remains transparent to L2TP/PPP sessions.

For more practical guides and VPN configuration tips, visit Dedicated-IP-VPN at https://dedicated-ip-vpn.com/.