Understanding the IKEv2 negotiation process is essential for network engineers, developers building VPN clients/servers, and administrators managing secure site-to-site or remote-access VPNs. This article walks through the protocol’s logical steps and technical details, from initial packet exchange to key derivation, Child SAs, rekeying, and operational features like NAT traversal and MOBIKE. The goal is to demystify the messages, payloads, and cryptographic operations so you can design, troubleshoot, or optimize IKEv2 deployments.

Overview of IKEv2 Security Associations

IKEv2 (Internet Key Exchange version 2) establishes two types of Security Associations (SAs): the IKE SA and one or more Child SAs. The IKE SA provides a secure, authenticated channel used to manage Child SAs, rekeying, and to carry control messages. Child SAs carry the actual user traffic (ESP or AH). Understanding the separation between control (IKE SA) and data (Child SA) is critical for architecture and troubleshooting.

High-Level Negotiation Steps

At a high level the negotiation follows these steps:

  • IKE_SA_INIT — exchange of cryptographic parameters, nonces, and Diffie-Hellman (DH) values to establish shared secrets and algorithm agreement.
  • IKE_AUTH — authentication of peers and establishment of the first Child SA (if not using EAP-only flows).
  • CREATE_CHILD_SA — creation and rekeying of Child SAs after the IKE SA is established.
  • INFORMATIONAL — error messages, notifications, delete messages, and keepalives (Dead Peer Detection).

Why two phases?

IKE_SA_INIT builds the cryptographic foundation (shared secret and negotiated transforms) without authenticating the peers, which helps mitigate some DoS vectors. IKE_AUTH then authenticates and uses the secure channel from IKE_SA_INIT to protect identity and credentials.

IKE_SA_INIT in Detail

The IKE_SA_INIT exchange consists of two messages (from Initiator and Responder) and carries payloads such as SA, KE (DH public value), and Nonce (Ni, Nr). Important points:

  • SA payload: lists proposals (encryption, PRF, integrity/authentication, DH groups, ESN, PFS options). The peers negotiate the first matching proposal.
  • KE payload: contains the Diffie-Hellman public value (g^x). Common groups: 14 (2048-bit MODP), 19 (256-bit ECP), 21 (521-bit ECP), and newer ones like Curve25519.
  • Nonce: random nonces Ni and Nr provide freshness and feed into key derivation (prevent replay and provide entropy).

Once both KE and Nonces are exchanged, the two peers compute the Diffie-Hellman shared secret (g^xy). This is combined with Ni and Nr to derive the IKEv2 seed secret (SKEYSEED) and subsequent keys via the negotiated PRF.

Key derivation basics

Classic IKEv2 key derivation steps (simplified):

  • SKEYSEED = prf(Ni | Nr, g^xy)
  • From SKEYSEED derive SK_d (for child SAs), SK_ai/SK_ar (integrity keys for IKE AUTH), SK_ei/SK_er (encryption keys for IKE messages), SK_pi/SK_pr (for AUTH payload signatures/MACs).

The exact sequence uses the negotiated PRF and expands the seed into keys. Implementations must follow the RFC-specified ordering for interoperability.

IKE_AUTH: Authentication and ID Exchange

With keys derived from IKE_SA_INIT, IKE_AUTH authenticates the peers and exchanges identification and optional certificates. The responses are protected using the keys established.

  • IDi/IDr payloads: contain identity information (IP address, FQDN, ASN.1 DN, or Key ID), used to match local policies.
  • AUTH payload: contains authentication data, which depends on the method—pre-shared key (PSK), RSA signatures, or EAP-derived credentials.
  • CERT/CERTREQ: used in certificate-based authentication; cert chains can be transferred or indicated via hash.
  • Encrypted payloads: IKE_AUTH messages are encrypted and integrity protected using SK_ei/SK_ai (or the reverse on the responder).

Successful completion results in an authenticated IKE SA and commonly the automatic creation of the first Child SA (ESP) carrying TSi/TSr (traffic selectors) telling which traffic to protect.

Child SA Creation and Rekeying

Child SAs are negotiated using CREATE_CHILD_SA exchanges. These can: create an initial Child SA inside IKE_AUTH, rekey an existing Child SA, or rekey the IKE SA itself.

  • Traffic Selectors (TSi/TSr): define the IP subnets/ports for the Child SA. Mismatched selectors are a common interoperability issue.
  • PFS (Perfect Forward Secrecy): when requested, a new DH exchange is performed to produce fresh keys for the Child SA, preventing compromise of long-term keys from exposing past traffic.
  • Key Update: Child SA rekeying can be initiated by either peer when lifetimes approach expiration. Rekey may use existing IKE SA keys or perform PFS as negotiated.

Lifecycle parameters include soft and hard lifetimes (kilobytes of traffic or seconds). Implementations typically initiate rekey before hard lifetime expiry to avoid service interruption.

Authentication Methods and EAP Integration

IKEv2 supports multiple authentication mechanisms:

  • Pre-Shared Keys (PSK): simple but less scalable and prone to credential leakage across multiple endpoints.
  • Certificate-based (RSA/ECDSA): uses PKI; recommended for site-to-site and enterprise environments.
  • EAP (Extensible Authentication Protocol): used for flexible remote-access flows (e.g., EAP-TLS, EAP-MSCHAPv2). IKE_AUTH can carry an EAP-Response/Request sequence to authenticate users.

EAP in IKEv2 necessitates careful handling: the IKE SA is used to protect EAP exchanges, and EAP may culminate in keying material (e.g., when using EAP for authentication-only vs. EAP-based keying). Implementations must track state transitions between EAP and IKE-authenticated phases.

Operational Considerations: NAT Traversal, Cookies, and Fragmentation

Practical deployments face middleboxes and unreliable networks; IKEv2 includes features to handle these:

  • NAT traversal (NAT-T): when NAT devices alter UDP/IP headers, ESP packets cannot be transported directly. IKEv2 uses UDP encapsulation (UDP/4500) for ESP and adds NAT-D payloads to detect whether NAT exists and to compute NAT-specific ports/addresses.
  • Cookies (CPR and NAT-D Cookies): to mitigate UDP-based DoS, responders may send a COOKIE notification requiring initiators to prove reachability/uniqueness before expensive crypto operations.
  • Fragmentation: large IKE messages (certificates, long proposal lists) can be fragmented using IKEv2 fragmentation; if not supported, large payloads can cause failures—monitor and enable fragmentation where necessary.

Dead Peer Detection and Keepalives

IKEv2 supports DPD via INFORMATIONAL messages (CPING and R_U_THERE) to verify peer liveness. Implementations typically complement DPD with monitoring and connection timeouts within administrative policies.

Mobility and Multi-Homing: MOBIKE

MOBIKE is an extension that allows an IKE SA to survive endpoint IP address changes (e.g., moving from Wi‑Fi to LTE). Key aspects:

  • Endpoint can notify peer of new IP/port pairs and negotiate which flow to use.
  • Requires both peers to support MOBIKE and to maintain an address list for the SA.
  • Preserves the IKE SA and Child SAs while rekeying may be performed as necessary for security policy.

Error Handling, Notifications, and Debugging Tips

IKEv2 uses Notification payloads to report errors and status. Some common notifications include:

  • INVALID_SYNTAX, NO_PROPOSAL_CHOSEN (mismatched transforms)
  • AUTHENTICATION_FAILED, CERT_UNTRUSTED
  • STATUS_CHANGE and COOKIE notifications for NAT/DoS handling

Debugging tips:

  • Log both IKE and ESP packet flows and look for mismatched proposals or ID types. Many failures occur because the ID format (e.g., FQDN vs. IP) doesn’t match policy.
  • Check NAT-T detection and whether UDP/4500 is used; mismatched NAT handling often causes one-way traffic.
  • Certificate chain problems: ensure correct CA, intermediate certificates, and matching SubjectAltName or DistinguishedName fields used as ID payloads.
  • Use packet captures to inspect IKE payloads: SA, KE, Nonces, and Notifications quickly reveal negotiation disagreements.

Security Best Practices

To maintain strong security posture:

  • Prefer certificate-based authentication or EAP-TLS for remote-access to avoid fragile PSK management.
  • Choose strong transforms: modern AEAD ciphers (e.g., AES-GCM) and ECDH groups (Curve25519, P-256) reduce attack surface and improve performance.
  • Enable PFS on Child SAs where long-term confidentiality is required.
  • Monitor lifetimes and rekey events to avoid downtime—ensure soft lifetimes trigger rekeying early enough.
  • Harden IKE endpoints with rate-limiting, cookie checks, and minimal exposure to the public Internet where possible.

Putting It All Together: Typical Message Flow

Here is a concise sequence you will see in a successful IKEv2 connection:

  • Initiator -> Responder: IKE_SA_INIT (SA, KE, Ni)
  • Responder -> Initiator: IKE_SA_INIT (SA chosen, KE, Nr) — both compute g^xy and SKEYSEED
  • Initiator -> Responder: IKE_AUTH (IDi, AUTH, SA for Child SA, TSi)
  • Responder -> Initiator: IKE_AUTH (IDr, AUTH, TSi/TSr, TSr) — encrypted/authenticated
  • CREATE_CHILD_SA exchanges as needed for additional Child SAs or rekey operations
  • INFORMATIONAL messages for DPD, deletes, or notifications

At each encrypted step, ensure the correct keys (SK_e, SK_a, SK_p*) are used, and verify nonces and cookies if present. Address mismatches or transform rejections by comparing SA payloads and local policy.

Conclusion

IKEv2 provides a robust, flexible framework for secure key exchange and SA management. The separation of IKE SA and Child SAs, well-defined payloads, and support for modern cryptography make it suitable for enterprise-grade VPNs. However, operational success depends on careful configuration of transforms, IDs, NAT handling, and authentication methods. For network operators and developers, mastering the IKE_SA_INIT and IKE_AUTH exchanges, understanding key derivation, and recognizing common failure modes are essential skills.

For more guides and practical configuration tips, visit Dedicated-IP-VPN at https://dedicated-ip-vpn.com/