Secure Socket Tunneling Protocol (SSTP) is a VPN tunneling protocol introduced by Microsoft that encapsulates PPP traffic over an SSL/TLS session. Because SSTP commonly operates on TCP port 443, the same port used by HTTPS, it poses a challenge for network operators and security teams who need to differentiate legitimate web traffic from VPN tunnels. This article dives into the technical behavior of SSTP-over-443, techniques for traffic analysis and detection, and practical approaches you can adopt to identify SSTP tunnels while minimizing false positives.

Background: How SSTP Works over TCP 443

SSTP establishes a tunnel by wrapping PPP frames inside an HTTPS-like SSL/TLS connection. The protocol sequence is:

  • TCP three-way handshake to destination port 443
  • SSL/TLS ClientHello and ServerHello handshake
  • Once TLS is established, SSTP frames are exchanged using a simple framing layer (SSTP packet headers and control messages)
  • PPP negotiation (LCP/CHAP/IPCP/IPv6CP) and then IP packet flow over the PPP session

Because SSTP runs over TLS, it appears superficially similar to HTTPS. However, the post-handshake payloads follow SSTP framing (message types, fragmentation fields, control codes) rather than HTTP semantics.

SSTP Framing Details

SSTP messages include a 4-byte header: a 1-byte version field (usually 1), a 1-byte reserved, and a 2-byte length. Control messages (e.g., CONNECT, DISCONNECT, KEEPALIVE) have defined types. Understanding these byte patterns is key to detecting SSTP in decrypted or partially observable traffic.

Observable Network-Level Differences from HTTPS

Even without decrypting TLS, SSTP flows often leave telltale signs at the network and transport layers. Detection strategies typically combine multiple signals to reduce false positives.

1. Flow Features

  • Long-lived TCP connections: SSTP VPNs commonly maintain persistent connections for hours, whereas many HTTPS sessions are short-lived. However, modern web apps also keep connections alive, so this is not definitive on its own.
  • Byte distribution and payload entropy: Encapsulated IP packets tend to produce higher and more uniform entropy than text-based HTTP. Measuring per-packet entropy or use of compression can provide hints.
  • Packet sizes and MTU behavior: SSTP tends to carry full-sized IP packets fragmented into a stream of TCP segments. This yields distinctive size distributions: many full-MTU-sized TLS records or consistent chunk sizes matching the encapsulated IP MTU.
  • Inter-packet timing and throughput: User data flows often show different burst patterns compared to web browsing (e.g., continuous upstream/downstream throughput for file transfers or streaming over VPN).

2. TLS Handshake Characteristics

TLS handshakes are rich sources of metadata. The ClientHello and ServerHello offer multiple features you can fingerprint:

  • Cipher suites and extension lists: SSTP client stacks (especially Windows SSTP VPN clients) default to particular cipher suite orders and extension combinations. Comparing ClientHello JA3 fingerprints against known SSTP client fingerprints can be effective.
  • SNI and ALPN fields: Many SSTP clients do not include a meaningful Server Name Indication (SNI) or set ALPN to values typical of HTTP/2 (h2) or HTTP/1.1. Missing or empty SNI is suspicious for HTTPS traffic but common for SSTP.
  • Certificates: SSTP servers often present certificates tied to VPN infrastructure and may reuse certificates across multiple connections. Certificate subject fields, validity durations, and issuer CAs can help classification.

3. TCP-Level Signatures

  • Initial window sizes and TCP options: The TCP options sequence (e.g., MSS, SACK, Timestamps, Window Scale) produced by Windows clients in SSTP mode can be correlated with known SSTP client stacks.
  • Keepalive patterns: SSTP control messages include keepalive semantics which cause regular small TLS records; monitoring for such periodic records can support detection.

Deep Packet Inspection (DPI) and TLS Decryption

When you can decrypt TLS (enterprise environments with consenting devices), DPI becomes straightforward. After TLS decryption you can parse SSTP headers and PPP frames to positively identify SSTP traffic.

Decrypted Traffic Signatures

  • Search for SSTP control message types and the 4-byte SSTP header sequence in the decrypted TCP payload.
  • Look for PPP negotiation packets: LCP, CHAP, IPCP packets include recognizable protocol fields.
  • Detect encapsulated IP packets by identifying Ethernet-less IP headers (0x45 for IPv4 version/header length at the start of PPP payloads).

Tools for Decryption and Inspection

  • Wireshark with TLS keys exported from clients (e.g., NSS/Chrome SSLKEYLOGFILE) to reconstruct SSTP payloads.
  • Intrusion detection systems like Suricata can inspect decrypted TLS streams if provided with keys.
  • Zeek (formerly Bro) scripts to parse post-TLS payloads and flag SSTP framing or PPP negotiations.

Heuristic and ML-Based Detection without Decryption

Given privacy and legal constraints, decryption is not always possible. Non-decrypt DPI, heuristic signatures, and machine learning models can be effective.

Heuristic Rules

  • Combine TLS fingerprint (JA3/JA3S) mismatches for regular HTTPS clients with persistent, symmetric throughput and periodic small TLS records to raise a detection score.
  • Flag TLS connections to endpoints that present certificates with VPN-related CNs or to IPs known for VPN hosting.
  • Monitor the absence of HTTP semantics post-handshake: if no HTTP headers appear after TLS is established within a short window, suspect non-HTTP protocols (including SSTP).

Machine Learning Approaches

Use flow-level features as input vectors: duration, packet sizes, bytes per second, inter-arrival times, TLS fingerprint vectors, and TCP option sets. Supervised models (random forests, gradient boosting) trained on labeled datasets can distinguish SSTP from HTTPS with reasonable accuracy. Important considerations:

  • Ensure representative training data across client OS variants and SSTP implementations.
  • Guard against concept drift—TLS 1.3 adoption and changes in client stacks will alter fingerprints over time.
  • Combine ML scores with deterministic rules to reduce false positives on critical infrastructure.

Signature Examples and Practical Detection Rules

Below are representative detection heuristics and example signatures you can adapt to Suricata, Zeek, or SIEM correlation rules. These are conceptual — adjust thresholds for your network.

Suricata / IDS Conceptual Rule

  • If JA3 fingerprint in client matches known Windows SSTP client fingerprint AND
    • TLS SNI is empty OR ALPN is not h2/http/1.1
    • Connection duration > 600 seconds AND average throughput > 50 KB/s for > 5 minutes

    THEN flag as probable SSTP.

Zeek Script Strategy

  • On new TLS sessions, record JA3/JA3S, SNI, ALPN, certificate CN/O, and TCP options. If no HTTP records are observed and a PPP/LCP signature appears in decrypted payloads, trigger an SSTP event. Otherwise, if heuristics match (JA3 + empty SNI + long-lived), raise an alert.

Evasion Techniques and Countermeasures

Attackers and privacy-minded users may try to bypass detection. Common evasions include:

  • TLS mimicry: modifying ClientHello to match common browsers (changing cipher order, extensions, SNI).
  • TLS 1.3 and TLS features: TLS 1.3 reduces observable handshake detail and can make JA3 less distinctive; also ECH (Encrypted Client Hello) hides SNI.
  • Obfuscation layers: wrapping SSTP inside another TLS with HTTP semantics (e.g., WebSocket over TLS) or using pluggable transports.

Countermeasures: use multi-signal detection (tls fingerprint + flow features), update fingerprint databases, and employ active probing (where permitted) to request HTTP resources on the same connection. Legal and ethical constraints must guide active measures.

False Positives, Privacy, and Operational Considerations

Be cautious: many legitimate services will resemble SSTP on various heuristics. Blindly blocking all suspicious connections risks disrupting business-critical applications. Best practices:

  • Use layered detection: combine signature, heuristic, and ML approaches and require multiple indicators before taking disruptive action.
  • Allowlist known good services: maintain allowlists for cloud providers and CDNs to reduce false positives.
  • Respect privacy and legal limits: ensure you have authorization before decrypting TLS or intercepting user traffic.

Operational Playbook for Detecting SSTP on TCP 443

Practical step-by-step approach:

  • Collect baseline telemetry: NetFlow/IPFIX, TLS metadata (JA3/JA3S), packet captures for representative flows.
  • Build a fingerprint corpus: catalog JA3s, certificate patterns, TCP options from known SSTP clients vs. normal browsers.
  • Create layered alerts: low-confidence heuristics (long-lived, empty SNI) feed into higher-confidence checks (JA3 match, certificate CN match).
  • When possible and authorized, decrypt suspect sessions to confirm SSTP framing and PPP negotiation.
  • Continuously validate and retrain ML models to adapt to client and protocol evolution.

Conclusion

Detecting SSTP VPN traffic on TCP port 443 requires a combination of protocol knowledge, TLS fingerprinting, flow analytics, and, where appropriate, decryption. There is no single silver-bullet signature — effective detection relies on aggregating multiple signals and maintaining updated fingerprints as client stacks evolve. Network operators should build rigorous, layered detection pipelines and apply caution to avoid interfering with legitimate services.

For implementation resources and VPN-related insights, visit Dedicated-IP-VPN at https://dedicated-ip-vpn.com/.