Secure SSTP deployments often rely on publicly issued TLS certificates, but self-signed certificates — when created and managed correctly — can provide a strong, flexible, and cost-effective alternative for private or enterprise VPNs. This article explains how to harden an SSTP VPN using self-signed certificates, with practical, technical steps, recommended cryptographic settings, and validation/testing techniques. The guidance targets system administrators, developers, and business operators deploying SSTP (Secure Socket Tunneling Protocol) on Windows RRAS or compatible servers.

Why use self-signed certificates for SSTP?

Self-signed certificates can be appropriate when you control the client fleet (enterprise laptops, managed devices) and want maximal control over certificate lifecycles, revocation, and trust anchors. Key advantages:

  • Full control over cryptographic algorithms, key lengths, and validity periods.
  • No third-party CA dependency or recurring costs.
  • Ability to implement strict certificate pinning across managed clients.

Note: For public-facing consumer services, public CA certificates are generally preferable because of ease of use and automatic trust without manual distribution.

High-level hardening principles

Before diving into commands and configuration, align on core security principles:

  • Use a private CA: Create an internal Certificate Authority (CA) to issue and revoke server and client certificates.
  • Prefer EAP-TLS over password-based auth: When possible, use certificate-based client authentication (EAP-TLS) instead of MS-CHAP v2 to eliminate credential-based attacks.
  • Enforce modern TLS and cipher suites: Disable TLS 1.0/1.1 and weak ciphers; enable TLS 1.2/1.3 and ECDHE for Perfect Forward Secrecy (PFS).
  • Short certificate lifetimes & revocation: Use shorter validity (e.g., 1 year or less) and implement CRL/OCSP for revocation checks.
  • Harden the server OS: Apply OS updates, restrict management ports, and use firewall rules and intrusion prevention tools.

Step-by-step: Build a private CA and sign certificates (OpenSSL)

The following OpenSSL examples are cross-platform and produce artifacts suitable for Windows SSTP/RRAS. Adjust file paths as needed.

Create the private CA

1) Generate a strong CA private key (RSA 4096 or ECDSA secp384r1):

RSA 4096:

openssl genpkey -algorithm RSA -out ca.key -pkeyopt rsa_keygen_bits:4096

ECDSA secp384r1 (shorter keys, strong):

openssl ecparam -genkey -name secp384r1 -noout -out ca.key

2) Create the CA self-signed certificate (validity: 5 years as an example):

openssl req -x509 -new -nodes -key ca.key -sha256 -days 1825 -out ca.crt -subj “/C=US/ST=State/L=City/O=Org/OU=IT/CN=My-Internal-CA”

Generate server certificate with SAN

SSTP requires the server certificate to match the FQDN clients connect to. Always include Subject Alternative Names (SANs).

1) Create server key and CSR config file (server.cnf has SAN entries):

server.cnf content (minimal):

  • [req]
  • distinguished_name = req_distinguished_name
  • req_extensions = v3_req
  • [req_distinguished_name]
  • [v3_req]
  • subjectAltName = @alt_names
  • [alt_names]
  • DNS.1 = vpn.example.local
  • DNS.2 = vpn.example.com

2) Generate key and CSR:

openssl genpkey -algorithm RSA -out server.key -pkeyopt rsa_keygen_bits:4096

openssl req -new -key server.key -out server.csr -subj “/C=US/ST=State/L=City/O=Org/OU=VPN/CN=vpn.example.com” -config server.cnf

3) Sign the CSR with the CA and include v3 extensions (1 year valid):

openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 365 -sha256 -extfile server.cnf -extensions v3_req

Export for Windows (PKCS#12)

Windows RRAS/IIS prefers a PFX for importing the private key and cert into the computer store.

openssl pkcs12 -export -out server.pfx -inkey server.key -in server.crt -certfile ca.crt -password pass:YourPfxPassword

Installing and configuring on Windows SSTP/RRAS

1) Import the PFX to the Local Computer’s Personal store via MMC Certificates snap-in. Ensure the private key is marked exportable only if required.

2) Bind the certificate to SSTP in RRAS: In Routing and Remote Access Manager, right-click server > Properties > Security > select the certificate matching your FQDN.

3) Configure authentication: In the same RRAS properties, under Security, prefer EAP for authentication and configure EAP-TLS for client certificate authentication. Disable MS-CHAP v2 if you exclusively rely on certs.

4) IIS & OCSP stapling: If you run IIS in front of RRAS for certificate stapling, enable OCSP stapling and configure the server to serve OCSP responses. RRAS alone may not support stapling, so consider fronting with IIS/ARR or a reverse proxy for stapling and advanced TLS features.

Hardening Windows TLS and Schannel

Use Group Policy, registry tweaks, or tools like IISCrypto to:

  • Disable TLS 1.0 and 1.1. Enable TLS 1.2 and 1.3 only.
  • Disable weak ciphers (RC4, 3DES) and export ciphers.
  • Prioritize ECDHE-ECDSA or ECDHE-RSA suites for PFS.
  • Set secure key exchange ordering via HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlSecurityProvidersSCHANNELCipherSuites and related registry values, or use Group Policy for cipher suite order.

Example registry changes should be tested in a staging environment; consider using Microsoft documentation or IISCrypto to apply recommended templates.

Client distribution and trust

For clients to trust the self-signed CA, you must deploy the CA certificate into the Trusted Root Certification Authorities store on all client devices. Methods:

  • Group Policy (AD deployments): deploy both server cert and CA cert, and configure EAP-TLS policies.
  • MDM solutions: push CA cert and PKCS#12 packages for managed devices.
  • Manual import for unmanaged endpoints (not recommended at scale).

Protect PKCS#12 files: Always encrypt exports with a strong passphrase and use a secure channel for distribution. Consider unique client certificates to allow per-device revocation.

Revocation and lifecycle management

Implement revocation via CRL or OCSP. For internal CAs, publish a CRL endpoint reachable by clients and issue CRLs frequently (daily or weekly depending on risk). If possible, configure OCSP responder for faster revocation checks and consider OCSP stapling on the server front-end.

Use short certificate lifetimes (e.g., 1 year for server, 90 days for clients if operationally feasible) to reduce exposure from key compromise. Automate renewal workflows via scripts or certificate management tools.

Testing and validation

Validate your configuration thoroughly:

  • Use openssl s_client: openssl s_client -connect vpn.example.com:443 -servername vpn.example.com -showcerts to inspect server cert chain and TLS handshake.
  • Run sslyze or testssl.sh to enumerate supported protocols and cipher suites and ensure TLS 1.2/1.3 and PFS suites are offered.
  • Use Nmap NSE scripts (ssl-enum-ciphers) to check client-visible cipher suite ordering.
  • Test client behavior: verify that clients with only the server cert but without CA trust fail, and that clients with CA trust succeed. Verify EAP-TLS client authentication and revoke a client cert to test CRL/OCSP enforcement.

Additional operational hardening

Consider these operational controls to strengthen the SSTP deployment:

  • Authentication layered approach: Use device certificates + machine group policies + 802.1X or conditional access for network segmentation.
  • Network controls: Limit administrative access, enable firewall rules to allow only necessary services, and restrict which IPs can access management interfaces.
  • Monitoring and logging: Enable detailed event logging for RRAS and network access, ingest logs into SIEM, and watch for repeated handshake failures or certificate errors.
  • Failover and redundancy: Use load balancers or reverse proxies that support TLS offload, OCSP stapling, and robust health checks while preserving client certificate passthrough when using EAP-TLS.

Common pitfalls and how to avoid them

Be mindful of these common mistakes:

  • Using a certificate without proper SAN entries — will cause client validation failures.
  • Leaving weak ciphers or old TLS versions enabled — creates downgrade attack surfaces.
  • Not distributing the CA cert securely — compromises trust integrity.
  • Relying solely on long-lived client certificates — complicates revocation and increases risk.

By following a rigorous approach — building a private CA, issuing SAN-aware server certificates, enforcing EAP-TLS where feasible, and tightening Schannel/TLS settings on the server — you can safely run an SSTP VPN based on self-signed certificates that meets enterprise security requirements. Regular testing, monitoring, and automated certificate lifecycle management will keep the environment resilient against modern threats.

For additional resources and tools to audit TLS configurations, see the OpenSSL documentation and community tools such as sslyze and testssl.sh. If you’re managing device fleets, integrate certificate deployment into your existing MDM or Group Policy workflows to keep distribution consistent and secure.

Published by Dedicated-IP-VPN — https://dedicated-ip-vpn.com/