Introduction

The rapid rise in encrypted traffic is transforming the threat landscape. As of August 2020, 95% of Google's traffic is encrypted, and nearly 85% of the webpages loaded by Firefox are encrypted. Based on an earlier Gartner estimate from 2019, more than 80% of enterprise web traffic was encrypted, and a recent Gartner report states that 54% of network threats are found in encrypted traffic.

Visibility into encrypted traffic is critical for effective network security. However, TLS decryption is a resource-intensive process and, if improperly configured, could potentially allow attacks and even degrade the end-user experience. This document covers the best practices for configuring TLS inspection on Secure Firewall.

📘

Note

Before we go any further, you may have already noticed that TLS and SSL are often used interchangeably. We use TLS/SSL to indicate that either protocol is being discussed. The IETF has deprecated the SSL protocol in favor of the more secure TLS protocol, so you can usually interpret TLS/SSL as referring to TLS only.

For more information about SSL and TLS protocols, visit SSL vs. TLS - What's the Difference?

Decryption Policy

The Decryption Policy governs how the Secure Firewall Threat Defense handles encrypted traffic. Visibility into TLS encrypted traffic provides better information for IPS inspection, File and Malware detection, and micro application visibility. Apart from inspecting flows, you can use the TLS/SSL policies to block server connections supporting older TLS/SSL versions or weak ciphers.

If configured, Decryption Policy evaluation occurs before an Intrusion or File Policy, as illustrated here:

Decryption Rule Evaluation

The Decryption rules are evaluated top-down. Based on the fields used to define the Decryption rule, some flows might match more than one rule, this is called an "ambiguous match." Figure 2 below shows the various fields available to configure a Decryption rule match. The Decryption rule's complexity determines the resources used to establish the policy match's verdict for a flow.

Decryption Policy Rule Actions

TLS Decryption Deployment Modes

1. A Self-Signed Certificate using the FMC as the Root CA
2. Use the Internal Organization CA to sign the FMC certificate
3. Import an Internal CA certificate

👍

Best Practice

Using a self-signed certificate to resign traffic is not recommended as the users' browsers will not trust it. This will result in browser certificate errors for decrypted sessions.

Configuration

You can configure multiple Decryption policies. However, only one Decryption policy is associated with an access control policy, which can be associated with one or more Secure Firewall devices. As explained earlier, Decryption rules require various degrees of information to match a flow and reach a verdict. The recommended Decryption policy configuration is comprised of three sections:

Global Settings ( Trusted CA Certificates and Undecryptable Actions )

Trusted CA Certificates

The Decryption policy includes a wide range of trusted CA root and intermediate certificates issued by third parties. When creating a Decryption policy, a trusted CA group object called "Cisco-Trusted-Authorities" is added by default. These are needed when configuring "Decrypt-Resign" rules using the certificate status of the external server.

To add a Trusted CA, we recommended uploading all certificates within a root CA's chain of trust to the Trusted CA certificates. A requirement to detect certificates issued by intermediate CAs.

Undecryptable Actions

These settings configure FTD to handle an undecryptable flow.

Default Action ( Default Decryption Policy Action and Logging )

When creating a new Decryption policy, the default action for traffic that does not match any Decryption rules is Do Not Decrypt. For most cases, this is the recommended setting. However, to block all TLS connections that do not match a decryption rule, set the Default Action to Block/Block with Reset.

Decryption Rule Recommendations

Rule Ordering

The recommended order of configuring Decryption rules is to arrange the rules such that the rules that require the least amount of information to determine the outcome are at the top. Also, bear in mind the need to only decrypt necessary traffic. Therefore place the Block and Do Not Decrypt rules before the Decrypt Known Key and Decrypt Resign rules.

Protection from 'weak' TLS Servers

Block Traffic to sites using weak TLS ciphers, older TLS versions or problematic certificates (e.g. invalid, self-signed).

Handling TLS 1.3 Servers

TLS 1.3 Introduction

TLS 1.3 is the latest release of the Transport Layer Security (TLS), providing significant security and efficiency improvements over its predecessor TLS 1.2. Since the official release in August 2018, the TLS 1.3 protocol has been widely adopted by the Internet community reaching over 60% of the 1 million top websites by the end of 2021.

The following is the list of the significant improvements introduced in the TLS 1.3 protocol:

• TLS 1.3 shortens connection setup time by reducing the handshake by one round trip time for new connections compared to TLS 1.2. The benefit of the round trip reduction particularly benefits scenarios when the client is connecting over high latency links like 2G or satellite.
• For Pre-Shared Key (PSK) resumed connections, TLS 1.3 introduces a new zero round-trip time (0-RTT) mode, allowing a client to send encrypted application data (e.g., HTTP GET) in the initial Client Hello packet. 0-RTT speeds application response time at the cost of sacrificing forward secrecy and non-replay protection of the request sent in the initial message.
• TLS 1.3 improves privacy by:
  • Complete forward secrecy due to removing static RSA and Diffie-Hellman cipher suites. Perfect forward secrecy ensures that future RSA private key compromises would not allow an attacker to decrypt captured packets.
  • Encrypting all handshake messages after the TLS Server Hello, including server certificate and the majority of server extensions.
• TLS 1.3 discontinued a set of unused and unsafe features, including the removal of compression, static RSA/DH cipher suites, Digital Signature Algorithms, and custom Ephemeral Diffie-Hellman groups.
• The list of TLS 1.3 supported symmetric encryption algorithms has been limited to five Authenticated Encryption with Associated Data (AEAD) algorithms, with AES_128_GCM_SHA256 and AES_256_GCM_SHA384 becomming the most widely adopted.
• Improved state machine with more consistency and removal of needless messages. For example, all session resumption flows of earlier TLS versions are replaced by a single new PSK exchange.

For more details please refer to the RFC 8446 - The Transport Layer Security (TLS) Protocol Version 1.3.

The Affect of TLS 1.3 on the Decryption Policy

The TLS 1.3 protocol enhancements vastly increase the security of the protected session. However, compared to previous versions of TLS, they introduce significant implications for firewalls performing decryption and deep payload inspection. This section outlines the principles of the Decryption policy for both TLS 1.2 and 1.3.

As discussed in the Decryption Rule Evaluation section, depending on the conditions configured in the Decryption decryption policy, the firewall might need visibility into attributes within the Client Hello, Server Hello, and server’s Certificate messages to make a decrypt/do-not-decrypt verdict:

• The Server Name Indication (SNI) in the Client Hello message is used for the initial Application/URL Category and Reputation-based match.
• TLS session settings, chosen by the server, are extracted from the Server Hello message and used to evaluate the version and cipher-based conditions.
• The Certificate message is required for certificate-related conditions and to ensure the client didn’t spoof the SNI.

With TLS 1.2 protocol, depicted in Figure 18, the Client Hello, Server Hello, and Certificate handshake messages are transmitted in clear text. The key exchange begins after the TLS server shares its certificate with the client, allowing the firewall to inspect most of the handshake messages without the need to decrypt.

Refer to The Illustrated TLS 1.2 Connection for a very detailed, packet-by-packet and byte-after-byte example of a TLS 1.2 handshake.

For the TLS 1.3 handshake, the key exchange starts with the initial packet sent from the client, as illustrated in Figure 19. The client provides key share information upfront to allow the server to calculate the shared keying material. In effect, most handshake messages, including the server’s certificate, are encrypted and not available to the firewall for Decryption policy processing.

The client initiates the TLS 1.3 session with a Client Hello message that contains key share and cryptographic parameters along with the SNI extension. The server processes the Client Hello and determines the appropriate cryptographic parameters for the connection. It then responds with a Server Hello, containing the negotiated connection parameters and the server’s key share.
All handshake messages, including server extensions and the certificate, are encrypted after this phase.

Refer to The Illustrated TLS 1.3 Connection for a very detailed, packet-by-packet and byte-after-byte example of a TLS 1.3 handshake.

The only moment in the TLS 1.3 session lifetime when the firewall can slip in between the client and the server as a Man-in-the-Middle (MITM) proxy is when the firewall receives the Client Hello. At this time, the firewall must decide to decrypt/do-not-decrypt before forwarding the Client Hello message to the server.

• If the decision is to decrypt, the firewall has to replace the key-share in the Client Hello with its own cryptographic material. From then on, the firewall maintains two encrypted connections in parallel, one towards each client and the server.
• If the firewall decides not to decrypt, the Client Hello packet is forwarded unmodified. The client and server continue negotiating a secure connection, opaque to the firewall.

📘

Note

Once the firewall decides to modify the Client Hello message, it cannot back off from being the MITM without breaking the connection altogether. There is no way to stitch back the TLS session between the client and server. The firewall may either continue decryption or deactivate the SSL proxy, causing the session to fail.

Making a decrypt/do-not-decrypt decision at the Client Hello stage is not optimal. At this time, the firewall only has the SNI information that the client could easily spoof. The firewall must review the server's certificate to ensure it matches the SNI and make a reliable decision. The challenge with TLS 1.3 is that the firewall has to decrypt the session to learn the server’s certificate, which cannot happen until the actual decrypt/do-not-decrypt verdict.

👍

Best Practice

Cisco Secure Firewall Threat Defense provides an elegant solution to this “chicken and egg” dilemma with the TLS Server Identity Discovery feature described in the following section.

TLS Server Identity Discovery

TLS Server Identity Discovery is a feature introduced in the Firepower 6.7 release to provide TLS 1.3 server’s unencrypted certificate information before a TLS session is established through a firewall. TLS Server Identity Discovery enables a firewall with the capability to open a “sidecar” TLS 1.2 connection to download and cache TLS servers’ certificates. TLS Server Identity Discovery shares the certificate information with the Access Control and Decryption policies to increase the reliability of the decision process.

Figure 20 depicts a sample TLS 1.3 handshake flow with the TLS Server Identity Discovery feature enabled with no server certificate in the cache. The steps are as follows:

1. The client initiates the TLS 1.3 connection with a Client Hello. The firewall intercepts the packet and reads the SNI extension. The destination IP address, along with the value of the SNI, is used to search the cache for a corresponding certificate for Decryption Policy rule processing.
2. The certificate for the client’s SNI is unavailable in the cache, and the firewall pauses the TLS 1.3 connection. The firewall initiates a TLS 1.2 “probe” connection request to the target server to obtain its clear text certificate. The firewall terminates the “probe” connection once it receives the certificate from the server.
3. When the TLS Server Identity Discovery engine obtains the server’s certificate using the probe, it installs the certificate in the cache along with the server’s IP address and the SNI value from the original TLS 1.3 Client Hello.
4. With the cached certificate, the firewall now has enough data to process the Decryption Policy and give a decrypt/do-not-decrypt verdict.
5. The firewall resumes the original TLS 1.3 connection and continues with the handshake. If the verdict was to decrypt, the firewall will modify the Client Hello message and become a MITM proxy for the connection.

📘

Note

The Server Certificate Cache is available to all Snort instances running on a firewall appliance. The firewall does not preserve the cached content across reboots.

Figure 21 illustrates the case of a subsequent TLS 1.3 session towards a server whose certificate is available in the Server Certificate Cache on the firewall.

The steps are as follows:

1. The client initiates the TLS 1.3 connection with a Client Hello. The firewall intercepts the packet and reads the SNI extension. A certificate is matched in the cache using SAN and the IP address of the destination server.
2. The firewall evaluates Decryption Policy rules and gives a decrypt/do-not-decrypt verdict.
3. Depending on the verdict, the firewall passes the TLS 1.3 connection unchanged or enables the MITM proxy.

⚠

️ Warning

If TLS Server Identity Discovery is disabled, the TLS 1.3 decryption still works using the SNI from the Client Hello. However, the firewall is more likely to perform suboptimal Decryption policy decisions without certificate visibility.

Enabling TLS Server Identity Discovery

To enable TLS Server Identity Discovery, open the advanced settings of your Access Control Policy and ensure the Early application detection and URL categorization option is enabled as per Figure 22.

👍

Best Practice

Cisco recommends enabling TLS Certificate Visibility along with TLS 1.3 decryption.

When TLS 1.3 inspection is enabled, the TLS Server Identity Discovery performance impact is expected to be minimal. The certificate visibility's benefit to the decryption engine evens out the additional CPU cycles spent on probing.

By default, the TLS Server Identity Discovery probe connections are not recorded by the firewalls. You can enable logging the probe connection events with the following CLI command on your individual firewalls:

system support ssl-certificate-visibility-probe-logging true

Once enabled, you will start seeing probe connections in your logs. You can recognize a probe connection by a Reason “TLS Probe” and the fact that they are triggered shortly after the Client Hello packet of the initial connection. Figure 23 provides an example of a TLS connection followed by a probe.

Enabling TLS 1.3 Decryption

Starting from the 7.2 software release, you can enable TLS 1.3 decryption in the Advanced Options of the Decryption Policy, as depicted in Figure 24.

⚠

️ Warning

If you choose not to enable TLS 1.3 decryption, the firewall by default modifies the TLS Client Hello message to negotiate the TLS1.2 connection. This is called Aggressive Downgrade and is controlled by a CLI setting:

system support ssl-client-hello-enabled aggressive_tls13_downgrade { true | false }

As some browsers and servers enforce TLS 1.3, the Aggressive Downgrade may result in errors in some cases.

If you would like to watch TLS 1.3 decryption in action, have a look at the TLS 1.3 demo video on our Cisco Secure Firewall YouTube channel .

Special Considerations

Replace Known Key for Decrypt with Resign

For the Decrypt -- Resign action, the system replaces the public key. The Replace Key checkbox determines how to apply the decrypt action to self-signed server certificates.

If deselecting the Replace Key checkbox, self-signed certificates are treated like any other server certificate. Secure Firewall replaces the key and resigns the certificate. Generally, the endpoint is configured to trust Secure Firewall and therefore trusts the resigned certificate.

If Replace Key is selected, self-signed certificates are handled differently. The system replaces the key and generates a new self-signed certificate causing the endpoint's browser to generate a certificate warning.

EUN (End User Notification) for Blocked Traffic

The Block Response Page is not displayed for TLS traffic not decrypted by FTD, irrespective of the reason for the traffic block.

TLS Client Hello Modification

If the TLS Client Hello message matches a Decrypt - Resign rule, FTD will modify it. This function is enabled by default. Along with other information present in the TLS Client Hello packet, FTD uses the SNI field to predict the need for modification. However, since the TLS SNI can be spoofed, the final decision is based on the Server response.

1. FTD might have to modify the TLS Client Hello more often

2. If the TLS Client Hello has already been modified, but the TLS flow was not supposed to be decrypted, FTD cannot disengage and direct the TLS connection to Do Not Decrypt. Due to the Client Hello modification, the Message Authentication Codes (MAC) computed by the client and the server no longer match, so FTD still has to proxy the complete TLS/SSL handshake between the client and server.

📘

Note

More information about how the changes to the TLS Client Hello are done is available here.

Browser Errors related to Untrusted Certificates

Traffic encrypted with a re-signed server certificate causes client browsers to warn that the certificate is not trusted. To avoid this, add the CA certificate to the organization's domain root trusted certificates store or the client trusted certificates store.

Exceptions from Decryption

You might exempt traffic from decryption for various reasons: performance gain or end-user experience. Many mobile devices and enterprise SaaS cloud applications use mutual authentication or certificate pinning to validate the application server's certificate. Since FTD modifies the server certificate during outbound decryption ( decrypt -- resign ), the client application does not accept the server certificate causing the TLS connection to fail.

Suggested exemptions from Decryption:

1. Trusted Vendor domains and Trusted Subnets ( Low risk )
2. Guest Networks
3. Internal Server Backups or Trusted Flows
4. Enterprise SaaS Application Traffic such as Office 365, Microsoft updates, Apple updates etc.
5. TLS traffic using TLS Heartbeat

Verification/Troubleshooting

The Connection Events Viewer provides insight into flow evaluation against an Decryption policy rule. Not only does the event viewer help identify if the correct rule is being matched, it also reports any encryption errors as part of the TLS handshake. If you do not see encryption information for the flow of interest, ensure that logging is enabled in the Decryption rules and corresponding Access Control Policy rules.

Use the output of the following commands to check for TLS oversubscription and various TLS processing statistics:

show snort tls-offload

show counters

Gather advanced information by using a combination of debugs such as:

system support debug

system support ssl-debug

🚧

Warning

The SSL debugs should only be enabled and collected under supervision by TAC. As SSL debugs are verbose, the right level of debugging information needs to be configured depending on the issues.

Summary

Decryption policies play an essential role in protecting against threats. An optimally configured Decryption policy protects your environment against attack vectors embedded in encrypted traffic and ensures a problem-free end-user experience. Decryption policies slowly evolve as the needs of the organization change. Beginning with release 6.4, series 1000, 2100, 3100, 4100, and 9300 devices use dedicated hardware-based decryption. Using hardware acceleration for TLS operations ensures high performance without compromising security.

📚 Additional Resources

Demystifying TLS Decryption and Encrypted Visibility Engine on Cisco Secure Firewall Threat Defense – BRKSEC-3320

Cisco Secure Firewall YouTube channel

FMC - Understanding Traffic Decryption

The Illustrated TLS 1.2 Connection

The Illustrated TLS 1.3 Connection

RFC 8446 - The Transport Layer Security (TLS) Protocol Version 1.3