Single Sign-On Service › SSO Policy Server Configuration Guide › Troubleshooting SSL Authentication Schemes

Troubleshooting SSL Authentication Schemes

This section contains the following topics:

Overview

Configuring the SSL Advanced Authentication Schemes requires Web Servers to be properly configured to use SSL. Many problems with configuring Authentication Schemes over SSL connections are likely to be SSL configuration issues. Therefore, the first step in troubleshooting Authentication Schemes over SSL is to verify that SSL is properly configured and working. Verify the configuration without the interaction of the agent so that these components can be individually analyzed.

Determine SSL Connection Ability

The first step in troubleshooting Authentication Schemes over SSL is to verify that SSL is properly configured and working.

Follow these steps:

Disable the agent protecting the realm for which you want to use an authentication scheme over SSL.
Using your browser, go to one of the following URLs (using a browser with a certificate):
- https://web_server_name:port (Netscape web servers)
- https://web_server_name:port/<SSL Virtual Directory> (IIS web servers)
- https://web_server_name:port (Apache‑based web servers)
If this SSL connection is configured to require certificates, you are to select a certificate.

SSL Configuration

Configure SSL and verify that it works properly before using the product. In order to make an SSL connection, you must be able to trust the certificate authority of an incoming certificate. For example, if a browser presents a certificate that is signed by VeriSign, you must have a VeriSign Certificate Authority that is installed and trusted in the web server. In addition to trusting client certificates that are presented, the server itself must have a certificate to present to the clients. The clients have to trust the Certificate Authority that issued the certificate. This setting allows mutual authentication. Once these certificates have been installed, you can configure the Web Server to use SSL and require certificates, if desired.

For detailed SSL configuration information, see the documentation that is provided with your web server software. This section contains step-by-step instructions for configuring your web server and browser to establish an SSL connection.

Enable the Web Server to Trust Client Certificates in Netscape

If a certificate authority is already installed in the Web Server, go on to the next section. Otherwise, install a certificate for the Certificate Authority on the SSL Web Server.

Follow these steps:

Obtain the Certificate Authority certificate and either keep it on your screen or save it to a file.
In Netscape Server Administration, select Keys & Certificates.
Click Install Certificate.
In the Certificate For field, fill out the Server Security Chain.
In the Certificate Name field, enter a description.
If you saved your Certificate Authority certificate to a file, enter the file name in the Message is in this file field. Otherwise, select the Message text (with headers) option button and paste the certificate in the Message text (with headers) field.
Click OK and restart the Web Server.

Configure the Netscape Web Server to use SSL

Configure the Netscape Web Server for SSL by requiring certificates.

Follow these steps:

In Netscape Server Administration, click Admin Preferences.
Click Encryption On/Off and ensure that Encryption is on.
If you are running the Certificate or Certificate with Basic Authentication Scheme, require certificates. See the Encryption Preferences setting where Require Certificates must be set to On. From a browser with an installed certificate, verify that you can get to https://servername:port.

Note: Do not turn on Required Certificates for the Certificate or Basic Authentication Scheme.

Establish Trust for the Netscape Certificate Authority

If a certificate authority is installed in the Web Server, you can establish trust between the two.

Follow these steps:

In Netscape Server Administration, select Keys & Certificates.
Select Manage Certificates.
Select the Certificate Authority. The system displays a dialog detailing the certificate.
Select Trust.
Click OK and restart the Web Server.

Enable the Web Server to Trust Client Certificates in Windows

Trust your client certificates by installing the appropriate Certificate Authority Certificates.

SSL Web Servers must have certificates for each Certificate Authority. Major certificate authorities could possibly have been installed. You can configure certificates in Windows operating environments by using the Certificates snap-in.

Configure the IIS Web Server to use SSL

Be sure that a secure port has been enabled on the Web Server. Generally this is port 443. You can verify this through the Management Console by right-clicking on the Web Server and in the Web site tab you will see an SSL Port. Be sure a port number has been installed.

The advanced authentication schemes will create virtual directories in the Web Server. These directories will automatically be configured to require SSL and certificates as required by the specific authentication scheme. However, for testing purpose, you may want to create a test virtual directory. You can configure this virtual directory to require certificates through the Directory Security tab, Secure Communications.

https://servername:port/virtual directory - Ensure that the browser is asked for a certificate.

Install the IIS Web Server Certificate

If you have not already done so, you will need to generate a key for your Web server. This is done through the Management Console, Key Manager. Access the Key Manager by doing the following:

Note: Note this process may be slightly different for IIS 3 and IIS 4.

To install the IIS Web Server Certificate

In the Management Console, right-click the Web Server and select Properties.
Click the Directory Security tab.
In the Secure Communications panel, click Key Manager.
Under Key, select Create New Key and a Wizard will guide you through the process.
Once you create a key, you can request a certificate using the file created in the steps mentioned earlier. Go to the Certificate Authority and request a certificate for this server. You will need to paste the certificate request information generated in Step 1 in order to receive a certificate. Once you received a certificate, go back to Management Console, Directory Security and click Key Manager to install the certificate for the key described in the next step.
Right-click the key name and select Install Certificate.
Restart the Web Server.

Enable the Web Server to Trust Client Certificates in Apache

If a certificate authority is already installed on your web server, go on to the next section. Otherwise, install a certificate for the Certificate Authority on the SSL web server as follows.

Follow these steps:

Download and build the following Apache components:
- Apache
- OpenSSL
- Mod_SSL
- Mod_so - This module is included as part of Apache, but it must be enabled during the build of the web server.
- RSAref-2.0
Copy the Certificate Authority certificate into the apache/conf/ssl.crt directory in x509 b64 format.
Run make in the apache/conf/ssl.crt directory.
Restart the web server.

Installing the Apache Web Server Certificate

The process for installing a certificate on an Apache Web Server varies with individual configurations. Consult the documentation for Mod_SSL and OpenSSL for details about how to configure these components.

SSL Troubleshooting

The following sections detail the most common problems encountered when dealing with SSL authentication schemes.

There Was No Prompt for a Certificate

If a certificate prompt did not appear, verify that SSL is configured appropriately. If the agent is installed, disable the agent. The first step is to verify a simple SSL connection.

Follow these steps:

Disable the agent protecting the realm for which you want to use an authentication scheme over SSL.
Using your browser, go to one of the following URLs (using a browser with a certificate):
- https://web_server_name:port (Netscape Web Servers)
- https://web_server_name:port/<SSL Virtual Directory> (IIS Web Servers)
- https://web_server_name:port (Apache Web Servers)
If this SSL connection is configured to require certificates, you are to select a certificate.

After Following Previous Procedure, Still No Certificate Prompt

Perform the following steps if you are still not receiving a certificate prompt.

Verify that all Firefox browsers are configured to ask every time.
Verify that all web servers are configured to use SSL and require certificates.
Verify the following settings for each Virtual Directory that is used by the product.
Verify the web server certificate expiration.
Verify that the browser certificate is valid.

Verify That All Firefox Browsers Are Configured to Ask Every Time

The Firefox web browsers can be configured to pass the same certificate automatically. This setting establishes the SSL connection using a certificate without prompting users to select a certificate.

Follow these steps:

In the Firefox browser, select Options from the Firefox menu.
Click Advanced.
Click the Encryption tab.
In the Certificates section, verify that the Ask me every time option is set.

Verify That All Web Servers Are Configured to Use SSL and Require Certificates

For Netscape Web Servers

In the Netscape Server Administration, click Admin Preferences.
Click Encryption On/Off and verify that the encryption is on, then click OK.
Click Encryption Preferences and verify that Required Certificates is set.
Restart the Web Server.

For IIS Web Servers

Verify that the virtual directories SMGetCredCert, SMGetCredCertOptional, SMGetCredNoCert are created and have the correct settings.

SMGetCredCert - Require Certificates will be selected
SMGetCredCertOptional - Accept Certificates will be selected
SMGetCredNoCert - Do not accept certificates will be selected

Note: As part of the CA SiteMinder® SSL Authentication setup, CA SiteMinder® configures SSL virtual directories based on the type of SSL connection required by the authentication scheme.

Verify the Following Settings for each SiteMinder Virtual Directory

Follow these steps:

For IIS web servers

In the Management Console, right-click a virtual directory and select Properties.
Click the Directory Security tab.
Click Edit Secure Communications.

For Apache web servers

In the httpd.conf file, be sure to set SSLVerifyClient as follows:

For Basic over SSL: SSLVerifyClient none
For Certificate or Basic: SSLVerifyClient optional
For Certificate/Certificate and Basic: SSLVerifyClient require
Note: For Apache-based web servers where Certificates are required or optional, the "SSL Verify Depth 10" line in the httpd.conf file must be uncommented.

Check the Web Server’s Certificate Expiration

Netscape Servers

In the Netscape Server Administration, click Keys & Certificates.
Click Manage Certificates.
Click ServerCert.
Verify that it is trusted, and has not expired.

IIS Servers

In the Management Console, right-click the Web Server and select Properties.
Click the Directory Security tab.
In the Secure Communications panel, click Key Manager.
Select a key. View its properties and verify that the key has not expired.
Restart the Web Server.

Apache Servers

If an Apache Web Server certificate expires, you receive an error message at server startup.

Verify Browser Certificate Validity

A missing certificate or an invalid certificate can prevent you from receiving a certificate prompt.

Open your Web browser and verify the validity of the browser certificate.

After Certificate Prompt, Authentication Failure Received

Apache Web Servers

Verify that the SSL Web Server contains the certificate authority of the certificate supplied.
Verify that the SSL Web Server Trusts the certificate authority of that certificate.
Ensure the SSL Verify Depth 10 is uncommented.

Netscape Web Servers

Verify that the Certificate Authority for the certificate is listed and that the Trust for the certificate has not expired. If it is missing or expired, install a new Certificate Authority certificate.

IIS Web Servers

Verify that the certificate is listed and that it is valid. If it is missing or expired, install a new certificate. If you are able to get to the destination directory, then certificates are installed correctly.

Verify Correct Policy Server and Web Agent Configuration

Verify your policy server and web agent configuration.

Follow these steps:

Check that the Policy Server is created correctly.
Check that the Web Agent contains the correct Policy Server information.
Verify that the Web Agent is enabled.
Restart the Web Agent and Policy Server.

SiteMinder Policy Should Allow Access, but SSL-Authentication Failed Message Received

This situation can result from a number of configuration errors. Some common errors include:

The SSL Server is not configured to Require Client Certificates. Therefore, the client is not passing a certificate; hence disabling the authentication process of the product. Verify this situation by enabling the logging option in the Web Agent. A proper log indicates that the user is unknown. To correct this problem, turn on Require Certificates in the SSL Web Server.
The Policy was not created properly. Check the users that are associated with the policy.
For Apache Web server, ensure that the SSL Verify Depth is set properly and uncommented.

More information:

Certificate Mapping for X.509 Client Certificate Authentication Schemes

How to Configure a Policy Domain

Error Not Found Message Received

This error occurs when the Authentication Scheme Parameter being configured improperly. The redirect is not configured properly so the web server is unable to find the SSL Web Agent component.

More information:

Authentication Schemes

Running Certificate or Basic but Cannot Enter Basic credentials.

On Netscape Web Servers, the Certificate or Basic scheme requires the Web Server to have encryption that is turned on, but does not require certificates. Be sure that in the Encryption Preferences section of the Netscape Server Administration, the Require Certificate setting is set to No.