Reference Information › Appliance Catalog Reference Guide › System Catalog › Gateways › INSSLR2 - Redundant HTTP Input Gateway with SSL Support

INSSLR2 - Redundant HTTP Input Gateway with SSL Support

Latest version: 1.0.3-2

The INSSLR2 appliance is a layer-7 gateway for secure HTTP requests. It converts the requests to unencoded HTTP requests. You can use this functionality to support secure HTTP on the client side.

Note: The INSSLR2 appliance only works with multiple raw interfaces. You can configure the raw interfaces in the Interface tab of the Application Configuration Editor. If you do not need multiple raw interfaces, please use the INSSLR appliance.

However, the back-end processing infrastructure does not or cannot support SSL, including:

• using a fast and light-weight HTTP server that does not have SSL support
• using multiple back-end servers for performance or for redundancy connected through a load-balancing switch
• using multiple back-end servers for unrelated functions connected through a URL switch
• offloading the SSL encryption or decryption to a separate server to improve throughput

If configured, INSSLR2 also supports client certificate authentication. In the case of SSL mutual authentication, both the client and server exchange their certificates and corresponding public keys. Both the client and/or server may contact the certificate authority (CA) that issued the certificate and confirm the certificate is authentic before proceeding.

In its default configuration, INSSLR2 acts as a layer-7 gateway for secure HTTP requests. It also provides a firewalled entry point for network traffic into an application. This can be configured with an Internet-accessible static IP address.

When configured, two INSSLR2 appliances can be used in failover mode to provide a redundant service. In this case, INSSLR2 IP address is running on only one of the nodes and is automatically transfers to the other INSSLR2 appliance in case of failure. Only one of the INSSLR2 appliances is active at one time. When running in failover mode, the INSSLR2 can be configured to run in several modes:

• Asymmetric - one of the appliances is primary. When the primary appliance is online, it takes over the IP address and becomes active.

  If the primary node fails or is stopped, the backup appliance becomes active. When the primary appliance becomes available again, it reassumes the IP address and becomes active. The primary appliance is the one with the lowest IP address (string comparison) on the fover interface.

• Symmetric - both appliances are primary. Whichever appliance starts first takes over the IP address and becomes active. The other appliance becomes active only, if the first appliance fails or is stopped.

  Whenever the first appliance becomes available again, it does not automatically take over the IP address.

When running in redundant mode, INSSLR2 provides an interface on its ctl terminal for:

• forcing INSSLR2 to become primary
• forcing INSSLR2 to become backup
• check the state of the primary or backup INSSLR2

INSSLR2 constantly monitors the health state of the backend appliance connected to its http terminal. The health state checks conducted by INSSLR2 may include a simple TCP connect check or a more complex HTTP request, if that is specified on INSSLR2 boundary.

In the case of a failure of the connected appliance, INSSLR2 reports an error to the grid dashboard or, if in redundant mode and configured to do so, it does a failover to the backup INSSLR2 appliance.

To support applications that need to appear at a single IP address for more than one service, configure the INSSLR2 to direct non-HTTP traffic transparently to a separate output terminal. For such connections, the appliance acts as a layer-3 firewall/NAT router.

Note: If two INSSLR2 appliances in the same application are configured to run in redundant mode, a warning message displays when you configure the same IP address on two different interfaces. Please ignore that error.

Boundary

Resources

Notes

• When using INSSLR2 in failover mode (fover_mode is not none), the minimum memory is 192M.
• The amount of memory that is given to INSSLR2 does not affect its throughput. Use more memory only to support more concurrent requests where the back-end servers may hold the requests for excessively long periods of time.

Terminals

Properties

path/to/keys/ca_list_client.pem. 

path/to/keys/ca_list_client.pem. 

Healthcheck Properties

Advanced Properties Used in Failover Scenarios

Volumes

Error Messages

The following messages may appear in either the appliance log file or the system log of the grid controller when the appliance fails to start:

• Failed to mount data volume
• Failed to set permissions on the data share
• Samba failed to start
• NFS failed to start
• Apache failed to start

Performance

Application Failover

INSSLR2 detects the failure of the INSSLR2 in approximately 10 seconds and requires an additional 10 seconds to failover to the other application. The total time from when the application failure first occurs to when the other application takes over the traffic is approximately 20 seconds.

Request Rate

INSSLR2 routes no less than 3000 transactions (request/response pairs) per second, depending on the document size and network bandwidth available.

Data Throughput

INSSLR2 routes no less than 7 MB/second, depending on the document size and network bandwidth available

Concurrent Connections

With its default memory, INSSLR2 supports no less than 500 concurrently pending requests. A pending request being an open TCP connection from the client, on which there is one or more un-completed HTTP request in progress.

The maximum amount of concurrent connections depends of available free memory. Each 16 MB additional memory increases the number of concurrent http transactions by 500.

For example, if you run in redundant mode and want to serve 2000 concurrent connections, you need 100M + 3*16M = 148M. The minimum memory when running in redundant mode is 100M.

Notifications

When running in redundant mode (fover_mode is not none), INSSLR2 triggers notifications when it becomes active or passive. This is performed on startup of the active node or when a failover occurs. Each node sends a notification that it became active or passive.

INSSLR2 sends two notifications:

• Notification to the grid dashboard: INSSLR2 appliance APP_NAME:COMP_NAME <acquired|gave up=""> resource ip_addr
• HTTP request using the nfy terminal. It is up to the receiving end to take action based on the notification.

  The requested URL is:

  http://nfy/ fover_nfy_prefix fover_mode= fover_mode &state= <start|stop> &ip_addr= ip_addr &fover_local_ip= fover_local_ip &fover_remote_ip= fover_remote_ip &fover_netmask= fover_netmask

You can use the fover_nfy_prefix to change the location of the remote script that is called and/or add additional parameters.

Examples for fover_nfy_prefix values:

/path/to/script.php?, /path/to/script.php?username=user&password=pass&. 

Notes:

• When using fover_nfy_prefix different than the default, verify that it ends with ? if fover_nfy_prefix is just the path or script or & if fover_nfy_prefix includes additional parameters.
• If a node becomes unavailable, it may not be able to send a notification that it became passive. Only the node that became active sends notification.
• The timeout for the http request is 5 seconds to ensure the failover is not slowed.

Healthcheck

INSSLR2 runs a cronjob every minute that checks the following:

• Is the free space on the data volume less than 20%
• Is rsync daemon running (only if configured as slave)
• Is replication process running (only if configured as master). If not - restart replication process.

If any is true, an error message is sent to the grid dashboard. If more than one test fails, a summary message with all errors posts on the grid dashboard.

Each error is sent only once per hour to the grid dashboard. No errors are reported in the first 5 minutes after the appliance start. This prevents false alarms when the other node in the replication has not started.

Server Certificates

To use SSL you need both the signed certificate and the private key used to encrypt it. The key and the certificate should be in PEM format and placed in a single file specified by the cert_file property.

Generate Server Certificates

You can generate a server certificate that can be signed by a trusted Certificate Authority (CA) for a fee or create a self-signed certificate.

Follow these steps:

1. Generate a private key using the following command:

   openssl genrsa -out privkey.pem 2048 
   

   You can also generate a pass protected key using the following command. However, if you create a pass protected key, you need to remove the password before using it in INSSLR2. INSSLR2 requires a passwordless key.

   openssl genrsa -des3 -out privkey.pem 2048 
   

2. Create a certificate using one of the following options:
   • Create a certificate request and have it signed by a trusted CA for a fee.

     Execute the following:

     openssl req -new -key privkey.pem -out server.csr 
     

     After you send the .csr file to your trusted certificate authority, it will return a signed certificate ( .crt file) which you can use.

   • Create a self-signed certificate for test purposes. Browsers requesting your site will issue warnings that the certificate is not signed by a trusted CA.

     Execute the following:

     openssl req -new -x509 -key privkey.pem -out server.crt -days 1095 
     

Using Server Certificates

You can now place the certificate and key in a file and use it in INSSLR2 by executing the following command:

cat privkey.pem  server.crt > server.pem 

If your key is password protected, you can remove the password by executing the following:

openssl rsa -in key_with_pass.pem -out privkey.pem 

Using Existing apache+mod_ssl Server Certificates

INSSLR2 allows you to use an existing certificate that you use in Apache. Verify the key is not password-protected using the above steps, then place the private certificate and key in a single file.

For example:

• cat privkey.pem server.csr > server.pem

If you are using a chained certificate, you should also include the intermediate certificate provided by the issuer. Place the private key, certificate and intermediate certificate in a single file in that order.

For example:

• cat privkey.pem server.csr sf_issuing.crt > server.pem

Important! The server signing key is your Web site's proof of identity. It is also vulnerable since it is not password-encrypted to allow the appliance to read it without your assistance.

When installing it on the data volume, take measures to protect the key file. Do not use the same data volume for other purposes and do not make it visible to a Web server. For example, to host Web-accessible data, such as HTML pages and scripts.

Client Certificates

The following example outlines how to create your own Certificate Authority and use it to sign your own certificates using OpenSSL. This was tested using OpenSSL 0.9.8b, the same version as is installed in INSSLR2.

Create a Certificate Authority

If you already have your own Certificate Authority used to create self-signed server certificate, you can skip this step and use that Certificate Authority. When creating the instruments of the trusted authority for your application, it is important that the environment be secure.

Important! The private key for certificate authority is the basis of trust for the application. Do not misplace it or provide it to other parties.

Follow these steps:

1. Create a working directory on a secure host and, in that directory, create:
   • mkdir CA
   • mkdir CA/private
2. Create a password protected RSA key for the CA with a 2048 bit key length:

   openssl genrsa -des3 -out CA/private/CA_key.pem 2048 
   

3. From the private key, create the public-key certificate for the CA:

   openssl req -new -key CA/private/CA_key.pem -x509 -days 365 -out CA/CA_cert.pem 
   

Create Client Certificates Signed by the Certificate Authority

You can create client certificates that are signed by a Certificate Authority.

Follow these steps:

1. Generate an RSA private key:

   openssl genrsa -out clientA_privkey.pem 2048 
   

   To create a password protected key, use the -des3 switch

2. Generate a certificate signing request (CSR) from the private key:

   openssl req -new -key clientA_privkey.pem -out clientA_request.csr 
   

3. Sign the certificate contained in the CSR using the certificate generated for the CA:

   openssl x509 -req -days 365 -in clientA_request.csr -CA CA/CA_cert.pem -CAkey CA/private/CA_key.pem -CAcreateserial -out clientA.cer 
   

Notes:

• The -CAcreateserial switch enables the assignment of unique serial numbers to issued certificates. If this is the first certificate issued by the CA, the file CA/CA_cert.srl is created to track serial number assignments. In creating subsequent client certificates, the -CAserial switch is used in place of the -CAcreateserial switch.

  For example:

  openssl genrsa -out clientB_privkey.pem 2048 
  

  openssl req -new -key clientB_privkey.pem -out clientB_request.csr 
  

  openssl x509 -req -days 365 -in clientB_request.csr -CA CA/CA_cert.pem -CAkey CA/private/CA_key.pem -CAserial CA/CA_cert.srl -out clientB.cer 
  

• To create a single PEM formatted file which includes both the client certificate and key:

  cat clientA_privkey.pem clientA.cer > clientA.pem 
  

• To create the equivalent file in pkcs12 format usable by most browsers:

  openssl pkcs12 -export -in clientA.cer -inkey clientA_privkey.pem -out clientA.p12 
  

Create the CA List Files Used by INSSLR2

Revoke Client Certificates Signed by the CA

To revoke a client certificate issued by the CA, use the following command:

openssl ca -config openssl.conf -revoke clientB.cer -crl_reason keyCompromise 

Where an example openssl.conf is shown below and the crl_reason parameter is one of: unspecified, keyCompromise, CACompromise, affiliationChanged, superseded, cessationOfOperation, certificateHold or removeFromCRL. The matching of reason is case insensitive.

To re-generate the certificate revocation list (CRL):

openssl ca -config openssl.conf -gencrl -out CA/crl.pem 

In this example, crl.pem is the file that should be provided to INSSLR2 as the cert_revocation_list property.

An sample config file used for the above examples is: 

      ####################################################################

      [ ca ]

      default_ca      = CA_default            # The default ca section

      ####################################################################

      [ CA_default ]

      dir             = ./CA                  # Where everything is kept

      certs           = $dir/certs            # Where the issued certs are kept

      crl_dir         = $dir/crl              # Where the issued crl are kept

      database        = $dir/index.txt        # database index file.

      #unique_subject = no                    # Set to 'no' to allow creation of

                                        # several ctificates with same subject.

      new_certs_dir   = $dir/newcerts         # default place for new certs.

      certificate     = $dir/cacert.pem       # The CA certificate

      serial          = $dir/serial           # The current serial number

      crlnumber       = $dir/crlnumber        # the current crl number

                                        # must be commented out to leave a V1 CRL

      crl             = $dir/crl.pem          # The current CRL

      private_key     = $dir/private/cakey.pem# The private key

      RANDFILE        = $dir/private/.rand    # private random number file

      default_days    = 365                   # how long to certify for

      default_crl_days= 30                    # how long before next CRL

      default_md      = sha1                  # which md to use.

      preserve        = no                    # keep passed DN ordering

      policy          = policy_anything

      [ policy_anything ]

      countryName             = optional

      stateOrProvinceName     = optional

      localityName            = optional

      organizationName        = optional

      organizationalUnitName  = optional

      commonName              = supplied

      emailAddress            = optional

Create the Certificate Authority List Files Used by INSSLR2

You can create certificate authority list files that are used by INSSLR2.

To create the file identified by the ca_list_client property:

1. Access the following file:

   cat CA/private/CA_key.pem CA/CA_cert.pem > ca_list_client.pem 
   

2. Place this file either on the key volume or the volume which is nfs mounted via the fs terminal.

   If desired, the INSSLR2 server certificate, such as e.g. server.pem, can also be created in the same manner as the client certificates and lsigned by the created certificate authority. Do not use a password for this certificate.

3. Once server.pem and ca_list_client.pem are in place, test the client certificate authentication as follows:
   • To test from a browser, import a pkcs12 formatted client certificate into FireFox using:

     Tools=>Options=>View Certificates=>Your Certificates=>Import. Point the browser at the IP address of INSSLR2.

   • To test the SSL handshake from a remote host use the following command::

     openssl s_client -host IP-address -port 443 -showcerts -ssl3 -cert clientA.cer -key clientA_privkey.pem -state 
     

Client Certificate Headers

If a client connects to INSSLR2 using HTTPS and presents a client certificate, INSSLR2 adds the following headers to the request returned to the backend server:

• X-SSL-Subject details about the certificate owner
• X-SSL-Issuer details about the certificate issuer (Certificate Authority)
• X-SSL-Serial certificate serial number (decimal)
• X-SSL-Cipher the cipher currently in use
• X-SSL-Certificate the full client certificate (PEM-format multi-line)

It is the responsibility of the application to use or not use these headers. INSSLR2 just passes this information without verifying it, except for signature and encryption correctness.

Typical Usage

Web Applications

To provide https access to your application, connect the http terminal directly to the WEB appliance.

For a scalable web application hook the http terminal to a HALB appliance.

The configuration examples list only properties set to non-default values.

Web Application with Additional Services

If you have more services than just http in your application, you can use INSSLR2 to pass http traffic to its http terminal and use the aux terminal for other services.

Web Applications with Multiple Additional Services

If you have multiple TCP/UDP services and HTTP, you can use INSSLR2 to pass HTTP traffic to its HTTP terminal and use the aux to feed the remaining traffic to PS8. PS8 then feeds the desired traffic to the backend servers.

Redundant Web Applications

To provide a redundant web application, copy the application and use INSSLR2 to provide failover capabilities for the external IP address.

Note: If two INSSLR2 appliances in the same application are configured to run in redundant mode, a warning message displays when you configure the same IP address on two different interfaces. Please ignore that error.

Backup Web Application

If you need a backup application that notifies users for the downtime, you can use INSSLR2 to build a backup application that requires minimum resources.

Appliances in use:

• user: input gateway for user requests
• web: web server displaying maintenance message

INSSLR2 on Primary Application

INSSLR2 on Backup Application

Note: The fover_local_ip and fover netmask properties are attributes of the interface connected to the fover terminal and can be configured on the Interface tab of the Application Configuration Editor.

Redundant Web Application in a Single Application

To run your application in redundant mode without creating a new application, you can double the appliances and run them in redundant mode. Since this involves using at least two web servers and two database appliances in normal mode, they are all used which provide scalability. However, each of them can serve the application alone in case the other appliance fails.

If you need additional scalability, you can add additional web and database appliances. In this example half of the appliances (in1, sw1, web1, db1) are running in one failover group and the remaining (in2, sw2, web2, db2) are in another failover group to allow the application can survive a server crash. This application provides redundancy at a very low cost as all the appliances (except for one INSSLR2 and one HALB appliance which require very low resources) are in active state and no resources are spent for a backup application that runs only when the primary fails.

in1

in2

db1

db2

Redundant Web application wth Two Identical Applications

You can run two identical applications on the same grid, but on separate servers or on different grids if the IP address is accessible from both grids. This allows the failure of a server to only affect one of the applications.

The example below shows an application that uses a database appliance which is also running in redundant mode. If one application fails, the other application can take over without data loss.

Appliances in use:

• insslr - redundant input gateway for user requests
• admin - input gateway for log files access
• sw - redirect port 8080 from admin to ui on db
• repl_in - input for the remote application to connect to the db appliance in order to replicate the database
• web_lb - web load-balancer for user requests
• web1, web2 - web servers with active content, such as CGI scripts
• db - MYSQLR64 configured to be both a master and a slave at the same time
• content - storage for database error log files, web content, and web logs
• logs - storage for database error log files
• repl_out - output gateway for the db appliance to connect to the remote application in order to replicate the database
• mon - MON appliance

The client requests arrive on the insslr gateway. The gateway forwards the requests to the web_lb load balancer, which directs the request to one of the web servers, web1 or web2.

The web servers access the db database. The db appliance connects to the remote application. In order to replicate the database, the remote application is an identical copy with the only difference being the server_id of db and the network setup.

The remote application connects to the db appliance using the repl_in gateway which is configured to allow connection only from the repl_out gateway of the remote application. The db appliances in the two applications are running in master-master setup to ensure they always have identical data.

Example Property Configuration

Properties that are not listed should be left to their default values. Web access to db is available using admin gateway on port 8080.

insslr

db

Important! Only one of the applications is active at any time, the other one is running just for failover and is used when the active application fails.

Notes:

• INSSLR2 supports HTTP 1.0 and HTTP 1.1, but, if the client identifies itself as MSIE, HTTP 1.1 support is turned off for that connection. This avoids bug issues in some versions of MSIE.
• If the client is not MSIE, INSSLR2 supports HTTP 1.1 for the client. including the multiple requests per TCP session ability. This occurs even if the back-end server uses supports only HTTP 1.0.
• INSSLR2 supports a single external IP and, therefore, only a single SSL certificate may be used.

Open Source and Third Party Software Used Inside the Appliance

INSSLR2 uses the following thrd party open source packages in addition to the third party open source packages used by its base class LUX6 .