Run a Stress Test

Policy Server Guides › Policy Server Configuration Guide › SiteMinder Test Tool › Run a Stress Test

Run a Stress Test

The Test Tool allows you to test SiteMinder’s performance when the Policy Server receives more than one request at a time. Using stress tests, you can simulate multiple agents talking to the Policy Server simultaneously or a single agent communicating with the Policy Server on multiple threads.

Stress tests are run in Advanced Playback mode. The Test Tool receives instructions from a Thread Control file that specifies which tests to run and how many times to run them. After executing the instructions in the thread control file, the results of the test are written to an output file.

More information:

Configure a Thread Control File

How to Use the Test Tool in FIPS-only Environments

Configure a Thread Control File

Configure a Thread Control file to define the Command Script files to run, number of repetitions, threads, and so on to automate complex tests for stress testing. A Thread Control file contains multiple instruction lines in the Test Tool’s own scripting language and comments, indicated by the # symbol at the beginning of a line.

The basic instructions are in the following format:

command_script_file_name, repetition_count, thread_count, [max_time_in_seconds]

command_script_file_name: Specifies the pathname of the previously recorded command script file.
repetition_count: Specifies the number of times that the Test Tool runs the Command Script.
thread_count: Specifies how many simultaneous threads the Test Tool initiates to run the Command Script.
max_time_in_seconds: (Optional) Specifies a limit (in seconds) for the duration of the test. If elapsed test time exceeds this limit then playback halts, regardless of whether the configured number of repetitions are complete.

For example:

# c:\temp\test_data.txt, 8, 6, 120

This line specifies that:

The input Command Script is c:\temp\test_data.txt
The Test Tool runs the script eight times
Six simultaneous threads run the script
The test ends after 120 seconds, regardless of whether eight repetitions are complete

The Test Tool writes the output of the test to a file. The output file name is the name of the input file with _out# appended to it, where # is the incremented thread number. For example, the output files for the test above are c:\temp\test_data.txt_out1 to c:\temp\test_data.txt_out6.

The Test Tool scripting language includes the commands in the following table to control the script file output. See the following figure for a sample thread control file.

Command	Description
.report	Generates a final report (as an output file) summarizing the test results. The report does not include the status of each server request. This is the default.
.reportspread	Generates a file containing a response time spread report. The file name is command_script_stats_spread (for example, isprotected-record.txt_stats_spread ).
.output	Generates a final report (as an output file) summarizing the overall results including the status of each server request.
.viewstats	Displays final statistics in a text editor.
.verbose	Generates output files containing the details of each server request.
.brief	Generates output files containing the brief results of each server request. This option is only valid when used with the .output command.
.sleep	Lets the Test Tool pause for a specified amount of time (in milliseconds). This simulates intermittent server requests.
.userselectionmode	Determines whether usernames are used sequentially or selected randomly. Valid values are 0 (sequential) and 1 (random). For more information, see (Optional) Configure How the Test Handles Usernames.
.randomseed	Allows the pseudo-random sequence of names to be repeated (up to thread ordering) for each test. For more information, see (Optional) Configure How the Test Handles Usernames.
.connect settings_file	Initializes the Test Tool using information from the Test Tool Settings file to set up a multi-threaded test with one simulated Agent. The default multi-threaded test comprises multiple simulated Agents with one simulated Agent per thread. You can also set up a test with one simulated Agent and multiple threads by using this option.
.disconnect	Un-initializes the Test Tool to indicate the end of one simulated Agent multi-threaded test.

Example Thread Control File

.output

.connect c:\test\smtest.ini
.brief
c:\temp\test_data1.txt, 2, 3
.verbose
.sleep 5000
c:\temp\test_data1.txt, 2, 2
.brief
c:\temp\test_data1.txt, 3, 4
.connect smtest.ini
c:\temp\test_data1.txt, 5, 6
.disconnect

Report Viewing

When you run a stress test, the Test Tool generates a report summarizing the results. This report contains the following information:

Time the test started and finished
Total elapsed time
Minimum, maximum, and average request time
Total number of requests
Throughput
The number of tests run and their results

The report is saved in the directory where the thread control file is located. The name of the report is the name of the thread control file with _stats appended to it. For example, the thread control file, thread.txt, yields a report named thread.txt_stats.

Control File:   C:\temp\control.txt
Started at:     0:02:05.481
Finished at:    0:03:22.672
Total Elapsed:  0:00:01.396
 
Minimum Request Time:   0:00:00.400
Maximum Request Time:   0:00:05.498
Average Request Time:   0:00:01.396
 
Total Requests          234
Throughput (Req/Sec):   3.241
 
Request         Count   Yes     No     Timeout   Error
--------------- ------  ------  -----  --------  -----
IsProtected     78       72     0      0         6
IsAuthenticated 78       78     0      0         0
IsAuthorized    78       72     0      6         0
------------------      ------  -----  -------   ----- 
Total:          234     222     0      6         6