The Test Tool allows you to test CA SiteMinder®’s performance when the Policy Server receives more than one request at a time. Using stress tests, you can simulate multiple agents talking to the Policy Server simultaneously or a single agent communicating with the Policy Server on multiple threads.
Stress tests are run in Advanced Playback mode. The Test Tool receives instructions from a Thread Control file that specifies which tests to run and how many times to run them. After executing the instructions in the thread control file, the results of the test are written to an output file.
Configure a Thread Control file to define the Command Script files to run, number of repetitions, threads, and so on to automate complex tests for stress testing. A Thread Control file contains multiple instruction lines in the Test Tool’s own scripting language and comments, indicated by the # symbol at the beginning of a line.
The basic instructions are in the following format:
command_script_file_name, repetition_count, thread_count, [max_time_in_seconds]
Specifies the pathname of the previously recorded command script file.
Specifies the number of times that the Test Tool runs the Command Script.
Specifies how many simultaneous threads the Test Tool initiates to run the Command Script.
(Optional) Specifies a limit (in seconds) for the duration of the test. If elapsed test time exceeds this limit then playback halts, regardless of whether the configured number of repetitions are complete.
For example:
# c:\temp\test_data.txt, 8, 6, 120
This line specifies that:
The Test Tool writes the output of the test to a file. The output file name is the name of the input file with _out# appended to it, where # is the incremented thread number. For example, the output files for the test above are c:\temp\test_data.txt_out1 to c:\temp\test_data.txt_out6.
The Test Tool scripting language includes the commands in the following table to control the script file output. See the following figure for a sample thread control file.
|
Command |
Description |
|---|---|
|
.report |
Generates a final report (as an output file) summarizing the test results. The report does not include the status of each server request. This is the default. |
|
.reportspread |
Generates a file containing a response time spread report. The file name is command_script_stats_spread (for example, isprotected-record.txt_stats_spread ). |
|
.output |
Generates a final report (as an output file) summarizing the overall results including the status of each server request. |
|
.viewstats |
Displays final statistics in a text editor. |
|
.verbose |
Generates output files containing the details of each server request. |
|
.brief |
Generates output files containing the brief results of each server request. This option is only valid when used with the .output command. |
|
.sleep |
Lets the Test Tool pause for a specified amount of time (in milliseconds). This simulates intermittent server requests. |
|
.userselectionmode |
Determines whether usernames are used sequentially or selected randomly. Valid values are 0 (sequential) and 1 (random). For more information, see (Optional) Configure How the Test Handles Usernames. |
|
.randomseed |
Allows the pseudo-random sequence of names to be repeated (up to thread ordering) for each test. For more information, see (Optional) Configure How the Test Handles Usernames. |
|
.connect settings_file |
Initializes the Test Tool using information from the Test Tool Settings file to set up a multi-threaded test with one simulated Agent. The default multi-threaded test comprises multiple simulated Agents with one simulated Agent per thread. You can also set up a test with one simulated Agent and multiple threads by using this option. |
|
.disconnect |
Un-initializes the Test Tool to indicate the end of one simulated Agent multi-threaded test. |
Example Thread Control File
.output
.connect c:\test\smtest.ini .brief c:\temp\test_data1.txt, 2, 3 .verbose .sleep 5000 c:\temp\test_data1.txt, 2, 2 .brief c:\temp\test_data1.txt, 3, 4 .connect smtest.ini c:\temp\test_data1.txt, 5, 6 .disconnect
Use the Advanced Playback test mode to run a stress test defined in a Thread Control file.
Follow these steps:
The Test Tool runs the Thread Control file and creates an output script file.
The following three parameters configure how a test handles usernames.
(Defined in the Command Script file.) Specifies the number of users of the format AAAAAA, BAAAAA, through ZZZZZZ that the test uses for authentication and authorization. If the UserCount: parameter is set, the UserName: parameter is ignored.
Note: Although it is ignored, the Username: parameter is still required. Do not delete it.
Add the UserCount: parameter using the same format as the other parameters in the Command Script file. That is, enter the complete parameter name (including the "34 " prefix) and the parameter value starting at column 24 of the line.
For example:
34 UserCount: 1000
(Defined in the Thread Control File) Determines whether usernames are selected sequentially or randomly. Valid values fare 1 and 2.
If set to 1, the test steps sequentially step through usernames until the value specified by the UserCount parameter is reached. Usernames then wrap back to "AAAAAA" until all repetitions are completed or time has elapsed.
If set to 2 (the default), users are selected randomly.
(Defined in the Thread Control File) If configured with a non-zero integer value, causes a randomly ordered set of usernames in the range "AAAAAA" until the user name indexed by the value of the "UserCount" parameter. This allows the pseudorandom sequence of names to be repeated (up to thread ordering) for each test.
If .randomseed is not specified, the current time is used.
To simulate a steady load in which requests are sent to policy server at a constant rate, configure the following parameter in the Thread Control File.
Specifies the amount of sleep time (in milliseconds) between each request to the Policy Server on each thread.
For example, if the.sleepbetweenrequests parameter is set to 5 (milliseconds), then requests are sent to the Policy Server approximately 200 times per second on each thread.
Note: The actual request rate can be influenced by the following factors: time requests take to complete, amount of CPU time that each thread gets, and external factors.
You can configure the Test Tool to write all playback test results to a comma-separated values (CSV) format file by setting the following system environment variable:
Specifies the pathname of the CSV file to which the Test Tool should write playback test results. Playback test results are written in comma-separated format to the specified file with a header line at the beginning.
If the file already exists, the file is appended. If the Thread Control file specifies playback of multiple recordings, the results are aggregated.
When you run a stress test, the Test Tool generates a report summarizing the results. This report contains the following information:
The report is saved in the directory where the thread control file is located. The name of the report is the name of the thread control file with _stats appended to it. For example, the thread control file, thread.txt, yields a report named thread.txt_stats.
Control File: C:\temp\control.txt Started at: 0:02:05.481 Finished at: 0:03:22.672 Total Elapsed: 0:00:01.396 Minimum Request Time: 0:00:00.400 Maximum Request Time: 0:00:05.498 Average Request Time: 0:00:01.396 Total Requests 234 Throughput (Req/Sec): 3.241 Request Count Yes No Timeout Error --------------- ------ ------ ----- -------- ----- IsProtected 78 72 0 0 6 IsAuthenticated 78 78 0 0 0 IsAuthorized 78 72 0 6 0 ------------------ ------ ----- ------- ----- Total: 234 222 0 6 6 |
|
Copyright © 2014 CA.
All rights reserved.
|
|