Using CA Application Test › Using the Workstation and Console with CA Application Test › Reports › Viewing Reports › Interpreting Reports

Interpreting Reports

Sometimes, the results of reports are not what you expect, and it could appear that the reporting data is incorrect.

Looping Tests

Loops in test cases can cause unexpected results in the reporting engine.

The workflow engine is designed to flow from the beginning step to the end step, with no loops. One execution of this chain of steps is considered to be one pass or fail or abort event. Looping is designed to be initiated externally to the test case using a staging document. By using a staging document to specify 10 virtual users executing a test case once, you have 10 pass/fail/abort events. When the looping is done in the test case, a pass/fail/abort does not occur until the end step is reached, therefore creating only a single pass/fail/abort event.

Think Time and Reports

Think time is how long DevTest waits to start the execution of a test step. The purpose of think time is to simulate a real user interacting with any system. You can set think time in the step editor or in a staging document.

In the main reporting panel, we display total execution time, which is the length of time from start to finish of the test. Because this is a duration, we include think times by design, to show how long it took for the entire test to run. To exclude think times, set your think time amount to 0 in your staging document or on your test step.

If you are looking for performance numbers, generate a performance report that gives you response times for each step that does not include think time, which is a more meaningful report for performance tests. Think time is not part of the average step performance time.