3. PERFORMANCE ANALYSIS TOOLS › 3.1 Batch Initiator Simulation › 3.1.5 Case Study › 3.1.5.1 Problem Description

3.1.5.1 Problem Description

Our sample site is running a validation run of a model
against its actual initiator structure.  You should always
use a validation run as the first step in applying a model to
a system, because the output of the validation run provides
insight into how well the model simulates the type of arrival
pattern and mix of priorities that occur for each job class
in the workload.  More specifically, the run helps you detect
features of your workload (if any) that the model handles
poorly.

You should not underestimate the importance of this step.
Using the model without this step is similar to trying to use
a map to get to point B without first determining your
current position.

One of the most important decisions in model validation is
the selection of the time period that you want to simulate.
The objective is to select a period that represents your
workload satisfactorily.  Represent does not necessarily mean
that some sort of average picture of the workload is the
desired input.  Rather, the goal is to represent the workload
with both stable periods and periods when more spectacular
events occur.  For example, if there are peaks in the arrival
rate for certain classes at certain times, these peaks should
be represented in your model input.

There is no set method for developing a good set of input for
the validation run.  However, the task should include the
steps described below.  Additional steps may be required,
depending on how the system being studied is used.

Characteristics of a job class that affect the model's
ability to model the job class are the arrival pattern of
jobs, the length of the job execution, and the priorities of
the jobs.  The steps below are intended to help you gain an
understanding of these items for the system.

    o  You must acquire an understanding of how job classes
       and priorities are defined and used on the system
       being studied.  For example:

       -  Are certain classes used only for production or
          test?
       -  Are some classes used only at certain times?
       -  Are some classes used only by some users?
       -  What resource constraints (if any) are associated
          with each class?
       -  How are job priorities used to control the
          workload?

    o  Consider cyclic patterns in the workload.  This should
       include daily, weekly, monthly, and even yearly
       patterns if they are strong.  Strength here is
       measured in terms of the effect on job arrival
       patterns and job execution times.  A clustering of the
       workload for the system would be useful in performing
       this task.  In addition, such areas as operating
       procedures for production jobs and procedures
       controlling test jobs can be useful in identifying
       important features of the system.

Once you have an understanding of the characteristics that
you believe best represent the batch environment that you are
modeling, choose data that has those characteristics.  In
choosing data, consider the following:

    o  You may need to select data from several different
       days in order to get a good representation of the
       workload.

    o  At a minimum, you should run the model against data
       for a full day.  Or, if you use different initiator
       structures at different times of the day, you should
       run the model against the period in which the
       structure you are studying is being used.

    o  If possible, choose a starting time when the input
       data is selected with all initiators unoccupied and
       queues empty.

       If this is not possible, exclude the first few hours
       of the simulation from the reports (except perhaps in
       a first validation run).  If these hours are not
       excluded, the simulation results will be distorted by
       jobs that arrived before the simulation period and
       were not finished when the simulation period began,
       because these jobs are not simulated.

    o  Save the data that you select so that you can use the
       same input to simulate proposed changes to the
       initiator structure.  This aids in before and after
       type comparisons, avoids the problem of losing the
       data because it is aged off the CA MICS database, and
       allows you to run the simulation without interfering
       with the regular database operation.   Section
       3.1.6.2.6 describes the Save selected input
       observations parameter of the Extended Options Menu.