Administration Guide › Mapping and Parsing

Mapping and Parsing

This section contains the following topics:

Event States

Mapping and Parsing Rules Tasks

How to Create a Message Parsing File

How to Create a Data Mapping File

Event Forwarding Rules Tasks

Event States

Information about events in your environment passes through a number of stages, from initial occurrence to possible final display by CA User Activity Reporting Module. Because the term "event" can refer to any one of these stages, we use the following terminology for the possible event states in your environment:

Native Event

Refers to the original occurrence of the state or action that triggers the event, a failed authentication, or firewall violation for example. The appropriate connector or listener service sends native events, parsed and mapped as appropriate, then inserted into the event log store, where it is available for display as raw or refined events.

Raw Event

Refers to the communication sent by the appropriate monitoring agent. Raw events contain information about the native event, often in the form of a syslog string or a name-value pair. This information is stored and searchable unless altered by suppression or summarization rules. Suppressed events are not recorded in the event log store; a set of summarized events is recorded as a single event expressing the outcome of the summarization.

Refined Event

Refers to the event information as mapped and summarized by CA User Activity Reporting Module. This information is stored and searchable.

Recorded Event

Refers to the raw or refined event information in the event log store. Raw events and refined events are always recorded unless suppressed or summarized. Mapped events have both raw and refined information available. This information is stored and searchable.

Consult the following diagram for information about event states:

More information:

Mapping and Parsing Rules Tasks

Suppression and Summarization Rules Tasks

Mapping and Parsing Rules Tasks

Message Parsing (XMP) and Data Mapping (DM) file pairs collect and normalize data from specific types of event sources. Most incoming native events pass through the parsing and then the mapping processes to create a reportable event that is inserted into the event log store. Events transmitted through SAPI or iTechnology do not require parsing, and proceed directly to the data mapping stage.

Note: To take full advantage of these advanced features, you need a thorough understanding of the raw and collected events in your environment, the target fields you want to parse, the regular expression syntax, the CEG, and DM and XMP files and how they parse events.

The XML-based XMP files read incoming raw event data and create name-value pairs, according to your specifications. DM files then map the events' name-value pairs assigned by message parsing into the common event grammar. When creating new parsing and mapping files, consider them as part of a process. For example, efficient and complete parsing allows quick and process-effective mapping.

More information:

How to Create a Message Parsing File

How to Create a Data Mapping File

Event Refinement Component Versions

How to Create a Message Parsing File

You can use the parsing file wizard to create, edit, or analyze a Message Parsing (XMP) file. Parsing files read incoming raw event data and create name-value pairs, allowing you to establish mappings even before the data mapping process. This improves overall mapping efficiency.

Note: The Common Event Grammar (CEG) names are not enforced for event parsing, allowing additional flexibility in creating name/value pairs. The CEG fields are available for selection, but the field names and values are not limited to CEG values.

Creating or editing an XMP file has the following steps:

1. Opening the parsing file wizard.
2. Providing file details, including file name, logname, and support information.
3. Locating sample events for file testing and construction.
4. Setting global values that will apply to all events parsed by the file.
5. Creating or editing prematch strings to begin event parsing
6. Selecting prematch filters for parsing filter attachment
7. Creating or editing parsing filters to complete event parsing.
8. Analyzing and saving the new or edited XMP file.

More information:

Define File Details

Load Sample Events

Analyze the XMP File

Create a Prematch Filter

Add Global Fields

Open Parsing File Wizard

To create a message parsing rule, or edit an existing one, you must open the parsing file wizard.

To open the parsing file wizard

1. Click the Administration tab, and then the Library subtab.
2. Click the arrow beside the Event Refinement Library folder to expand it, and then select the Mapping and Parsing folder.

   The product integration buttons appear in the details pane.

3. Click New Message Parsing Rule:

   The Parsing File Wizard opens.

   When using the wizard:

   • Click Save to save without closing the wizard.
   • Click Save and Close to save the file and close the wizard.
   • Click Reset to restore the wizard display to the last-saved settings.

Define File Details

You can add new parsing file details, including name, source, and reference information. Newly created or edited files are displayed in the User folder in the Mapping and Parsing area.

To add new parsing file details

1. Open the parsing file wizard.
2. Specify Parsing File Information area as outlined in the following substeps:
   1. Type a name for the file. The file name is required, and cannot contain the characters: / \ : * ? “ < > ^;'`,&{}[]. or |.
   2. Type the source logname to identify the logname of the event type you wish the file to parse. The auto-complete feature presents available lognames as you type. The logname you choose will be displayed in the event_logname field of the refined event.
   3. Add a description for reference if needed.
3. (Optional) Add Support Information for reference as outlined in the following substeps.
   1. Click Add Product in the Support Information area.

      A new support information row appears.

   2. Click the New Product or New Version text to enable entry fields, and type the product/version information you want.
4. Click the appropriate arrow to advance to the wizard step you want to complete next, or click Save and Close.

   If you click Save and Close, the new file appears in the Parsing File User folder, otherwise the step you choose appears.

Load Sample Events

You can provide sample events to use in testing the new XMP file by searching through the event log store or by accessing a log file. Sample events provide a template against which you can test the parsing file as you construct it in the other wizard steps. You can also use sample events to test the parsing output in the final step of the wizard.

To provide sample events

1. Open the parsing file wizard and advance to the Load Events step.

   The Load Events screen appears.

2. Select the Log Store or Log File radio button in the Find Sample Events area.
   • If you select Log Store:
     1. Select the sample event source type you want from the Parsing Column drop-down menu. Choose result_string for WMI event sources, or raw_event for syslog event sources.
     2. Select the query you want to use to provide sample events using the Query Tag Filter and Query List.

     The query appears, displaying sample events you can use to test parsing as you advance through the wizard.

     Note: You can use any available or custom query to locate sample events. If you plan to use a custom query, we recommend that you create and test it before beginning the message parsing file design process. We recommend using a sample event file with less than 1500 events for ease of analysis.

   • If you select Log File, browse to find the log file you want, and click Upload.

     Events from the log file appear in the Sample Events pane. You can use the events to test parsing as you advance through the wizard.

     Note: The wizard assumes that each line in the file is an event. Multiple line events are not supported.

3. Click the appropriate arrow to advance to the wizard step you want to complete next.

   If you click Save and Close, the new file appears in the Parsing File User folder, otherwise the step you choose appears.

Add Global Fields

You can add global fields, which are static pairs that match a field name with a specific value. The parsing process adds the global fields to all parsed events, so they are best used for default values such as the ideal model.

To add global fields

1. Open the parsing file wizard and advance to the Global Fields step.

   The Global Fields screen appears.

2. Click Add Global Field in the Global Fields area.

   A new global field row appears in the fields table, displaying New Global Field and New Value entries.

3. Click the New Global Field text to enter the name information you want. The auto-complete feature presents available CEG field names as you type. You can click one to select it, or type a non-CEG field name.
4. Click the New Value text to enter the name information you want.
5. (Optional) Repeat steps 2-4 to add additional global fields as needed.
6. Click the appropriate arrow to advance to the wizard step you want to complete next, or click Save and Close.

   If you click Save and Close, the new file appears in the Parsing File User folder, otherwise the step you select appears.

Create a Prematch Filter

You can create a prematch filter to help the XMP file narrow its search for event information you want to parse. The prematch filter identifies a selected text string to narrow the event selection process, which is then completed by parsing filters. If you consider the parsing file as a funnel, the prematch filter forms the mouth and the parsing filter is the spout.

The more complete your prematch filtering is, the more efficient your parsing process is. This is because narrow prematch categories held reduce the processing effort required to parse events.

For example, if you wanted to parse access attempt events, you might create a prematch filter that searches for the text "login", and add appropriate parsing filters to that prematch filter.

Note: Deleting a prematch filter also removes its associated parsing filter or filters.

To create a prematch filter

1. Open the parsing file wizard and advance to the Match and Parse step.

   The wizard displays any existing prematch filters in the Prematch Filters list. Each one displays the number of prematches to any sample events in parentheses beside it.

2. Click Add a Prematch String at the top of the Prematch Filters list, or select a prematch filter to edit.

   Note: To select a prematch filter, type the first few characters of the prematch string in the Search field. All the prematch strings matching the entered characters are displayed. Within the resulting matching prematch strings, you cannot use the up-down arrows to move a prematch string.

3. Type the text you want the filter to search for in the Prematch String entry field.

   Any sample events that match the text you enter immediately appear, along with the number of matched events found and parsed.

4. (Optional) Click Add prematch based on unmatched events to show all unmatched sample events.

   Any sample events that are currently unmatched appear in the Events area for easy reference in creating a new prematch filter.

5. (Optional) Add or edit additional prematch filters as needed.
6. Set the order in which you want the parsing process to search for prematches, using the up-down arrows beside the Prematch Filters list. Setting prematch filters that match more events higher in the priority list improves the efficiency of your parsing process.
7. Click the appropriate arrow to advance to the wizard step you want to complete next, or click Save and Close.

   If you click Save and Close, the new file appears in the Parsing File User folder, otherwise the step you choose appears.

Create a Parsing Filter

You can create a parsing filter to define how the XMP file parses event data. Each parsing filter is attached to a prematch filter. After the parsing process locates a prematch string, it uses each parsing filter attached to that prematch in turn to locate its specified information. The parsing process returns the first positive match it makes.

When you click the Add a Parsing Filter button in the Match and Parse step of the Message Parsing wizard you start the Parsing File Filter wizard. To create effective parsing filters you need a good understanding of the regular expression syntax.

To create a parsing filter

1. Open the Parsing File Filter wizard, and type a filter name and optional description in the Filter Details page.
2. Click Add new to add a static field value that you want to appear in all events parsed by the filter.

   A static field row appears, displaying New Field and New Value cells.

3. Type an entry in the New Field cell, and type an entry in the New Value cell. The auto-complete feature narrows available CEG field names as you type in the New Field cell, and presents a menu of choices.
4. (Optional) Repeat steps 2-3 to add static field values as needed.
5. Advance to the Regular Expression step.

   The Parsing Expression Testing window opens, displaying any current regular expression. Immediately below the regular expression is the Event pane. This area shows one or more sample events, if you previously loaded sample events. The wizard can test these events against the regular expression as you build it.

6. Click Add or Remove Tokens from Library to display a list of predefined regular expressions you can add for use in the current filter. Select the tokens you want to add and click OK to add them to the Parsing Tokens list.
7. (Optional) Click New Regular Expression Token to create a Parsing Token, and enter its regular expression syntax in the Token Details pane. You can now create custom expressions for your environment. You can add a custom token to your local library by clicking Add Selected Token to the Library at the top of the Parsing Tokens pane.

   Note: When you create a new datetime token, select the 'Treat as a datetime value' check box to enter a format for parsing the time value. This value does not affect the display format.

8. Add regular expression statements for the filter in the Regular Expression entry field. You can drag and drop expressions from the Parsing Tokens list. You can also type or edit the expression directly in the Regular Expression entry field.

   Note: Selecting a token in the Parsing Tokens list displays its regular expression syntax in the Token Details pane. You can view the parsing token mapping in a given rule to repeat it in other parsing rules.

9. (Optional) Select the Dynamic Name/Value Pairs checkbox if your target events include key pairs you want to display. (Optional) If you want to use dynamic parsing, enter a dynamic parsing expression in the dynamic pairs entry field. For example, enter:

   (_PAIR_KEY_)=(_PAIR_VALUE_);
   

   Any pairs separated by an equal sign and spaced by a semicolon appear. You can enter more expressions to find pairs displayed in other formats.

10. Preview how the file parses the sample events using the Event and Parsed Event panes. As you modify the parsing filter regular expression, parsed portions of the sample event are highlighted in blue text and dynamically parsed pairs appear in green. You can verify the effectiveness of the parsing.
11. (Optional) Change the sample event for additional testing by using the back and forward arrows under the Event pane to move through the available sample events.
12. Click Save and Close when you are satisfied with the regular expression. You can use Reset to return the regular expression to its initial state.

    The Parsing File Filter wizard closes, returning you to the Match and Parse step of the Parsing File wizard.

More information:

Dynamic Parsing

Parsing Tokens

Add a Custom Token to the Library

Dynamic Parsing

You can use dynamic parsing, which allows the display of multiple, unaltered name-value pairs that already exist in the raw event. Unlike normal parsing where each parsed token can be allotted to a CEG field or a user-defined field, the name part of the name/value pair becomes the field and cannot be assigned to any CEG field or user defined field. Dynamic parsing is useful where applications or formats record event data in key pairs that you wish to protect from change, not parsed into CEG names or other values. It also improves parsing performance in the cases where it is applicable.

The regular expression which allows dynamic parsing contains four elements:

1. A pair key indicator "(_PAIR_KEY_)"
2. A pair value indicator "(_PAIR_VALUE_)"
3. A key-value separator between the pair and key value
4. A pair separator between the whole expression and the next expression.

The separators you use must match the structure of the event source you are parsing. If your event source uses a comma as a separator, your regular expression must do as well.

Example

(dest_objectclass)=(ServerE);

In this example the key-value separator is "=" and the pair separator is ";"

Using this expression after other regular expressions allows the XMP file to locate and display any key pairs that appear in parsed events.

Parsing Tokens

A parsing token is a regular expression template that you can use to build parsing filters. CA User Activity Reporting Module includes a parsing token library that contains predefined parsing tokens. For example, the _IP_ token sets the regular expression that parses the typical IP address format. When you want a parsing filter to extract an IP address you can insert the _IP_ token into the filter rather than constructing the full regular expression language each time.

You can also create your own custom parsing tokens, and add them to the local library, or export them for use in another CA User Activity Reporting Module environment. If you want to export a custom token, add it to the library first. You can also import custom tokens from another CA User Activity Reporting Module environment to create parsing tokens in a test environment and move them to a live environment.

Datetime Token Values

CA User Activity Reporting Module supports various syntax options for datetime parsing tokens. You can use these options, in the parsing file datetime format, to customize your datetime stamp appearance.

Each datetime token is composed of one of the following:

• An ordinary character (neither '%' nor a white-space character), which is displayed as entered: a colon to separate time values, for example.

  or

• A conversion specification. Each conversion specification is composed of a '%' character followed by a conversion character which defines the display output: %m to display the month, for example.

CA User Activity Reporting Module supports the following conversion specifications:

%a or %A

Displays the local weekday name, in full or abbreviated form. On Windows, this specification is available in US English only.

%b or %B or %h

Displays the local month name, in full or abbreviated form. On Windows, this specification is available in US English only.

%c

Displays the local date and time.

%C

Displays the century number (0-99).

%d or %e

Displays the day of the month (1-31).

%D

Displays the American style date: Month/Day/Year - the equivalent of entering %m/%d/%y.

Note: The syntax %d/%m/%y is used in Europe. The ISO 8601 standard format is %Y-%m-%d.

%H

Displays the hour on a 24-hour clock (0-23).

%I

Displays the hour on a 12-hour clock (1-12).

%j

Displays the day number of the year (1-366).

%m

Displays the month number (1-12).

%M

Displays the minute (0-59).

%n

Inserts an arbitrary whitespace.

%p

Displays the local equivalent of AM or PM, if any.

%r

Displays the 12-hour clock time: Hour:Minute:Second AM/PM - the equivalent of entering %I:%M:%S %p. If t_fmt_ampm is empty in the local LC_TIME section then the behavior is undefined.

%R

Displays the 24-hour clock time: Hour:Minute - the equivalent of entering %H:%M.

%S

Displays the second (0-60 - 60 can occur for leap seconds).

%t

Displays an arbitrary whitespace.

%T

Displays the 24-hour clock time: Hour:Minute:Second - the equivalent of entering %H:%M:%S.

%U

Displays the week number. Sunday is the first day of the week (0-53). The first Sunday of January is the first day of week 1.

%w

Displays the weekday number (0-6) with Sunday = 0.

%W

Displays the week number with Monday the first day of the week (0-53). The first Monday of January is the first day of week 1.

%x

Displays the date, using the local date format.

%X

Displays the time, using the local time format.

%y

Displays the year in the current century (0-99). When a century is not specified, values in the range 69-99 refer to years in the twentieth century (1969-1999); values in the range 00-68 refer to years in the twenty-first century (2000-2068).

%Y

Displays the year, including century (for example, 1991).

%z

Displays an RFC-822/ISO 8601 standard time zone specification. This specification is not available on Windows.

The default CA User Activity Reporting Module datetime token format is:

%d/%b/%Y:%H:%M:%S %z

Add a Custom Token to the Library

You can add custom parsing tokens to the token library, making them available for other users. For example, if you create a custom token during the message parsing file creation process, and it would be useful for other parsing, you can add it to the library for re-use.

The following procedure assumes that you add tokens during the creation of parsing files or filters.

To add a custom parsing token to the library

1. Open the Message Parsing wizard, and advance to the Match and Parse step.
2. Open the Parsing File Filter wizard, and advance to the Regular Expression step.
3. Click New Regular Expression to create a Parsing Token, and enter its regular expression syntax in the Token Details pane.
4. Select the new parsing token, and click Add selected token to the library.

   A confirmation dialog appears

5. Click Yes.
6. (Optional) Click Add or Remove Tokens from Library to view the new token.

   The parsing tokens library dialog appears, showing custom tokens in black and predefined tokens in green.

Remove a Custom Token from the Library

You can remove unneeded or obsolete custom tokens from the token library. Predefined tokens cannot be removed.

To remove custom tokens from the library

1. Click the Administration tab, and then the Log Collection subtab.

   The Log Collection folder list appears.

2. Click the arrow next to the Event Refinement Library folder to expand it, and then select the Mapping and Parsing folder.

   The product integration buttons appear in the details pane.

3. Click New Message Parsing Rule:

   The Parsing File Wizard opens.

4. Advance to the Match and Parse step.
5. Select any prematch filter, and click Edit or click Add a Parsing Filter at the top of the Parsing Filters list.

   The Parsing File Filter wizard appears.

6. Advance to the Regular Expression step.
7. Click Add or Remove Tokens from Library.

   The parsing tokens library dialog appears, showing custom tokens in black and predefined tokens in green.

8. Select the custom token or tokens you want to remove, and click Remove Selected Token from the Library.

   A confirmation dialog appears.

9. Click yes, then Click OK.

Import Parsing Tokens

You can import parsing tokens to add custom parsing tokens created on another management server to your current server, from a test environment to your live environment, for example.

To import parsing tokens

1. Click the Administration tab, and then the Log Collection subtab.

   The Log Collection folder list appears.

2. Click the arrow next to the Event Refinement Library folder to expand it, and then select the Mapping and Parsing folder.

   The product integration buttons appear in the details pane.

3. Click New Message Parsing Rule:

   The Parsing File Wizard opens.

4. Advance to the Match and Parse step.
5. Select any prematch filter, and click Edit or click Add a Parsing Filter at the top of the Parsing Filters list.

   The Parsing File Filter wizard appears.

6. Advance to the Regular Expression step.
7. Click Import User Tokens at the top of the Parsing Tokens pane.

   The Import File dialog appears.

8. Browse to find the tokens (.tok) file you want to import, and click OK.

   A confirmation dialog appears.

9. Click Yes if you want to import the file, overwriting any other user tokens in the library.

Export Parsing Tokens

You can export parsing tokens that you have added to the token library to move custom parsing tokens created on the current management server to another server. For example, you could move your custom tokens from a test environment to your live environment.

To export parsing tokens

1. Click the Administration tab, and then the Log Collection subtab.

   The Log Collection folder list appears.

2. Click the arrow next to the Event Refinement Library folder to expand it, and then select the Mapping and Parsing folder.

   The product integration buttons appear in the details pane.

3. Click New Message Parsing Rule:

   The Parsing File Wizard opens.

4. Advance to the Match and Parse step.
5. Select any prematch filter, and click Edit or click Add a Parsing Filter at the top of the Parsing Filters list.

   The Parsing File Filter wizard appears.

6. Advance to the Regular Expression step.
7. Click Export User Tokens at the top of the Parsing Tokens pane.

   A download location dialog appears.

8. Select the location where you want to save the exported file, and click Save.

   The exported file is saved in your chosen location.

Analyze the XMP File

You can use the Message Parsing utility to analyze your new or edited file and determine how effective the parsing file is against the sample events. Analysis lets you make modifications to improve the efficacy of the file before saving it.

The utility analyzes an XMP file against your selected sample event set using the following process:

1. Locating all events containing the prematch strings defined in the XMP file. The utility runs a separate search for each prematch string, finding all events containing that string.
2. Finding the first parsing filter for each of the prematched events that can parse the event into tokens.

To analyze the XMP file

Open the parsing wizard and advance to the Parsing Analysis step. The wizard displays the number of matches for the prematch strings and filters. The more matches you have, the more efficient the new or edited XMP file will be. This also allows you to determine if there is any significant information that remains unparsed.

The XMP analysis can take some time to process if the XMP file and the number of sample events are both large. It should not usually take over a minute. You can cancel this process if it is taking too long and then re-analyze using a smaller number of events.

When you create a new rule, it is saved as version 1.0. If you later rule edit the rule, a separate copy of the rule is stored as a new version. You can view earlier versions, and apply or copy them as needed.

How to Create a Data Mapping File

You can use the mapping file wizard to create and edit Data Mapping files, which convert native events into refined events by mapping the parsed text string or field/value pairs to CEG-compatible fields. The mapping file wizard allows you to create and edit various types of mapping to accomplish this.

The process of creating or editing a DM file contains the following steps:

1. Opening the mapping file wizard.
2. Providing file details.
3. Locating and adding sample events using parsing files.
4. Setting direct mappings as needed.
5. Setting function mappings as needed.
6. Setting conditional mappings as needed.
7. Setting block mappings as needed.

   Note: You can set direct or function mappings using block mappings. They are an alternative to setting mappings with steps 4 and 5.

8. Analyzing and saving the completed DM file.

When creating a DM file, you should consider the data mapping priorities of the file itself, as well as the individual mapping types within the file. The completed DM file checks event information in the order of the mapping type screens (steps 4-7 in the wizard). If duplicate mapping types exist, the last value the DM file finds is the one assigned.

For example, if a DM file finds a Direct mapping for a given native event value, and then a different Conditional mapping for the same value, the refined event uses the Conditional mapping result.

Duplicate mappings within a given mapping type are handled differently, depending on the type:

• Direct Mappings and Function mappings -- The DM file uses the last matching value found. If a duplicate function mapping is found, the last function will be the one called. For example, you could set a duplicate mapping to call a second function if the first was not found or did not function as expected.
• Conditional mappings and Block mappings -- The DM file applies the first value found, and stops searching. To improve performance, you should place more common conditions earlier in the file for both these mapping types.

More information on the design implications of the mapping order is included in the individual mapping type procedures.

Open Mapping File Wizard

To create a new DM file, or edit an existing one, you must open the mapping file wizard.

To open the mapping file wizard

1. Click the Administration tab, and then the Library subtab.
2. Click the arrow beside the Event Refinement Library folder to expand it, and then select the Mapping and Parsing folder.

   The product integration buttons appear in the details pane.

3. Click New Mapping File:

   The Mapping File Wizard appears.

   When using the wizard:

   • Click Save to save the file without closing the wizard.
   • Click Save and Close to save the file and close the wizard.
   • Click Reset to restore the wizard display to the last-saved settings.

More information

Set Block Mappings

Perform Mapping Analysis

Set Concat Function Mapping

Provide File Details

Provide file details for a new DM file. You can save a subscription file as a custom file under a different name.

To provide mapping file details

1. Open the mapping file wizard.
2. Enter a name for the DM file. The file name is required, and cannot contain the characters: / \ : * ? “ < > ^;'`,&{}[]. or |.
3. Select the Parsing File name and version you want to use to parse the sample events from the Parsing File drop-down list.

   The log name field is automatically populated with the name of the parsing file you enter.

4. (Optional) Enter a description.
5. (Optional) Click Add Product in the Support Information area to enter product name and versions for reference.
6. Click the appropriate arrow to advance to the wizard step you want to complete next, or click Save and Close.

   If you click Save and Close, the new file appears in the Mapping File User folder, otherwise the step you select appears.

Provide Sample Events

You can use the Mapping File wizard to search for sample events to use in analyzing the DM file. You can search through the event log store or provide sample events directly from a log file. Sample events provide a template against which to test the mapping output in the final step of the wizard.

To provide sample events

1. Open the mapping file wizard and advance to the Sample Events step.

   The Sample Events screen appears.

2. Select the Log Store or Log File option button in the Find Sample Events area.
3. If you select Log Store:
   1. Select the sample event source type you want from the Parsing Column drop-down menu. Select result_string for WMI event sources, or raw_event for syslog event sources.
   2. Select the query you want to use to provide sample events, using the Query Tag Filter and Query List.

      The query appears, displaying the sample events.

      Note: You can use any available or custom query to locate sample events. If you plan to use a custom query, we recommend that you create and test it before beginning the data mapping file design process.

4. If you select Log File:
   1. Browse to find the log file you want, and click Upload.

      Events from the log file appear in the Sample Events pane.

      Note: The wizard assumes that each line in the file is an event. Multiple line events are not supported.

   2. Click Extract Dynamic Fields, if your sample log file contains dynamic pair values you want to include in the parsed sample.
5. Click the appropriate arrow to advance to the wizard step you want to complete next, or click Save and Close.

   If you click Save and Close, the new file appears in the Mapping File User folder, otherwise the step you select appears.

More information:

Dynamic Parsing

Set Direct Mappings

Direct mappings set 1-1 correspondences between a native event and a single refined event value. Thus, it is best to use direct mappings only for default values, or common values that rarely change, such as the ideal_model field.

A mapping can be set to derive a refined event value in the following ways:

Text value

Sets specific text for a specific CEG field. This value appears each time an appropriate event is mapped. For example, setting the CEG ideal_model field to "Firewall" results in the ideal_model field displaying "Firewall" for all rules that contain that mapping.

Field value

Sets a raw event field whose content is included for a specific CEG or parsed field. A field value is distinguished from a text value by prefacing the value with a dollar sign, $. For example, setting the CEG event_logname field to "$Log" results in any event mapped displaying whatever text appears in the native event Log field.

To set direct mappings

1. Open the mapping file wizard, enter a name and select a Logname for the mapping file, and advance to the Direct Mappings step.

   The Direct Mappings screen appears, displaying current or default mappings. The Name column shows the CEG or parsed field name. The Value column shows either a text value or a field value.

   Note: Select a parsing file in the Provide Sample Events step for parsed field values to appear.

2. Click Add Direct Mapping to add a new mapping entry at the bottom of the table and then select it, or select a current direct mapping to edit.

   The direct mappings for the field, if any, appear in the Mapping Details area.

3. Select a CEG field or parsed event field, if available, to map to from the Field drop down menu. When you begin typing, the auto-complete feature narrows the list of available CEG fields.
4. Enter a new value in the Add Value entry field, and click Add Direct Mapping next to it. Precede the value with "$" to denote a field value rather than a text value.

   The value appears in the Selected Fields area.

5. (Optional) You can enter multiple direct mappings for a single field, using the up and down arrows to set the order in which the DM file considers them. The refined event displays the last direct mapping located by the DM file.

   Note: Adding multiple values decreases performance of the mapping, so you use this feature conservatively.

6. (Optional) Use the shuttle control to move unneeded values to the Available Fields area to prevents them from being considered for the current mapping.
7. When you have added all the direct mappings you want, click the appropriate arrow to advance to the wizard step you want to complete next, or click Save and Close.

If you click Save and Close, the new file appears in the Mapping File User folder, otherwise the step you select appears.

Set Function Mappings

Function mappings link a CEG field to a value using a function to retrieve or define the refined event information that appears in the refined event. All function mappings consist of a CEG field name, a predefined or class field value, and the function.

For example, a function mapping can concatenate a series of native event values to a single CEG field using the concat function.

If there are duplicate function mappings, the DM file uses the last one it finds. You could set a duplicate mapping to call a second function if the first was not found or did not function as expected.

To set function mappings

1. Open the mapping file wizard, enter a name and select a Logname for the mapping file, and advance to the Function Mappings step.

   The Function Mappings screen appears, displaying current or default mappings. The Name column shows a CEG or parsed field, the Function column the current linking function, and the Value column a text or field value.

   Note: Select a parsing file in the Provide File Details step for parsed field values to appear.

2. Click Add Function Mapping to add a new mapping entry, or select a current mapping to edit.

   The mapping entry appears in the Mapping Details pane.

3. Select a CEG field to map to from the Field drop-down menu. When you begin typing, the auto-complete feature narrows the list of available CEG fields.
4. Select a function to use for the mapping from the Function drop down menu.
5. Note: The concatenate (concat) function works differently than the others, because you specify multiple target values. Enter a target value for the mapping in the Add Value entry field and click the Add Value button next to it. You can precede the value with "$" to denote a field value rather than a specific value.

   The value appears in the Selected Fields area.

6. (Optional) You can enter multiple mappings for a single field, using the up and down arrows to set the order in which the DM file considers them.

   Note: Adding multiple values decreases performance of the mapping, so use stand-alone function mappings only if necessary.

7. (Optional) Use the shuttle control to move unneeded values to the Available Fields area to prevent them from being considered for the current mapping.
8. When you have added all the function mappings you want, click the appropriate arrow to advance to the wizard step you want to complete next, or click Save and Close.

   If you click Save and Close, the new file appears in the Mapping File User folder, otherwise the step you select appears.

Set Concat Function Mapping

A Concat function mapping is a type of function mapping. Unlike other function mappings, which specify one target field or value, the concat function specifies multiple mapping targets, which it concatenates into one CEG field.

You can use the Data Mapping wizard to create concat function mappings. Because concat mappings are different from other function mappings, the procedure for creating them is somewhat different.

To set a concat function mapping

1. Open the mapping file wizard, enter a name and select a Logname for the mapping file, and advance to the Function Mappings step.

   The Function Mappings screen appears, displaying current or default mappings. The Name column shows a CEG field, the Function column the current linking function, and the Value column a text or field value.

2. Click Add Function Mapping to add a new mapping entry.

   The mapping entry appears in the Mapping Details pane.

3. Select a CEG field to map to from the Field drop-down menu.
4. Select the concat function from the Function drop-down menu.

   The Format and Value fields appear.

   Note: The value for the concat function is displayed as {…} in the Function Mappings pane. This means that there is a set of values instead of one value.

5. (Optional) Enter a specifier in the Format field to control the placement of the target fields. The format specifier, %s, indicates a field position. Anything other than %s is considered static supporting data to be included in the final table collector field. For example, to separate two target fields with a colon, enter "%s:%s" in the Format field.
6. Click Add Concat Value in the Concat Values area to add a target input/value pair.
7. Enter a value in the Add value entry field, and click Add Value.

   The value appears in the Selected Fields area.

8. Repeat steps 6 and 7 to add additional values to concatenate. You must add at least two target values.
9. When you have added all the concat mappings you want, click the appropriate arrow to advance to the wizard step you want to complete next, or click Save and Close.

   If you click Save and Close, the new file appears in the Mapping File User folder, otherwise the step you choose appears.

Set Conditional Mappings

Conditional mappings link a CEG field to different possible results, allowing you to set default and conditional values for a given field. For example, you could use conditional mappings to map success or failure values, or to identify event sources by name or group.

Conditional mappings assign a default value and one or more conditional values to a given CEG field. You can set qualifications for each conditional value. If an event matches those qualifications, the appropriate conditional value is assigned to the chosen field. Otherwise the refined event field displays the default value.

If there are duplicate conditional mappings, the DM file uses the first one it finds, and considers no further mappings. To improve performance, place more common conditions first.

Note: Stand-alone conditional mapping is slower than block mapping. We recommend that you used it only when necessary.

To set conditional mappings

1. Open the mapping file wizard, enter a name and select a Logname for the mapping file, and advance to the Conditional Mappings step.

   The Conditional Mappings screen appears, displaying any current default mappings. The Field column shows the CEG or parsed field name, and the Value column shows the current default value.

   Note: Select a parsing file in the Provide File Details step for parsed field values to appear.

2. Click Add Conditional Mapping in the Conditional Field Mappings list, and select the new row.

   The Mapping Details pane appears, displaying the Field drop-down list and Value shuttle control.

3. Select the CEG field you want to map to from the Field menu. When you begin typing, the auto-complete feature narrows the list of available CEG fields.
4. Enter the default mapping you want in the Add Value entry field, and click Add Value to display it in the Selected Fields pane. You can remove unwanted values by moving them to the Available Fields pane.
5. Click Add Conditional Value in the Conditional Values list.

   A new value appears.

6. Select the New Value text to highlight it and change the name.

   The new name appears in the list, and the filters dialog appears in the details pane.

7. Construct a filter to define the conditional value. For example, you could build one or more filters to link the event_source_address field to IP addresses, identifying event sources with a geographical or other business group.
8. When you have added all the conditional mappings you want, click the appropriate arrow to advance to the wizard step you want to complete next, or click Save and Close.

   If you click Save and Close, the new file appears in the Mapping File User folder, otherwise the step you select appears.

More information:

Using Advanced Filters

Set Block Mappings

Block mappings link a selected condition to a defined series of mappings, allowing you to create a cascade of mappings triggered by that condition. A given block mapping can use any combination of direct or function mappings. Both types of internal block mapping work exactly as they would for stand-alone mappings.

You can create as many blocks as you need for a single mapping file. Each one includes a name and a condition.

If there are duplicate mappings in a given block, the DM file will use the first one it finds, and consider no further mappings. To improve performance, you should place more common conditions first.

To set block mappings

1. Open the mapping file wizard, enter a name and select a Logname for the mapping file, and advance to the Block Mappings step.

   The Block Mappings screen appears, displaying any current block mappings.

2. Click Add Block Mapping in the Block Mappings pane.

   A new block appears in the Block Mappings list.

3. Select the New Block text.

   The Block Definition pane opens, displaying Step 1. Define a Condition

4. Enter a block name, and construct a filter to define the condition for this block. For example, you could define event_result to equal "S" which would invoke the block mappings when a success is detected for the event process.
5. Click the Step 2 bar, and enter any direct mappings you want, using the same process as the stand-alone mapping step.
6. Click the Step 3 bar, and enter any function mappings you want, using the same process as the stand-alone mapping step.
7. When you have added all the block mappings you want, click the appropriate arrow to advance to the wizard step you want to complete next, or click Save and Close.

   If you click Save and Close, the new file appears in the Mapping File User folder, otherwise the step you choose appears.

More information:

Using Advanced Filters

Perform Mapping Analysis

You can use the mapping wizard to analyze a data mapping file, allowing you to test and make changes to increase the efficiency of your mapping file. The sample events are tested against the DM file and the results are then validated against the CEG.

To perform mapping analysis, click the Mapping Analysis step of the Mapping File Wizard. The wizard displays a table, showing the parsing result of the sample events you entered in the Sample Event step.

The completed DM file saves your mappings and considers event information in the order of the mapping type screens (wizard steps 4-7). If duplicate mappings exist, the last value the DM file finds is the one assigned. For example, if a DM file finds a direct mapping for a given native event value, and then a different conditional mapping for the same value, the refined event displays the conditional mapping result. More information on the design implications of the mapping order is included in the individual mapping type procedures.

When you create a new rule, it is saved as version 1.0. If you later rule edit the rule, a separate copy of the rule is stored as a new version. You can view earlier versions, and apply or copy them as needed.

Event Forwarding Rules Tasks

Event forwarding rules allow you to select CA User Activity Reporting Module events to forward to remote listeners in outside applications or systems. You can use forwarding rules to identify the events you want to forward, set when they are transmitted, and control how they are received. When an incoming event matches a forwarding rule filter, CA User Activity Reporting Module creates a copy of the event and forwards it. The event is still recorded in the event log store.

Event forwarding rules tasks are carried out from the Log Collection area of the CA User Activity Reporting Module interface. You can create, edit, and delete event forwarding rules. You can also import or export event forwarding rules.

More information:

How to Create Event Forwarding Rules

How to Create Event Forwarding Rules

You can use event forwarding rules to send CA User Activity Reporting Module events to outside applications. For example, you could send events to CA NSM using syslog. Event forwarding rules allow you to set criteria for the events you want to forward, and set one or more receivers.

The process of creating event forwarding rules, using the forwarding rule wizard, has the following steps:

1. Opening the forwarding rule wizard.
2. Setting a name and optional description for the rule.
3. Creating simple and advanced filters to identify events to forward.
4. Setting rule attributes including forwarding destination and CEG fields to include in the forwarded event.

More information:

Create an Advanced Event Filter

Name Forwarding Rule

Set Forwarding Rule Attributes

Create a Simple Event Filter

Using Advanced Filters

Open Forwarding Rule Wizard

To create a forwarding rule, or edit an existing one, open the forwarding rule wizard.

To open the forwarding rule wizard

1. Click the Administration tab, and then click the Library subtab.
2. Select the Forwarding Rules folder.

   The forwarding rule buttons appear in the details pane.

3. Click New Forwarding Rule:

   The Forwarding Rule Wizard opens.

   When using the wizard:

   • Click Save to save the rule file without closing the wizard.
   • Click Save and Close to save the rule file and close the wizard.
   • Click Reset to restore the wizard display to the last-saved settings.

Name Forwarding Rule

You must name a forwarding rule. You can also enter description information for reference.

To name a forwarding rule

1. Open the forwarding rule wizard.
2. Enter a name for the rule. The name is required, and cannot contain the characters: / \ : * ? < >;'`,&{}[]. or |.
3. (Optional) Enter description information about the rule for reference.
4. Advance to the Filtering Step.

Create a Simple Event Filter

You can create simple filters to set search parameters for common CEG fields. For example, you could set the Ideal Model field to "Content Management" to identify all events with that value in the Ideal Model CEG field. Simple filters are used by many features, including queries, suppression and summarization rules, and event forwarding rules.

To create a simple filter

1. Select the check box for Ideal Model, or any of the Event fields you want to define, and select a value from the drop-down list, or enter the value you want in the text entry field.
2. (Optional) If you are creating a query filter, select any of the Source, Destination or Agent field check boxes, and enter the value you want in the text entry field.
3. Repeat steps 1-2 to add additional simple filters.
4. Click Save when you have added all the simple filters you want.

More information

Using Advanced Filters

Create an Advanced Event Filter

Using Advanced Filters

You can use SQL-based advanced filters to qualify any function that queries the event log store, including narrowing queries, or adding additional qualifications to simple filters. The Advanced Filters interface helps you create the appropriate filter syntax by providing a form for entering logic columns, operators and values according to your filtering requirements.

Note: This section contains a brief overview of the SQL terms used in advanced filters. To use advanced filters to their full potential you need a thorough understanding of SQL and the Common Event Grammar.

The following SQL terms join multiple filter statements:

And

Displays the event information if all the joined terms are true.

Or

Displays the event information if any of the joined terms are true.

Having

Refines the terms of the main SQL statement by adding a qualifying statement. For example, you could set an advanced filter for events from specified hosts, and add a "having" statement to return only events of a specified severity level from those hosts.

The following SQL operators are used by advanced filters to create the basic conditions:

Relational Operators

Include the event information if the column bears the appropriate relation to the value you enter. The following relational operators are available:

• Equal to
• Not Equal to
• Less than
• Greater than
• Less than or equal to
• Greater than or equal to

For example, using Greater than would include the event information from your chosen column if its value is greater than the value you set.

Like

Includes the event information if the column contains a pattern you enter, using % to set the pattern you want. For example, L% would return any values beginning with L, %L% would return any values with L included as neither first nor last letter.

Not like

Includes the event information if the column does not contain the pattern you specify.

In set

Includes the event information if the column contains one or more of the values in the quote-delineated set you enter. Multiple values in the set must be comma-separated.

Not in set

Includes the event information if the column does not contain one or more of the values in the quote-delineated set you enter. Multiple values in the set must be comma-separated.

Matches

Includes any event information that matches one or more of the characters that you enter, allowing you to search for key words.

Keyed

Includes any event information that is set as a key value during Report Server configuration. You can use key values to set business relevance or other organizational groups.

Not Keyed

Includes any event information that is not set as a key value during Report Server configuration. You can use key values to set business relevance or other organizational groups.

More information:

Create an Advanced Event Filter

Create an Advanced Event Filter

Advanced filters are used by many features, including query creation, report scheduling, and local and global filters.

To create an advanced filter

1. Click New Event Filter.

   The first row of the event filter table becomes active, and its Logic and Operator columns are populated with the default values "And" and "Equal to" respectively.

2. (Optional) Click the Logic cell and change the logic value as needed.
3. Click the Column cell, and select the event information column you want from the drop-down menu.
4. Click the Operator cell, and select the operator you want from the drop-down menu.
5. Click the Value cell, and enter the value you want.
6. (Optional) Click the open and closed parentheses cells and enter the number of parentheses you need.
7. (Optional) Repeat steps 1 through 6 as needed to add additional filter statements.
8. Click Save when you have entered all the filter statements you want.

More information:

Using Advanced Filters

Create a Simple Event Filter

Set Forwarding Rule Attributes

Set required attributes for a forwarding rule, including forwarding exit points, CEG fields included in the forwarded event, and destination settings.

To set rule attributes

1. Open the forwarding rule wizard and advance to the Policy Attributes step.
2. Set forwarding rule actions in the Actions area:
   1. Select a syslog Facility and a syslog Severity in the appropriate drop-down lists. Any events forwarded by the rule include the syslog attributes you set.
3. Set information about CA User Activity Reporting Module event transmission in the General Information area:
   1. Select whether you want to sent the events identified by the rule before or after suppression and summarization:
   • If you select before, all incoming events are verified against the forwarding rule filters.
   • If you select after, only refined events are verified against the forwarding rule filters; Suppressed events are not forwarded, and summarized events are forwarded only as summarized, not in presummarization detail.

     Note: Choosing before has a larger effect on system performance, because the events are unrefined.

   1. Select the CEG fields you want to be displayed in the transmitted event. If you do not select a CEG field, only the raw event value is sent. If you select any CEG field, also select raw_event to forward the raw event.
4. Set forwarding destination information in the Destination area:
   1. Click Add Destination to create a destination row.
   2. Click the text in the Host column to add a destination hostname, or IP address. The IP address can be IPv4 or IPv6.
   3. Click the Port column cell to add the port number that the target application listens on.
   4. Click the text in the Protocol column to select TCP or UDP to set the transmission protocol you want to use.
   5. Repeat steps a-d to add more destinations as next.
5. Click Save or Save and Close.

   The new rule appears in the User subfolder of the Forwarding Rules folder.

About Forwarded syslog Events

The maximum syslog packet size (including PRI, Header, Tag and Content fields) is 1024 bytes, so the forwarded event may not be able to include all of the CEG name-value pairs the user has specified.

When necessary,CA User Activity Reporting Module truncates the message value to keep the length under 1024 bytes. If the forwarding rule specifies CEG fields to include in the generated syslog event, then the generated syslog event's Content field contains the specified CEG name-value pairs.

The name-value pairs have the format CEG_field_name:field_value from the event that matched the simple filter rule. The string “null” designates a null CEG field value. These CEG fields are in the order specified in the forwarding rule.

The CEG field order specified in the forwarding rule is significant. CA User Activity Reporting Module may truncate the value portion specified, but it will not truncate any CEG field names. If CA User Activity Reporting Module cannot fit the next full CEG field name and the colon and at least one byte of the associated value, then it terminates the syslog content field with the prior CEG name-value pair.

Edit a Forwarding Rule

You can edit a forwarding rule.

To edit a forwarding rule

1. Click the Administration tab, and then click the Library subtab.
2. Expand the Forwarding Rules folder, and click the folder which contains the file you want to edit.
3. Select the rule you want to edit, and click the Edit Forwarding Rule icon.

   The forwarding rule wizard appears, displaying your selected rule.

4. Change the rule as you like, and click Save and Close.

   The rule appears in the appropriate list as a new version of the edited rule.

Delete a Forwarding Rule

You can delete an unneeded forwarding rule.

To delete a forwarding rule

1. Click the Administration tab, and then click the Library subtab.
2. Expand the Forwarding Rules folder, and click the folder which contains the file you want to delete.
3. Select the rule you want to delete and click the Delete Forwarding Rule icon. The current version is selected by default. You can select an earlier version to delete from the Version pull-down list in the details pane.

   A confirmation dialog appears.

4. Click Yes.

   The deleted rule is removed from the appropriate list.

Import a Forwarding Rule

You can import a forwarding rule, allowing you to move rules from one environment to another. For example, import rules created in a test environment to your live environment.

To import a forwarding rule

1. Click the Administration tab, and then click the Library subtab.
2. Select the Forwarding Rules folder.

   The forwarding rule buttons appear in the details pane.

3. Click Import Forwarding Rule.

   The import file dialog appears.

4. Browse to find the rule you want to import, and click OK.

   The Forwarding Rule Wizard appears, displaying the details of the rule you selected.

5. Change the rule as you like, and click Save and Close. If the imported rule shares a name with a rule already in your management database, you are prompted to change the name.

   The imported rule appears in the Event Forwarding Rules user folder.

Export a Forwarding Rule

You can export a forwarding rule, allowing you to move rules from one environment to another. For example, export rules created in a test environment to your live environment.

To export a forwarding rule

1. Click the Administration tab, and then click the Library subtab.
2. Expand the Forwarding Rules folder, and click the folder which contains the file you want to export.
3. Select the rule you want to export, and then click Export Forwarding Rule. The current version is selected by default. You can select an earlier version to export from the Version pull-down list in the details pane.

   An export location dialog appears.

4. Enter or browse to the location where you want to store the exported rule, and click Save.

   An export successful confirmation dialog appears.

5. Click OK.

   The rule is exported.

   Note: If you examine the exported rule, the values for Facility and Severity are shown only numerically. You can use the wizard interface to determine the text descriptions associated with these values.