Previous Topic: Enrich OperationNext Topic: Normalize Operation


Enrich Property—Create New Event Properties From Retrieved Properties

Enrich operations begin with an <Enrich> property. The <Enrich> property looks up additional properties from an external source using current event property values and creates new event properties from the retrieved properties.

This property has the following format:

<Enrich>
   <Field input= type= outputtype= [inputtype= connectionstring= jdbcdriver= 
   query= returntype= column=]|[jclass= method=]|[cmdline=] output= />
       [<mapentry mapin= mapout=>]
</Enrich>

Note: Only the input, type, outputtype, and output attributes are required for all enrich properties. The other attributes you must use depend on the type of enrich operation you are writing. The type definition specifies the requirements for each enrichment type.

input

Defines a list of properties to enrich with information from external sources.

If are using a single multiple-column property as input, enter the column values in a comma-delimited list using the following format:

property_column_order
property

Specifies the event property name.

column

Specifies a column value from the column attribute.

order

Specifies an integer, starting with 0, indicating the order in which the specific column value is returned.

For example, if you are using a users property as input that is made up of column values firstname and lastname, the input property would read as follows:

user_firstname_0,user_lastname_1
type

Defines the type of enrichment. You can have multiple fields of the same or different types within a single <Enrich> property with no restrictions. The following are available types:

map

Matches and enriches properties using regular expressions. Map enrichment uses mapentry elements that represent an expression and an output to assign to the property if the expression is matched. These elements are read from top to bottom until a property matches an element, after which additional mapentries are not considered. The map type requires the following attributes:

  • mapin
  • mapout
jdbc

Uses property values as input parameters in a JDBC query to determine an enriched value for the properties. The jdbc type requires the following attributes:

  • inputtype
  • connectionstring
  • jdbcdriver
  • query
  • returntype

You can return multiple columns and rows from a complex JDBC query and use the column attribute in the policy to identify each of these returned columns uniquely.

Note: The jdbc enrichment type supports only the pairedlist output type. Therefore, the returned output value is always a paired comma-delimited list of values. This enrichment type does not support other output types (such as std and list).

methodcall

Uses property values as input parameters in a Java method call to determine an enriched value for the properties. Properties are treated as strings using this option and the Java method must accept a string array as its only parameter. The methodcall type requires the following attributes:

  • jclass
  • method
exe

Uses property values as input parameters in an executable to determine an enriched value for the properties. The exe type requires the following attribute:

  • cmdline
outputtype

Defines the type of output to return. The Enrich operation can return standard output, a list output, or a paired list output. The following are valid values for this attribute:

std

Indicates that the returned output is a singular value.

ref

Indicates that the given mapout value is a variable that contains the output to return.

list

Indicates that the returned output value is a comma-delimited list of values.

Example: red,blue,green

The resulting properties are referenced as follows:

xxxxxx_y

xxxxxx is the name of the output property (as designated by the output attribute)

y is the index of the list value (where 0 is the first element in the list)

For example, if the output attribute is color, red would be referenced as color_1.

pairedlist

Indicates that the returned output value is a paired comma-delimited list of values.

Example: color,red,size,large,name,fido

The resulting properties are referenced as follows:

xxxxxx_zzzzz

xxxxxx is the name of the output property (as designated by the output attribute)

zzzzz is the name of the returned property in the list (color, size, and name in the example).

For example, if the output attribute is myenrichsource, size in the example (whose value is large) is referenced as myenrichsource_size.

mapin

(map only) Defines a regular expression pattern that compares the input property value.

mapout

(map only) Defines the assigned value to the output property if the input property matches the mapin regular expression. Specify an event property for mapping to the event property's value.

inputtype

(jdbc only) Defines the value types for the input properties. Valid values are any Java primitive types such as int, string, long, and bool.

connectionstring

(jdbc only) Defines a JDBC connection string to a database instance. This string must include the database instance, name, user name, and password. The subsequent JDBC example shows use of a string.

jdbc driver

(jdbc only) Defines the JDBC driver Java class. Write the class for this attribute without the .class extension.

query

(jdbc only) Defines a SQL SELECT query that returns the value to use for the input property.

returntype

(jdbc only) Defines the value type of the value returned from the JDBC query. Valid types are any Java primitive type such as int, string, and bool. If you are using a complex JDBC query that returns multiple values, specify the return type for each value in a comma-delimited list. The order of the list must correspond to the values defined in the column attribute.

column

Defines aliases for returned columns when multiple columns of data are returned from a JDBC query. Specify a comma-delimited list of identifiers for each column value so that each piece of data is uniquely referenced in separate properties. This attribute is only required with a multiple column JDBC query.

jclass

(methodcall only) Defines the Java class full name where you run a method.

method

(methodcall only) Defines the name of the Java method that returns the value used for the input property.

cmdline

(exe only) Defines the command line that includes the full pathname and returns the value for the input property. Use substitution markers ({0}, {1}, and {2}) that are replaced with the input property values.

output

Defines the property that is assigned the output value of the enrich operation. The output property is assigned the following value for each enrichment type:

For all enrichment types, the output property can be a new or existing property.

Example: Enrich city and state output with region information

The following example maps the city and state input properties to one region output property according to regular expressions:

<Enrich>
   <Field input="city,state" type="map" output="region" outputtype="std">
       <mapentry mapin="^Cin.*,OH$" mapout="Midwest" />
       <mapentry mapin="^New York.*,NY$" mapout="East" />
   </Field>
</Enrich>

The <Enrich> property makes the following searches:

This property enriches the tags by adding the appropriate region for each of these locations.

Example: Enrich resource output with a complex JDBC query

The following example enriches an event with multiple columns and rows from a database table. This scenario is similar to the previous example with the name and description being queried from the ca_resource_department_table:

<EventClass name="OPR">
   <Enrich>
       <Field input="internal_resourceaddr" inputtype="string" type="jdbc"
       outputtype="pairedlist" column="name,desc,org"
       connectionstring="jdbc:sqlserver://server01;databaseName=mdb;
       user=nsmadmin;password=admin;"
       jdbcdriver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
       query="select name,description,organization_uuid from 
       ca_resource_department where id=?"
       returntype="string,string,string" output="department" />
   </Enrich>
</EventClass>

The <Enrich> property uses the internal_resourceaddr value as in the previous example. The query returns the corresponding name, description, and organization_uuid. The query also assigns this information to new properties using the defined column attributes as follows:

When a query returns multiple rows, the trailing number on the property represents the row number. For example, a second returned row would be labeled as department_name_1, department_desc_1, and department_org_1.