Previous Topic: Services and CA AdaptersNext Topic: Subscription


Log Storage

This section contains the following topics:

About Log Storage

Event Log Database States

Automating Backup and Restore

Data Integrity Checks

Configuring Non-Interactive Authentication for Restore

Query the Archive Catalog

Restore Auto-Archived Files

Restore–Script for Restoring Archived Databases

Manually Backing Up Archived Databases

Manually Restoring Archives to the Original Event Log Store

Manually Restoring Archives to a New Event Log Store

LMArchive–Backup/Restore Tracking

About Log Storage

You can manage two aspects of log storage through CA User Activity Reporting Module:

You can manage backups of event log databases in one of two ways:

You can query the archive catalog to identify database files to restore. You can restore databases on an as needed basis in one of two ways:

More information:

Manually Backing Up Archived Databases

Manually Restoring Archives to the Original Event Log Store

Log Storage

Manually Restoring Archives to a New Event Log Store

LMArchive–Backup/Restore Tracking

Configure Max Archive Days for Restored Archives

About Auto Archive

Example: Auto-Archiving Across Three Servers

Restore Auto-Archived Files

Restore–Script for Restoring Archived Databases

Configuring Non-Interactive Authentication for Restore

Event Log Database States

When you configure auto-archive across three servers (collection, reporting, and remote storage), all event log databases progress through three states: hot, warm, and cold. With this architecture, a hot database of uncompressed logs exists only on the collection server. The reporting server holds compressed warm databases; the remote storage server holds only cold databases. When a cold database is restored with the restore shell script, it is restored in a warm state. When it is restored manually with LMArchive utility, it is restored in a defrosted state.

The following four event log storage states describe uncompressed, compressed, backed up and moved, and restored databases, respectively:

Hot

A hot database state is the state of the single uncompressed database in the event log store of a collection server where newly processed events are inserted. You can configure the maximum number of new records to store in a hot database (Maximum Rows) before compressing it into a warm database. You can schedule auto-archiving to move warm databases from the collection server to the configured reporting server hourly. (A hot database also exists on the reporting server for inserting self-monitoring events.)

Warm

The warm database state is the state of databases retained in the event log store of the reporting server. If you configure daily auto-archiving between the reporting server and a remote storage server, warm databases are retained until they are moved to the remote storage server; then they are auto-deleted from the reporting server. If you do not configure auto-archiving between the reporting server and a remote storage server, warm databases can remain on the reporting server until their age in days reaches the configured value for Max Archive Days or when the configured Archive Disk Space threshold is reached, whichever comes first. When one of these thresholds is met, the database is deleted and its state changed to cold. Without auto-archiving, you must manually back up warm databases with a third party tool before they are deleted and then run LMArchive utility to notify CA User Activity Reporting Module of the names of the databases you backed up and moved. The warm state is also applied when a recatalog is performed with the restore-ca-elm.sh script or the Recatalog button after restoring a cold database.

Cold

A cold database state applies to a database on the remote storage server. A record of a cold database is created on the reporting server when the database is auto-archived to the remote management server and deleted from the reporting server. If handled manually, a record of the cold database is created when the LMArchive utility is run with the -notify arch option. You can query the archive catalog of a reporting server to identify cold databases to restore.

Defrosted

A defrosted database state is the state applied to a physical cold database that has been restored to the archive directory after the Administrator runs the LMArchive utility with the -notify rest option to notify CA User Activity Reporting Module that it has been restored. Defrosted databases are retained for the number of hours configured for the Export Policy.

You can query databases in every state. A normal query returns event data from the hot and warm databases on the reporting server and defrosted databases, if they exist. A federated query returns event data from all servers in the federation, including the federated collection servers that include hot databases. An archive query returns a list of databases that no longer exist on the reporting server, that is, databases in a cold state. The physical databases represented by an archive query can exist on the remote storage server used for on-site storage or in off-site storage.

More information

Manually Backing Up Archived Databases

Manually Restoring Archives to the Original Event Log Store

Manually Restoring Archives to a New Event Log Store

Automating Backup and Restore

Automating Backup and Restore

A backup process ensures that no data is lost due to the deletion of aged databases. The preferred method of backing up archived databases is to use auto archive. Auto archive is a scheduled, automated transfer of archived databases between pairs of servers. Auto archiving between a source server and a destination server requires non-interactive authentication. Non-interactive authentication uses RSA public-key authentication without a passphrase. You can configure non-interactive authentication and auto archiving:

A restore process moves archived databases from remote storage to a CA User Activity Reporting Module server for investigation. The preferred method of restoring archived databases is to use the restore-ca-elm.sh script. This restore utility automates the transfer of archived databases. Like the auto archive process, the restore-ca-elm.sh script uses non-interactive authentication. You can configure non-interactive authentication and run the restore script:

You configure auto archiving to take place on a regular schedule. You invoke restore on an as needed basis.

More information:

Example: Auto-Archiving Across Three Servers

Restore Auto-Archived Files

Configuring Non-Interactive Authentication for Auto Archive

Configuring Non-Interactive Authentication for Restore

Database Move and Backup Strategy Flowchart

Data Integrity Checks

You can check a raw event, archived or recataloged data for tampering if you are logged in as a user with Administrator rights. Checking verifies that the incoming raw events and archived data are not tampered, thus helping meet regulatory requirements.

CA User Activity Reporting Module uses digital signatures to validate databases and raw events. If a database is corrupted, or the signature of a database is missing or corrupted, the data integrity check considers the database as tampered. If a raw event signature is missing or corrupted, the data integrity check considers the raw event in a database as tampered.

CA User Activity Reporting Module lets you perform data integrity checks on a database or a raw event. By default, CA User Activity Reporting Module performs data integrity checks on a database.

More information:

Sign Quarantined Databases

How Data Integrity Checks on Raw Events Works

To perform data integrity checks on raw events, enable the Enable Raw Event Signature Generation option. Enabling this option adds a value to the column rawevent_signature of each received raw event.

When a raw event is generated, CA User Activity Reporting Module performs one of the following steps:

Note: Events received using the previous releases of CA User Activity Reporting Module do not contain the rawevent_signature column.

To verify that databases do not contain a mix of events with and without raw event signatures, CA User Activity Reporting Module performs the following tasks whenever you switch the Enable Raw Event Signature Generation option:

  1. Stops the iGateway.
  2. Appends the existing raw databases to the hot databases.
  3. Moves the existing hot databases for archival.

    Note: As CA User Activity Reporting Module moves an existing hot database when you switch, the size of the archived database may vary from the configured database size.

  4. Starts the iGateway.

More information:

Enable Data Integrity Check on Raw Events

Enable Data Integrity Check on Raw Events

Important! Performing data integrity checks on raw events impacts CA User Activity Reporting Module performance due to an increase in the database size.

Follow these steps:

  1. Click Administration, Services, Global Configuration.

    The Global Service Configuration: Global Configuration page opens.

  2. Select Enable Raw Event Signature Generation option in Data Integrity section.
  3. Click Save.

How to Set Up Data Integrity Checks

You can set up data integrity checks in the following ways:

Note: You can set up a data integrity check on raw events only on demand.

You can view the results of any of these checks from the Data Integrity interface. Any tampered databases are quarantined, and appear in the Quarantined Databases list. When you run a data integrity check on raw events, CA User Activity Reporting Module does not quarantine a database that contains tampered events. CA User Activity Reporting Module only highlights a tampered database and provides an option to export the tampered raw events of the database onto your local computer in a CSV.gz format.

More information:

Enable Automatic Integrity Check

Schedule a Data Integrity Check

Check Data Integrity on Demand

Enable Automatic Integrity Check

You can set a data integrity check on a database to occur automatically whenever you restore or recatalog data.

To enable an automatic data integrity check

  1. Click the Administration tab, and expand the Services subtab.
  2. Select the Event Log Store node to enable global automatic checks, or expand the Event Log Store node and select a CA User Activity Reporting Module server to enable a local automatic check.
  3. Select Validate Integrity on Recatalog and Restore.
Schedule a Data Integrity Check

You can schedule daily data integrity checks on a database to occur at set times and on selected CA User Activity Reporting Module servers. Any tampered databases detected by a scheduled integrity check are automatically quarantined.

To schedule a data integrity check

  1. Click the Administration tab, and expand the Services subtab.
  2. Select the Event Log Store node to schedule a global check, or expand the Event Log Store node and select a CA User Activity Reporting Module server to set a local check.
  3. Select Enabled.
  4. (Optional) Select Federated to run the scheduled check on any federated servers visible from the selected server.
  5. Set the Daily Start Time.
Check Data Integrity on Demand

You can run a data integrity check on a database or raw events at any time on a selected CA User Activity Reporting Module server.

To run a data integrity check on demand

  1. Click the Administration tab, and click the Archival Management subtab.
  2. Expand the Data Integrity Folder, and select the CA User Activity Reporting Module server where you want to run a check.
  3. Select Database or Raw Events.
  4. Select a time range for your check.
  5. (Optional) Select Federated to run the check on any federated servers visible to the selected server.
  6. Click Validate Now.

    CA User Activity Reporting Module displays the following details according to the selected data integrity check:

    Note: You cannot perform a data integrity check on raw events received from previous versions of CA User Activity Reporting Module r12.5.02.

Sign Quarantined Databases

You can regenerate the digital signature on a quarantined database, making it available for queries.

To regenerate a quarantined database signature

  1. Click the Administration tab, and click the Archival Management subtab.
  2. Expand the Data Integrity Folder, and select the CA User Activity Reporting Module server where the quarantined databases are located.
  3. Click the Quarantined Databases tab in the right pane.
  4. Select the database for which you want to regenerate a signature.
  5. Click Generate Signature.

    A confirmation message appears

Rotate Keys

You can rotate the registration keys used to secure archived databases for improved security.

Registration keys use a public/private encryption key combination to secure the database files. When you rotate keys, the the old public key is retained, so that CA User Activity Reporting Module can verify files which used the old private key.

To rotate registration keys

  1. Click the Administration tab, and expand the Services subtab.
  2. Select the Data Integrity Folder, and click Rotate Keys

    A confirmation message appears.

Import Keys

You can import public registration keys used to secure archived databases from an outside source. Importing allows you to retain keys used by other older CA User Activity Reporting Module servers, or by previous servers. For example, if you maintain a backup list of public keys, you could import to be able to verify old database signatures on a newly built CA User Activity Reporting Module server.

To import registration keys

  1. Click the Administration tab, and expand the Services subtab.
  2. Select the Data Integrity Folder, and click Import Keys.

    An import file dialog appears.

  3. Browse for the XML key file you want to import and click OK.

    A confirmation message appears.

    Note: You can only import keys in XML format.

Export Keys

You can export the public registration keys used to secure archived databases. Exporting allows you to back up the keys for later import to other CA User Activity Reporting Module servers.

To export registration keys

  1. Click the Administration tab, and expand the Services subtab.
  2. Select the Data Integrity Folder, and click Export Keys.

    An export dialog appears.

  3. Select the location you want and click Save.

Configuring Non-Interactive Authentication for Restore

After you configure non-interactive ssh authentication between the remote storage server and the destination server, you can use the restore-ca-elm shell script to restore the archived databases on demand. For restore, the remote storage server is the source and the reporting CA User Activity Reporting Module or restore point CA User Activity Reporting Module is the destination.

The processes are slightly different, depending on whether the destination is a reporting server or a dedicated restore point.

Both processes assume that you previously prepared the remote storage server to act as the destination server for auto archive, which requires non-interactive authentication.

More information:

Create a Directory Structure with Ownerships on the Remote Storage Server

Example: Configure Authentication From Remote Storage to a Restore Point

Example: Configure Authentication From a Storage Server to a Reporting Server

Example: Configure Authentication From Remote Storage to a Restore Point

Dedicating a CA User Activity Reporting Module server to use as a restore point makes setting up non-interactive authentication easy. Once you set up authentication between the remote store server and the restore point, you can use the restore-ca-elm.sh script for every restore without any additional steps for authentication.

The process for configuring non-interactive authentication from a storage server to a restore point CA User Activity Reporting Module involves the following procedures:

  1. From the remote storage server, generate the RSA public/private key pair. Copy the public key as authorized_keys to the /tmp directory on the restore point.
  2. From the restore point, create the .ssh directory in /opt/CA/LogManager and set ownership to caelmservice. Copy authorized_keys from the /tmp directory to the .ssh directory. Change ownership and set permissions on authorized_keys.
  3. Validate successful non-interactive authentication between the remote storage server and the restore point.

More information:

Prepare the Public Key File for Use

Generate Keys and Copy the Public Key to the Restore Point

Generate Keys and Copy the Public Key to the Restore Point

From the remote storage server, generate an RSA key pair as the caelmservice user. Then, copy the public key file id_rsa.pub as authorized_keys, to the /tmp directory on the restore point CA User Activity Reporting Module. A restore point is a server dedicated to investigating restored data.

It is assumed that the /opt/CA/LogManager/.ssh directory structure exists on the storage server with the ownership set to caelmservice user and group. It contains authorized_keys copied from reporting servers. When you generate the key pair, you save id_rsa.pub to the /opt/CA/LogManager/ssh directory.

To generate the RSA public/private key pair for remote storage to restore point server authentication

  1. Log on to the remote server used for storage through ssh as the caelmadmin user.
  2. Switch users to the root account.
    su -
    
  3. Switch users to the caelmservice account.
    su - caelmservice
    
  4. Generate an RSA key pair as the caelmservice user.
    ssh-keygen -t rsa
    
  5. Press Enter to accept the default when each of the following prompts appears:
  6. Change directories to /opt/CA/LogManager.
  7. Change the permissions of the .ssh directory using the following command:
    chmod 755 .ssh
    
  8. Navigate to .ssh, where id_rsa.pub key is saved.
    cd .ssh
    
  9. Copy the public key as authorized_keys to the /tmp directory on the restore point server.
    scp id_rsa.pub caelmadmin@<restore_point>:/tmp/authorized_keys
    
Prepare the Public Key File for Use

You create the .ssh directory on the restore point server and set ownership to caelmservice. Then, you copy authorized_keys from the /tmp directory to the .ssh directory. Last, you set ownership and permissions on the public key file.

To prepare the public key on the restore point server for non-interactive authentication

  1. Log into the restore point CA User Activity Reporting Module server through ssh as caelmadmin.
  2. Switch users to root.
  3. Change directories to the CA Enterprise Log Manager directory.
    cd /opt/CA/LogManager
    
  4. Create the .ssh directory:
    mkdir .ssh
    
  5. Change the ownership of .ssh to the caelmservice user and group:
    chown caelmservice:caelmservice .ssh
    
  6. Change directories to /opt/CA/LogManager/.ssh.
  7. Copy the authorized_keys file from /tmp to .ssh:
    cp /tmp/authorized_keys .
    
  8. Change ownership of the authorized_keys file to caelmservice:
    chown caelmservice:caelmservice authorized_keys
    
  9. Change permissions on the authorized_keys file:
    chmod 755 authorized_keys
    

Example: Configure Authentication From a Storage Server to a Reporting Server

You can restore archived databases from a remote storage server back to their original reporting server, that is, the server from which they were auto archived. The advantage of this method is that you do not have to recatalog the CA User Activity Reporting Module archive database. The databases of log files you are restoring are already known to the reporting server. If you have multiple reporting servers, you configure non-interactive authentication between the remote storage server and each reporting server. The authorized_keys file exists in the .ssh directory on the reporting server. This authorized_keys file has the public keys of each key pair generated on a collection server that auto archives to this reporting server. Therefore, you create an authorized keys file with a suffix and then concatenate that file to the original authorized_keys.

The process for configuring non-interactive authentication from a remote storage server to a reporting CA User Activity Reporting Module involves the following procedures:

  1. From the remote storage server:
    1. Configure the RSA public/private key pair for remote storage to reporting server authentication.
    2. Copy the public key as authorized_keys_RSS from the storage server to the /tmp directory on the reporting server.
  2. From the reporting server:
    1. Copy the current authorized_keys from .ssh to /tmp.
    2. Concatenate authorized_keys_RSS in the /tmp directory to the authorized_keys file.
    3. Copy the appended authorized_keys file back to the .ssh directory.
  3. From the remote storage server, validate successful non-interactive authentication between servers.
  4. Repeat these steps for each remote storage server to reporting server combination.

More information:

Generate Keys and Copy the Public Key to a Reporting Server

Update the Existing Public Key File

Generate Keys and Copy the Public Key to a Reporting Server

From the remote storage server, generate an RSA key pair as the caelmservice user and then copy the public key as authorized_keys_RSS to the /tmp directory on a reporting CA User Activity Reporting Module server. The reporting server typically has an authorized_keys file in the .ssh directory that contains a concatenation of public keys from various collection servers. Send the key with a unique name so that it can be appended to the existing authorized_keys file.

To generate the RSA public/private key pair and copy the public key from the remote storage to a reporting server

  1. Log on to the remote storage server through ssh as the caelmadmin user.
  2. Switch users to root.
  3. Switch users to the caelmservice account.
    su - caelmservice
    
  4. Generate an RSA key pair as the caelmservice user.
    ssh-keygen -t rsa
    
  5. Press Enter to accept the default when each of the following prompts appears:
  6. Change the permissions of the .ssh directory using the following command:
    chmod 755 .ssh
    
  7. Navigate to the .ssh directory.
  8. Copy id_rsa.pub as authorized_keys_RSS to the /tmp directory on the reporting server.
    scp id_rsa.pub caelmadmin@<reporting_server>:/tmp/authorized_keys_RSS
    

Update the Existing Public Key File

You copied the public key, authorized_keys_RSS, to the /tmp directory on the reporting server. Now you prepare the existing public key file for use. Preparation involves appending the authorized_keys_RSS to authorized_keys. The correct ownership and permissions are already set on the existing authorized_keys file.

To append authorized_keys_RSS to authorized_keys and copy it to the correct location

  1. Log into the reporting CA User Activity Reporting Module server through ssh as caelmadmin.
  2. Switch users to root.
  3. Change directories to the /tmp directory containing authorized_keys_RSS.
  4. Copy the existing authorized_keys from .ssh to the current directory, /tmp.
    cp /opt/CA/LogManager/.ssh/authorized_keys .
    
  5. Add the contents of the public key from the remote storage server to the authorized_keys file that contains public keys from collection servers.
    cat authorized_keys_RSS >> authorized_keys
    
  6. Change directories to /opt/CA/LogManager/.ssh.
  7. Copy the authorized_keys file from /tmp to.ssh, the current directory:
    cp /tmp/authorized_keys .
    

Query the Archive Catalog

You can create queries to search the local archive catalog for cold (remote-stored) databases, using quick or advanced filters. The query results can help you identify the backed up database files that must be restored to conduct an investigation.

To query the archive catalog

  1. Click the Administration tab, and then click the Archival Management subtab.

    The Archival Explorer folder list appears.

  2. Click the Archival Query and Recatalog folder.

    The Archive Query dialog appears in the details pane.

  3. Select or enter the time period for your query.
  4. Click Add Filter, select a column, and enter the column search value. You can add multiple filters.
  5. Select Exclude to query for all logs except those logs with the value you enter.

    Note: If you create a filter specifying a column that is not in the catalog, CA User Activity Reporting Module returns all the databases in the specified time range, rather than an empty set. You are not required to know all the cataloged columns to create a useful archive query.

  6. (Optional) Click the Advanced Filters tab to add advanced filters. Include the event information if the column bears the appropriate relation to the value you enter. Select a column, select an Operator, and then select or enter a value. Operator descriptions follow:
    Relational Operators

    Equal to, Not Equal to, Less than, Greater than, Less than or equal to, Greater than or equal to.

    Like

    Includes the event information if the column contains a pattern matching your entry of text with the wildcard character, %. L% includes values beginning with L. %L% includes values containing L, excluding an L that is either the first or last character.

    Not like

    Includes the event information if the column does not contain the pattern you specify.

    In set

    Includes the event information if the column contains one or more of the values in the quote-delineated set you enter. Multiple values in the set must be separated with commas.

    Not in set

    Includes the event information if the column contains one or more of the values in the quote-delineated set you enter. Multiple values in the set must be separated with commas.

    Matches

    Includes any event information that matches one or more of the characters that you enter, allowing you to search for key words.

    Keyed

    Includes any event information that is set as a key value during Report Server configuration. Use key values to set business relevance or other organizational groups.

    Not Keyed

    Includes any event information that is not set as a key value during Report Server configuration. Use key values to set business relevance or other organizational groups.

  7. Click Query.

    The query results appear. The files containing records matching your query criteria are displayed with the full relative path, relative to $IGW_LOC. Examples follow:

    ../../LogManager/data/archive/<databaseFilename>

    <remoteHostname>-.../../LogManager/data/archive/<databaseFilename>

Restore Auto-Archived Files

If you copy archived files from external storage to the remote server configured for auto-archiving, you can restore them with the restore-ca-elm.sh script. This alternative is preferred over the manual use of LMArchive utility.

To restore auto-archived files

  1. Use your caelmadmin credentials to log on to the CA Enterprise Log Manager server with the event log store where you want to restore the databases.
  2. At the command prompt, switch users to root, that is:
    su - root
    
  3. Change directories to /opt/CA/SharedComponents/iTechnology with the following shortcut:
    cd $IGW_LOC
    
  4. At the command prompt, switch users to the caelmservice account.
    su - caelmservice
    
  5. Run the following command, where userid and pwd are the credentials of a CA Enterprise Log Manager user account with the Administrator role.
    restore-ca-elm.sh -euser userid -epasswd pwd -rhost hostname -ruser userid -rlocation path -files file1,file2,file3...
    

    Note: To allow a non-Administrator to run the restore-ca-elm shell script, create a custom role and custom policy. Then, users to whom you assign this custom role can specify their credentials for userid and pwd.

More information:

Automating Backup and Restore

Restore–Script for Restoring Archived Databases

Example: Allow a Non-Administrator to Manage Archives

Restore–Script for Restoring Archived Databases

You cannot query or report on data that resides in a cold database on a remote storage server. To query and report on such data, that data must reside in a warm state on a CA User Activity Reporting Module server. The restore shell script, restore-ca-elm.sh, is a command-line utility that moves a specified cold database and its digital signature to a specified CA User Activity Reporting Module server and restores it to a warm state. You can use the restore utility to move a database back to the original reporting server or to a dedicated restore point. Configuring non-interactive authentication is a prerequisite to running the restore script.

You run the restore script from the CA User Activity Reporting Module server to which you want to restore the files. The remote host you identify in the command refers to the remote storage server. Cold databases reside in the archive directory of the remote storage server.`

Requirements for restoring database files to either the original reporting server or a restore point server follow:

If restoring files to a restore point server, take the following additional actions:

  1. Copy the RSA key from the remote storage server to the restore point server
  2. Set the RSA key file ownership on the restore point server.

The command has the following format:

restore-ca-elm.sh -euser userid -epasswd pwd -rhost hostname -ruser userid -rlocation path -files file1,file2,file3...
-euser username

Specifies the user name of a CA User Activity Reporting Module user account with the Administrator role.

-epasswd pwd

Specifies the CA User Activity Reporting Module password associated with the user name.

-rhost host

Specifies the hostname or IP address of the remote host where cold database files reside in the archive directory. The remote host is not a CA User Activity Reporting Module server.

-ruser remote user

Specifies the user account with permissions to the /opt/CA/LogManager path and ownership of the .ssh folder containing the authorized keys file. Typically, this user account is the caelmservice user account.

-rlocation path

Specifies the path to the database files on the remote storage server. If the remote storage server is a UNIX server, the path is /opt/CA/LogManager/data/archive.

files file1,file2,file3...

Specifies a comma-separated list, without spaces, of the database files to restore.

Example: Restore Shell Script

The following example command is run from the CA User Activity Reporting Module to which the archived database files are to be restored. It is run by a user with account credentials of Administrator1, calm_r12. The remote server to which the archived databases have been moved from off-site storage is named NY-Storage-Svr. This remote server has been configured with a caelmservice account that has ownership of the .ssh folder where the RSA public key has been copied. This account also has full privileges on the directory structure /opt/CA/LogManager. This command specifies that the files to be restored are in the data/archive directory path of the NY‑Storage‑Svr server and identifies the database file to be restored as NY‑Storage‑Svr_20081206192014.db.cerod.

restore-ca-elm.sh -euser Administrator1 -epasswd calm_r12 -rhost NY‑Storage‑Svr -ruser caelmservice -rlocation /opt/CA/LogManager/data/archive -files NY‑Storage‑Svr_20081206192014.db.cerod

Manually Backing Up Archived Databases

CA User Activity Reporting Module creates a new archived database automatically each time data is moved from hot to warm storage, as controlled by your settings. Although it is recommended that you configure auto-archive to move warm databases to a remote server, you can use your own tools to perform backups of the archive databases and then run the LMArchive utility to notify the system that you performed the backup.

We recommend that you back up your warm databases daily, using either the automated method or the manual method described here. This is important since warm-storage archive files are automatically deleted after the time you specify or when disk space drops to the percentage you specify.

Backing up warm databases manually involves the following steps:

  1. Identify the warm databases that are not yet backed up.
  2. Perform the backups.
  3. Record the backups.

More information

Perform the Backups

Identify Databases Not Backed Up

Record the Backups

Identify Databases Not Backed Up

You can view a list of archived databases that are not yet marked as backed up with the LMArchive utility. The ability to get reliable results assumes that someone has run this utility with the -notify arch option each time archived database are backed up.

Important! To avoid confusion, keep current with notifying CA User Activity Reporting Module of completed backups.

To display the names of all current archived database files not marked as backed up

  1. Use your caelmadmin credentials to log on to the CA User Activity Reporting Module server with the event log store that contains the databases that need to be backed up for archiving.
  2. At the command prompt, switch users to root, that is:
    su - root
    
  3. Change directories to /opt/CA/SharedComponents/iTechnology with the following shortcut:
    cd $IGW_LOC
    
  4. Execute the following command, where username and pwd are the credentials of a CA User Activity Reporting Module user account with the Administrator role.
    LMArchive -euser username -epassword pwd -list inc
    

Example: Display All Current CA User Activity Reporting Module Archived Files Not Marked as Backed Up

The following command issued by an Administrator requests the list of all warm databases that are not marked as backed up.

LMArchive -euser Administrator1 -epassword calmr12 ‑list inc

A list of archive files not marked as backed up appear, in a format similar to the following:

CAELM Archived Files (not backed up): 
calm04_20091206192014.db.cerod
calm04_20091206192014.db.sig

More information:

LMArchive–Backup/Restore Tracking

Perform the Backups

If you have not configured auto-archive from a CA User Activity Reporting Module reporting server to a remote storage server that is not a CA User Activity Reporting Module server, you must manually back up the archived databases and move them to a safe storage location, such as a separate disk or server.

Important! Be sure to back up the databases and move them before they are deleted from the CA User Activity Reporting Module reporting server.

Warm databases are automatically deleted when the configured value for Max Archive Days is reached or the percentage of disk space falls below the configured value for Archive Disk Space. To prevent data loss from deleted files, perform the backups regularly.

To back up warm databases manually

  1. Use your caelmadmin credentials to log on to the CA User Activity Reporting Module reporting server with the event log store that contains the target databases.
  2. Switch users to root, that is:
    su - root
    
  3. Change directories to /opt/CA/LogManager/data/archive.
  4. Back up the warm databases with the backup tool of your choice and move them to an on-site interim storage server or to an off-site location for long-term storage, according to your site procedures.

More information:

Identify Databases Not Backed Up

Record the Backups

Each time you perform a backup of one or more archived database, be sure to record that fact in the CA User Activity Reporting Module where you took the backup.

Note: Failure to record each backup causes incorrect data to be reported when using the LMArchive utility to list backed up databases.

To record the backups of specific archived databases

  1. Use your caelmadmin credentials to log on to the CA User Activity Reporting Module server with the event log store that contains the databases that you backed up.
  2. At the command prompt, switch users to root, that is:
    su - root
    
  3. Change directories to /opt/CA/SharedComponents/iTechnology with the following shortcut:
    cd $IGW_LOC
    
  4. Execute the following command, where username and pwd are the credentials of a CA User Activity Reporting Module user account with the Administrator role.
    LMArchive -euser username -epassword pwd -notify arch -files file1,file2,file3...
    

Example: Notify CA User Activity Reporting Module that Certain Files Have Been Backed Up

The following command issued by the Administrator named Administrator1notifies the CA User Activity Reporting Module event log store that the warm database, calm04_20091206192014.db.cerod, has been backed up. Backed up databases can be manually moved to external storage for long-term retention.

LMArchive -euser Administrator1 -epassword calmr12 -notify arch ‑files calm04_20091206192014.db.cerod

The archive file notification appears, in a format similar to the following:

Archive notification sent for file calm04_20091206192014.db.cerod...

More information:

LMArchive–Backup/Restore Tracking

Manually Restoring Archives to the Original Event Log Store

You may occasionally need to restore cold database files for querying or reporting to the archive directory on a CA User Activity Reporting Module server. This may be required to investigate a security breach or for an annual or semi-annual compliance audit. The procedures you use depend on the following two factors:

When you are restoring files to the original reporting server, use the following procedures:

  1. Prepare to restore archived databases by identifying the files to restore and determining the archive directory.
  2. Move the databases from external storage to the archive directory on either the remote server location you configured for auto-archive or to the original reporting server.
  3. If you moved the archived files to the remote storage server configured for auto-archiving, log on to the reporting CA User Activity Reporting Module and restore the auto-archived files from the remote storage server with the restore-ca-elm.sh script.
  4. If you moved the archived files to the archive directory on their original reporting CA User Activity Reporting Module server, restore the manually archived files with LMArchive utility.
  5. Verify that the defrosted database can be queried by running a query with the end date set to the date of the restored database and examining the query results.

More information

Move Archived Databases to an Archive Directory

Restore Manually Archived Files

Verify Restoration

Prepare to Restore Archived Databases

Restore Auto-Archived Files

Prepare to Restore Archived Databases

Before you restore archived databases, you need to know the following:

You can query the archive catalog through the CA User Activity Reporting Module Administration tab, Log Collection Explorer, where you can specify simple or advanced filters. Or you can use the command line utility as described here.

If you already have the needed information at hand, skip this procedure.

To prepare to restore archived databases

  1. Use your caelmadmin credentials to log on to the CA Enterprise Log Manager server with the event log store where you want to restore the databases.
  2. At the command prompt, switch users to root, that is:
    su - root
    
  3. Change directories to /opt/CA/SharedComponents/iTechnology with the following shortcut:
    cd $IGW_LOC
    
  4. Identify the databases that you want to restore from a list of files that includes those that have been backed up and moved to external storage. To view the list of all archived files in this archive catalog, execute the following command, where userid and pwd are the credentials of a CA User Activity Reporting Module user account with the Administrator role.
    LMArchive -euser userid -epassword pwd -list all
    

    The list of all archived files appears.

  5. (Optional) If restoring from manual backups, determine the location of the archive directory to which the identified cold archive files are to be copied. Execute the following command, where userid and pwd are the credentials of a CA User Activity Reporting Module user account with the Administrator role.
    LMArchive -euser userid -epassword pwd -list loc
    

    The archive directory appears.

Example: Display All Current CA User Activity Reporting Module Archive Files

The following command issued by the CA User Activity Reporting Module Administrator, Administrator1, requests a list of all databases located in the event log store's archive directory.

LMArchive ‑euser Administrator1 ‑epassword calmr12 ‑list all

A list of the current archive files appears, in a format similar to the following:

CAELM Archived Files:
calm04_20091206191941.db.cerod
calm04_20091206191958.db.cerod
calm04_20091206192014.db.cerod
calm04_20091206191941.db.sig
calm04_20091206191958.db.sig
calm04_20091206192014.db.sig

Example: Display the CA User Activity Reporting Module Archive Directory

The following command issued by a CA User Activity Reporting Module Administrator, Administrator1, requests the directory location of the archived databases:

LMArchive -euser Administrator1 -epassword calmr12 -list loc

The following is a typical response:

CAELM Archive Location (localhost) : 
../../LogManager/data/archive

More information:

Query the Archive Catalog

Move Archived Databases to an Archive Directory

If you moved your archived files to an off-site location, use your site procedures to retrieve them and bring them back on-site.

Move the archived databases back to the archive directory of either the original CA User Activity Reporting Module server or a remote server configured for non-interactive authentication. The archive directory is /opt/ca/LogManager/data/archive.

To move an archived database from an external storage to your network

  1. Move the database files to restore from external storage back to your network in one of the following ways:
  2. Proceed in one of the following ways, depending on the location of the archived files.

More information:

Restore Manually Archived Files

Restore Auto-Archived Files

Restore Manually Archived Files

After you have restored one or more databases from long-term storage to the archive directory, you must change ownership of the archive directory to the caelmservice user before notifying CA User Activity Reporting Module that the databases have been restored with the LMArchive utility. Archived files owned by root are not recognized by the LMArchive utility.

Running LMArchive with the -notify rest option changes the state of the archived database files from cold to defrosted so that they are available for querying and reporting.

Administrators configure the number of hours a defrosted archived database is retained before being automatically deleted from the archived directory with the Export Policy setting on the Event Log Store service configuration.

To restore manually archived database files

  1. Use your caelmadmin credentials to log on to the CA User Activity Reporting Module server with the event log store that contains the restored databases.
  2. At the command prompt, switch users to root, that is:
    su - root
    
  3. Change directories to the /data directory. For example:
    cd /opt/CA/LogManager/data
    
  4. Assign ownership of the archive directory (/opt/CA/LogManager/data/archive) to the caelmservice account.
    chown -R caelmservice:caelmservice archive
    

    Ownership of the archive files is changed to caelmservice, the internal operating system user, which is a non-login account.

  5. Change directories to /opt/CA/SharedComponents/iTechnology with the following shortcut:
    cd $IGW_LOC
    
  6. Execute the following command, where username and pwd are the credentials of a CA User Activity Reporting Module user account with the Administrator role.
    LMArchive -euser username -epassword pwd -notify rest -files file1,file2,file3
    

    A restoration confirmation appears. CA User Activity Reporting Module defrosts the specified files. Defrosted files are retained for the configured number of hours, where you can configure retention for up to seven days.

Note: You can now run queries and reports against the event data contained in the restored archive files.

Example: Notify CA User Activity Reporting Module that Certain Databases Have Been Restored

The following command issued by a CA User Activity Reporting Module user with an Administrator role notifies the CA User Activity Reporting Module event log store that the specified cold database, calm04_20091206192014.db, has been copied into the archive directory.

LMArchive -euser Administrator1 -epassword calmr12 -notify rest ‑files calm04_20091206192014.db.cerod

A restoration confirmation appears in a format similar to the following:

Restore notification sent for file calm04_20091206192014.db.cerod

More information:

LMArchive–Backup/Restore Tracking

Verify Restoration

You can quickly verify that the restored database is available for examination by running a quick query. Normal queries display data from restored databases in warm and defrosted states.

Consider the following process.

  1. Copy a subscription query designed to display the type of event details that the restored database holds.
  2. Advance to the query design wizard step where you set result conditions and enter a date range that corresponds to the newly defrosted database files.
  3. Save the query.
  4. Run the query.

Manually Restoring Archives to a New Event Log Store

You may occasionally need to restore cold stored files for querying or reporting, as for an annual or semi-annual compliance audit. If you designate one CA User Activity Reporting Module to act as a restore point for investigations on cold data, you must force a rebuilding of the catalog each time you restore a new database to this CA User Activity Reporting Module. A rebuilding of the catalog, or recatalog, is required only when restoring data to a different server than the one on which it was generated.

Important! Ensure the Max Archive Days setting for this server's event log store is adequate. Otherwise, restored files are immediately deleted.

A recatalog is automatically performed when iGateway is restarted, if needed. If databases were incompletely cataloged before iGateway was shut down, the recataloging process completes when iGateway is restarted. If one or more databases are added to the archive database directory while iGateway is down, the recatalog process is performed at the next startup of iGateway.

Restoring archived files from external storage to a different CA User Activity Reporting Module from where they were backed up involves the following steps:

  1. Identifying the databases that you want to restore. For help, query the archive catalog with filters.
  2. Moving the identified cold archive files from external storage to your network.
  3. Copying the moved databases to the archive directory. To display the archive directory, run the LMArchive utility with the -list loc option.
  4. Rebuilding the archive catalog (recatalog).

    Rebuilding the archive catalog to add a single database can take several hours. After waiting long enough for the recatalog process to complete, you can begin your investigation by running queries and reports on the event logs from the restored databases.

  5. Verify restoration by issuing a query.

Note: If you dedicate a CA User Activity Reporting Module as restore point, be sure to exclude it from the federation.

Configure Max Archive Days for Restored Archives

When you configure the Event Log Store for a CA User Activity Reporting Module dedicated as a restore point, we recommend you override the global setting for Max Archive Days and set it to the maximum, 28000. If the number of days to store archived database files before deleting it is set to a lower value than the age of the restored database files, those files are deleted by the system immediately after being restored to a warm state.

Note: This procedure applies only to files restored to a new event log store.

To set the max archive days to accommodate the age of restored files

  1. Click the Administration tab and the Services subtab.
  2. In the Service List, expand the Event Log Store folder and select the CA User Activity Reporting Module that is the dedicated restore point.
  3. Click the toggle button next to Max Archive Days to switch to local configuration and enable the entry field.
  4. Set the field to a value in days that accommodates the oldest file you will restore. The maximum value is 28000.
  5. Click Save.

Add Restored Databases to the Catalog

If you copy the restored database directly into the archive directory on a different server than the one where it was generated, rebuild the archive catalog to add the restored database.

Do not use recatalog in either of the following cases:

The recatalog process sets the restored database to a "warm" state, not a "defrosted" state as with the LMArchive -notify rest option. Therefore, it follows normal archiving rules, rather than the Export Policy set in the event log store configuration.

To rebuild the archive catalog to add the restored database

  1. Click the Administration tab, and then the Archival Management subtab.

    The Archival Management folder list appears.

  2. Click the Archival Query and Recatalog folder.

    Three buttons, including Recatalog, appear above tabs for Quick Filters and Advanced Filters.

  3. Click Recatalog.

    A success confirmation message appears. The restored database is added to the catalog in a warm state.

More information:

Example: Allow a Non-Administrator to Manage Archives

LMArchive–Backup/Restore Tracking

The LMArchive is the command-line utility that tracks the backup and restoration of warm databases to the event log store on a CA User Activity Reporting Module server. Use LMArchive to query for the list of warm database files that are ready for archiving. After backing up the listed database and moving it to long-term (cold) storage, use LMArchive to create a record on the CA User Activity Reporting Module server that this database was backed up. After restoring a cold database to its original CA User Activity Reporting Module server, use LMArchive to notify CA User Activity Reporting Module, which in turn defrosts the cold database files to a defrosted state that can be queried.

The command has the following format:

LMArchive -euser username -epassword pwd {-list [loc|all|inc] | -notify [arch|rest] -files file1,file2,file3...}
-euser username

Specifies the user name of a CA User Activity Reporting Module user account with the Administrator role.

-epassword pwd

Specifies the CA User Activity Reporting Module password associated with the user name.

-list [ loc | all | inc ]

Requests a list of one of the following: the archive directory locations, the names of all warm and cold databases, the names of just the warm databases

loc

Requests the location of the archive directory.

all

Requests a list of all filenames located in the archive directory of the event log store.

inc

Requests an incremental list of filenames of the current warm database files that have not been archived. The request returns the names of files that have not been backed up, moved to external storage, and changed to a cold state. Files are set to a cold state upon notification of the move through this utility's notify command.

-notify [ arch | rest ]

Notifies the CA User Activity Reporting Module event log store that the specified files have been successfully backed up or restored.

arch

Notifies the CA User Activity Reporting Module event log store that the specified files have been successfully backed up.

rest

Notifies the CA User Activity Reporting Module event log store that the specified files have been successfully restored.

-files file1,file2,file3...

Specifies the names of the database files you have backed up or restored.