Implementation Guide › Configuring Services › Configuring the Event Log Store › Example: Auto-Archiving Across Three Servers

Previous Topic: Example: Configure Non-Interactive Authentication Across Three Servers

Next Topic: Event Log Store Settings in the Basic Environment

Example: Auto-Archiving Across Three Servers

When using the collection-reporting architecture, you must configure auto-archiving from the collection server to a reporting server. This configuration automates the move of a warm database of collected and refined event log data to the reporting server where you can report on it. It is a good practice to schedule this auto-archiving to recur hourly, rather than daily, to avoid devoting an extended period of time every day for doing huge data transfers. Choose a schedule based on your load and whether it is better to consolidate processing or spread it out over the day. When databases are copied through auto archiving from a collection server to its reporting server, those databases are deleted from the collection server.

After you identify a local server with a lot of storage space, you can configure auto-archiving from the reporting server to this remote storage server. When databases are copied through auto archiving from a reporting server to a remote storage server, those databases stay intact on the reporting server until the time you configure as Max Archive Days has elapsed. At that point, they are deleted. The benefit of this phase of auto-archiving is to protect archived databases from being lost due to not being manually moved to a long-term storage location before auto-deletion.

Note: Before you configure a remote server to receive auto-archived databases, you must set up a directory structure on this destination server like that on the source CA Enterprise Log Manager server and assign various ownerships and permissions for authentication. For details, see "Configuring Non-Interactive Authentication" in the Implementation Guide. Be sure to follow instructions described in "Set Key File Ownership on a Remote Host."

For this example scenario, assume you are a CA Enterprise Log Manager Administrator in a New York data center with a network of CA Enterprise Log Manager servers, each with a dedicated role, plus a remote server with a lot of storage capacity. Names of the servers used in auto-archiving follow:

Note: This example assumes the existence of a management server dedicated to managing the CA Enterprise Log Manager system of servers. This server is not depicted here because it has no direct role in auto-archiving.

Three servers NY-Collection-ELM, NY-Reporting-ELM, NY_Storage_Svr

To configure auto-archiving from a collection server to a reporting server and then from the reporting server to a remote storage server, use the following example as a guide:

  1. Select the Administration tab and the Log Collection subtab.
  2. Expand the Event Log Store folder and select a collection server.

    Select NY-Collection-ELM under Event Log Store.

  3. Specify Auto-archiving to recur hourly, where the destination is the reporting server. Enter credentials of a CA Enterprise Log Manager user with an Administrator role. If you have custom policies, this must be a user with edit rights to the Database resource, which grants the ability to delete the archived database.

    Select autoarchive to recur hourly to the reporting server.

  4. Select the reporting server from the Services list.

    Select the reporting server from the services list.

  5. Specify Auto-archiving to recur daily, where the destination is a remote server that is used for storage. Enter credentials of a user account with an Administrator role. Optionally, create a CALM access policy with the edit action on the database resource and assign a user as the Identity. Enter the credentials of that low-privileged user here.

    Select auto-archiving to recur daily to the remote server.

The numbers on the following diagram depict two configurations of auto-archiving: one from the collection server to the reporting server and another from the reporting server to a remote server on the network.

Auto-archive from collection server to reporting server, then auto-archive from reporting server to remote storage server.

After such a configuration, that automatic processing works as follows:

  1. NY-Collection-ELM, the Collection CA Enterprise Log Manager server, collects and refines events and inserts them into the hot database. When the hot database reaches the configured number of records, the database is compressed into a warm database. Since auto-archiving is scheduled to recur hourly, each hour the system copies the warm databases and moves them to the NY-Reporting-ELM, the reporting CA Enterprise Log Manager server. The warm databases are deleted from NY-Collection-ELM when they are moved.
  2. NY-Reporting-ELM retains databases that can be queried until they are the age configured for Max Archived Days, after which they are deleted. Since auto-archiving is scheduled to recur daily, each day the system copies the warm databases and moves them as cold databases to NY-Storage-Svr. The cold databases can remain on the remote storage server for an extended period of time.
  3. You move the cold databases stored on the network NY-Storage-Svr to an off-site long-term storage solution where they can remain for the mandated number of years.

    The reason for archiving is to keep event logs available for restoration. Cold databases can be restored if a need arises to investigate old events that have been logged. The manual step of moving archived databases from the on-site storage server to an off-site long-term storage location is depicted on the following diagram.

    Use manually moved warm databases to off-site long-term storage.

  4. Assume that a situation surfaces that makes it necessary to examine logs that have been backed up and moved off-site. To identify the name of the archived database to restore, search the local archive catalog on NY-Reporting-ELM. (Click the Administration tab, select Archive Catalog Query from the Log Collection Explorer, and click Query.)
  5. Retrieve the identified archived database from off-site storage. Copy it back to the /opt/CA/LogManager/data/archive directory on the NY-Storage-Svr. Then, change the ownership of the archive directory to caelmservice user.
  6. Restore the database either to its original reporting server or to a restore point dedicated to investigating logs from restored databases as follows:

    Off-site databases can be manually restored to the local storage server and then copied to a restore point if available or back to the reporting CA Enterprise Log Manager.

    Note: You can now query and report on the restored data.

More information:

About Auto Archive

About Archive Files

Event Log Store Settings in the Basic Environment

Example: Federation Map for a Large Enterprise