Implementation Guide › Case Study Examples › Case Study 19: Adapter Wizard for File-based Data Source

Case Study 19: Adapter Wizard for File-based Data Source

This case study examines the best practice methods for integrating with a file-based data source. The example scenario handles a CSV data file that is produced by the source system. With most file-based integrations CA recommends following some basic guidelines, to ensure minimal risk to the integration. These are as follows:

• Where the option is available, we should request that the data is delivered to the CA Business Service Insight Application server’s file system. This ensures the delivery mechanism is not dependent upon the adapter trying to fetch data from a remote store (minimizes permission issues with user accounts, and synchronization issues etc.)
• The file naming convention is important since the adapter orders the files according to alphanumeric naming. If we can control this, CA recommends requesting two parts:
  • A sensible naming convention based upon the source file content (especially if more than one file comes from the source)
  • A reverse-order timestamp to ensure the files are ordered with the most-recent file last. (i.e. <file_name>_yyyymmdd_hhmiss.csv). The depth of the timestamp used would depend on the frequency of the data delivered

In this scenario, the source files are from a Topaz data source (now known as Mercury Global Monitor, owned by HP). This was created using an API of the product to include the required files for the specific KPIs required. The files are delivered to the CA Business Service Insight Application server directly by an external automated process. The source files are named: Topaz_yyyymmdd_hhmiss.csv.

Note: The timestamp of the file is the time at which it was created, hence all entries within the file are leading up to this time.

A sample of the data inside the file can be seen below.

Note: CA recommends checking CSV files in Notepad (rather than just Excel) to ensure that the date format is as expected. Excel has a tendency to format the dates according to regional settings of the machine and may not match the actual format seen by the adapter.

Before commencing any adapter creation process, it is also critical that you have done all necessary analysis and investigation into the data source and related KPIs to ensure that the following are known:

• What fields are required by the business logic
• What is the format of the dates in the file
• What is the time zone of the date/time values in the source file (these time zones should be created in the system before commencing the Adapter creation process)
• Are there any date fields that may have blank/NULL entries
• What is the behavior of the data source (are all records added, or are some records an update to a previous event)
• What drill-downs are required for the KPIs (this may impact your choice of “resource”)
• What “Event Types” are required for the effective filtering of the events in the business logic

Once these points are known, you can commence the creation process.

Now, we can perform the adapter creation process based upon this scenario, using the Adapter Wizard.

In our scenario we use the “TopazTransaction” as the resource, the “Profile” as the Event Type, and the “Time” field as the timestamp. We also bring the “Status”, “ResponseTime”, and “WtdDownloadTime” fields into the event structure for use with the business logic.