One of the most common requests I receive (and my #1 out of the box feature) is to automate data processing through FDM.
To meet this need, FDM includes a powerful automation tool known as the Batch Loader. This functionality is not to be confused with windows/dos batch (i.e., .cmd, .bat) files. FDM batch loader functionality uses FDM scripts – similar to import scripts – to mimic the action of an end user performing the Workflow steps of Import, Validate & Export.
As with an interactive session, batch loader can process data using the standard or multiload Workflow. For each workflow method, parallel or serial processing is available.
Each file in the collection of files is processed sequentially.Parallel Processing:
The application spawns multiple server processes to process groups/sets of files concurrently. The application is optimized to ensure that files that share a data segment table are assigned to the same process. This built-in intelligence ensures that the batch process runs as efficiently as possible.
This reporting is generally provided by email but most communication protocols can be supported. Click the image to the right to see an example email.
When preparing status reporting, you need to think about the various possible outcomes for each data set processed. For example:
- Are the unmapped items output to an excel file and attached to the email?
Do you prefer FDM to load unmapped members to a suspense member?
- How do you transform the XML generated by the application to a format that can be attached to an email and viewed on any PC?
Does a custom process need to be created to “override” the invalid intersections – thereby allowing the valid intersections to be loaded to HFM?
- If the batch includes multiple locations, how do you filter the status to the appropriate recipients?
These are just a few of the questions and considerations when designing an end to end lights-out batch process. The inline image is an example of how I’ve addressed some of the most common questions/requirements.
The out of the box functionality of the batch loader and the capability to layer in automated real time status reporting truly helps you realize an even greater return on your investment – not only for FDM but in your entire Oracle/Hyperion EPM product suite. The end to end data load automation process can be scheduled to run at set intervals. This reduces the workload for the various FDM data loaders while also eliminating the bottleneck that can be caused by time zone differences, resource availability, etc. The end result is a process that is more efficient and more dependable.
I hope you are as excited about FDM’s automation capabilities as I am. As you consider the use of FDM’s batch loader it may ease your trepidation to know that I have yet to (nor do I expect to) see FDM fail to address the various needs and requirements of an automated data load process. I strongly encourage you to take the next step with your application and leverage the batch loader – you won’t regret the investment!
If you have question of comments, feel free to post them below – I will respond.