Enhancing Data Quality with Smart Replace

You may be familiar with or even using the data protection feature of FDM because there is certain data in HFM that you don’t want to be cleared when you load data in replace mode.  If you are using this feature or if you’ve reviewed my Top 10 list, you know this feature is 100% out of the box and required no custom coding to enable this functionality.  In this post I step outside the box and introduce you to a custom solution that addresses another common data quality problem in HFM (or any other target EPM application).

How often have you encountered the following scenario?  A person working in AP receives an invoice.  They enter the necessary information in the subledger to process payment BUT they inadvertently enter the wrong cost center or company code that is responsible for the expense.  Maybe that cost center is “closed” – as in it should not receive any charges – but systematically no one in your organization has actually closed the cost center.  Unheard of, right?

That incorrect data flows up through the general ledger unnoticed.   The general ledger extract gets generated at month end – including this rogue record – and gets processed through FDM.  Let’s assume in HFM that your entity dimension has cost centers as its base members.  So now FDM has loaded data to a base entity that should not have any data.  You might say, no problem, that data will be cleared after I fix that entry in the ledger and reload in Replace mode with FDM.  In most cases you would be right except one.  If no other data exists for the cost center once the errant posting gets corrected, the data in HFM will not be cleared – even when loading in replace mode – when data is reloaded.

Let’s take a moment to explore why this happens.  Conceptually, data not clearing when you load in replace mode this is a good thing.  Why?  Replace is designed to clear data only for the entities that are being loaded.  This allows subsets of data to be loaded to HFM without wiping out data for entities that are not part of the data set.

A problem arises when data has been mistakenly loaded to a base entity that has no other data.  Once that mistake gets corrected, the data will persist in the entity since the entity is no longer part of the data set to be loaded.  The data in HFM is now incorrect.

HFM - Preliminary Load

HFM Data after errant GL posting has been corrected

You might say to yourself, well I’ll just change the process to load zeros to wipe out that data.  While technically that will work, in practice, that is one of the worst decisions you can make.  Zeros are actual data points in HFM and require storage – and if you decided to load zeros – a lot of storage.  Over time, your application performance will suffer.  So loading zeros is out.  You might also think, I’ll just clear the data out of HFM when this happens.  Do you really want to spend the time chasing down all of the records that need to be cleared when FDM can do it for you?

HFM Data when using Smart Replace

Smart Replace is a custom process that systematically clears orphaned HFM data.  This process interrogates the data that is to be loaded to HFM and looks for any base entities that were previously part of the data set but no longer are.  Pulling from our example above, data existed for a cost center in the first load but the posting was corrected so it no longer is part of the general ledger extract.  When a base entity is detected as no longer being part of the data set, FDM clears data from that entity in HFM thereby eliminating the need for you to do so manually.  Your data in HFM is now correct.

This entire process is transparent to the end user of FDM.  As designed, it happens as part of the normal Export workflow process.  This powerful enhancement to FDM ensures that your data quality is further enhanced with no additional day to day effort.

Smart Replace is such a value add that is has become a common – almost standard – part of many of the FDM implementations that I and my team perform.  If you are interested in learning more about it or adding it to your application, feel free to post a comment or contact me.

This entry was posted in Extending FDM and tagged , , . Bookmark the permalink.

2 Responses to Enhancing Data Quality with Smart Replace

  1. sakthihyperion says:

    Hi Tony

    Great stuff ! does this need custom coding ?
    Please explain us , how to implement this feature ?

    Also , I need your advise on performance boosting in FDM application.
    My Application is a simple one with no complex scripting , Logics but i see the load time increases for every month.

    Load time during inception – 4mins
    3rd month – 10 mins
    8th month (now) – 20 mins.

    we loading same volume of data every month !
    Please advise , why we have difference in load time ?


    • Tony Scalese says:


      This functionality is a custom solution. If you are interested in adding it to your application, we can discuss further. You can send an email to info@ranzal.com and this will be routed to me.

      In terms of your performance, FDM performance tuning is a bit of science and a bit of an art. I would suggest starting with your initial implementation partner to address the issues. If you need additional assistance after working with them, feel free to contact me and I’d be happy to discuss how I can help.

Leave a Reply