Saturday, April 25, 2020

// // 7 comments

FDMEE: System Maintenance Tasks: Maintain Application Folder

Hi Friends,

This is the third post under Oracle Hyperion Financial Data Quality Management (FDMEE): System Maintenance Tasks series where we are going to cover Maintain Application Folder script.

If you have not yet read previous two posts of this blog series, I would suggest you to go through them here to get a comprehensive picture of Oracle Hyperion Financial Data Quality Management (FDMEE) System Maintenance Tasks:

FDMEE: System Maintenance Tasks: Maintain FDMEE Data Tables
FDMEE: System Maintenance Tasks: Maintain ODI Session Data

In this third post, we will focus on the ‘Maintain Application Folder’ task and will see how to manually execute ‘Maintain Application Folder’ purge script.  

Important Note:
  • This post has been written and associated activities have been demonstrated on Oracle Hyperion Financial Data Quality Management (FDMEE) version 11.1.2.4.210.
  • The demonstrating Hyperion environment has Oracle database server 12.2.0.2 (18c) as backend database. 
  • There is no outage required on the Oracle Hyperion Financial Data Quality Management (FDMEE) application to execute these purge scripts. But it is advisable not to run any data loads during the purge activity.
  • Based on the data retention policy of your organization, you should pre-decide how many days of data you want to retain for FDMEE Application folders.
  • Don't forget to take complete Oracle Hyperion Financial Data Quality Management (FDMEE) schema backup before attempting to run these purge scripts.
Maintain Application Folder

Whenever you load data (files) through Hyperion Financial Data Quality Management (FDMEE), there are three types of files stored:
  1. The file which you uploaded, is stored in the inbox directory
  2. The export file, which is stored in the outbox directory
  3. The data file, which is stored in the data directory.
All these files remain on your FDMEE server or network location (server) and often gradually cause your disk storage space to get full. These files themselves are not that important, as the data is stored in both, the FDMEE database as well as your target EPM application (Essbase, Planning, HFM, etc.) database.

The “Maintain Application Folder” script is used to clean up FDMEE Inbox, Outbox, and Data (excluding Data\Scripts subfolder) folders by the number of days entered as an input parameter for these folders.

Hyperion Financial Data Quality Management (FDMEE) checks the inbox and outbox subfolders under the respective folders and deletes any files present over there. In the data subfolder, FDMEE skips the scripts subfolder because it contains your custom, event, and import scripts. Cleanup also includes Reports and Logs under the Outbox folder that are associated with Process IDs.

Maintain Application Folder (deleting old files from FDMEE inbox, outbox and Data directories) script saves you a lot of disk space as these folders' size gets bigger with time, if not housekept regularly.

When you execute ‘Maintain Application Folder’ purge script, it seeks following input parameters to be entered:

  • Target Application
  • Days to keep Inbox directory
  • Days to keep Outbox directory
  • Days to keep Data directory
If the Days to keep value is not specified for a specific folder (Inbox/Outbox/Data), FDMEE skips that folder for file deletion.

Step-by-Step process to execute ‘Maintain Application Folder’ purge script

1- Log in to Workspace with Admin credentials.

2- Open Hyperion Financial Data Quality Management (FDMEE) console (Data Management) from Workspace and on the Workflow tab, under Other, select Script Execution:

FDMEE: System Maintenance Tasks: Maintain Application Folder

3In Script Execution, and then in Custom Script Group, select System Maintenance Tasks.

FDMEE: System Maintenance Tasks: Maintain Application Folder

4- From the Scripts grid, select Maintain Application Folder script and Click Execute.

FDMEE: System Maintenance Tasks: Maintain Application Folder

5It will prompt to enter the following input parameters: 

FDMEE: System Maintenance Tasks: Maintain Application Folder

6- Click on the search icon for Target Application and select the application name from the list.

FDMEE: System Maintenance Tasks: Maintain Application Folder

7- Now enter Days to keep for Inbox, Outbox and Data directories.

For example, to retain files under these three folders for the last 3 months, enter 90 (days):

FDMEE: System Maintenance Tasks: Maintain Application Folder

8- Keep Execution mode selected as Online. The online method processes the report immediately.

Execution modes:
online  — ODI processes the data in sync mode (immediate processing).
offline — ODI processes the data in async mode (runs in background).

FDMEE: System Maintenance Tasks: Maintain Application Folder

7- Optionally, If you want to schedule the purge script run rather than running it instantly, Click on Schedule option and enter the scheduling details as per your requirement.

FDMEE: System Maintenance Tasks: Maintain Application Folder

FDMEE: System Maintenance Tasks: Maintain Application Folder

8- If you want to instantly run the purge script, then directly click on OK (not on the Schedule option).

You will see a message informing the process ID of your script run as shown below:

FDMEE: System Maintenance Tasks: Maintain Application Folder

9- To see the status of your script execution, click on the Process Details link under the Workflow tab—Monitor—Process Details

FDMEE: System Maintenance Tasks: Maintain Application Folder

10- Above we can see our script with Process ID 8144 has executed successfully. 

Click on the Show button for Process ID 8144 to open the corresponding log file.

FDMEE: System Maintenance Tasks: Maintain Application Folder

In the log file, look for the message "Maintain Application Folder: Completed" to confirm that script has run successfully without any error. If you encounter any errors, troubleshoot them based on the cause mentioned in the log file. In our case, we see no issue in the script execution.

Note, you need to repeat all of the above steps for other target applications too where your FDMEE data loads are happening. 

Once you are done with running the Maintain Application Folder purge script for all of your selected target applications, you can check FDMEE Inbox, Outbox and Data directories to verify the deleted files.

Maintain Application Folder purge script can be automated to run using Executescript batch file located in the folder: 

<drive>\Oracle\Middleware\user_projects\epmsystem1_fdm\FinancialDataQuality.

We will cover the automation part in a separate post of this blog series. 

That's all for this post.

I hope this article has helped you. 
Your suggestions/feedback are most welcome.
Keep learning and Have a great day!!!    

7 comments:

  1. Great post.

    Quick question -
    If you run this, won't it also clear the records from your FDMEE Workbench?

    ReplyDelete
    Replies
    1. @Saurabh...I’m glad you liked the post. Thanks.
      The data you see in the FDMEE Data Load Workbench resides in the TDATASEG table of FDMEE schema. When you run "Maintain FDMEE Data Tables" purge script, it deletes the data from many tables including TDATASEG (FDMEE Data Load Workbench data). Please refer this post for more details: FDMEE: System Maintenance Tasks: Maintain FDMEE Data Tables

      Delete
    2. @Sunil - thank you for your reply :)
      I don't really want to clear records from the FDMEE workbench. Do you mean if I only run the 'Maintain Application Folder' it won't clear any records from FDMEE workbench? Apparently there are a lot of .dat files. Are those used by FDMEE to show the records in the workbench?

      Delete
    3. @Saurabh...unless you delete all the records from TDATASEG table (which we never do), you should be able to see data in Workbench. Moreover the .dat files what you see under Application outbox directory are generated after the export of your data to target applications. Point to note is that in all the cases, we are deleting only the historical data (older than 6 months or 1 year) whether it is FDMEE tables or FDMEE application folders. So you will always be having the latest data available in your tables, application folders as well as in Workbench.

      Delete
  2. Hi Sunil,

    Thank you for the Great posts. I have noticed .DBF files are taking lot of space on our FDMEE server, Any idea which of these system maintenance Tasks purge or reduce size of the .DBF files under oradata folder

    ReplyDelete
    Replies
    1. @Ganesh....These .DBF files are database files and in your case probably it is your backend FDMEE schema related DB files.
      Have you installed your backend relational database on the same FDMEE server? What is the file path where you observe these .DBF files on your FDMEE server?

      Delete
    2. Thank you Sunil, sorry for the delay in response
      Yes relational database was installed on the same server
      The file path is E:\oradata\FDMEEPROD

      Delete