Description:
The DPAS modernization / tech refresh effort has changed the DPAS model of a single database source to a database per microservice model. With this change, there is a need to having monitoring tools to ensure synchronization is occurring. The tools need to compare the sync data the source of truth data. When the data is not in sync, the tools need to systematically report the condition to the DPAS administration team in order to have the issue resolved in a timely manner. As the number of synchronizations grow in number, it's no longer feasible / economical to perform the checks manually.
Recommended:
Create a process that will monitor all of the DPAS synchronization processes. The tool should support easy setup of new sync processes and the removal of sync processes that are no longer used. The UI should provide for the user set the each sync process as active/inactive, and set the frequency of the check, what job to run to perform the check, the auto correction routine to run (auto/manual) when an issue is encountered. When the resync is set to auto, the system when it encounters an issue, should run the auto sync without user intervention. When set to manual the, the UI should provide for kick-off of the sync when user permissions grant that access. There should be a log file maintained of each sync check, datetime run, status of the (failed to run/ran to completion),and the results (in balance/out of balance). The log file should also contain the re-sync that was performed, how they were run (auto or manual), when they were run, status and the number of rows that were re-synced. Upon successful re-sync, the system should auto run the sync process to ensure the system did come back into balance. There needs to be so checks to ensure these processes do not go into an eternal cycle of validating, re-sync, validating, re-sync due to a bug in one of the processes, a permission issue, or when the users are very active with the system being out of sync as soon as it is re-synced. There also needs to be a reporting mechanism. The tools need to be able to report out balances that are encountered where the system is not set to re-sync, and when an auto sync does not bring the system into balance. It is recommended a summary of the day's results be included in the DPAS daily status report, and there should be a UI that enable the user to view the log files.
Mission Critical:
This changes is required due to the new architecture. It will provide systematically validation that the system is functioning as it should.
Benefits:
This will correct current deficiencies and help development teams identify and enhance the processes that are deficient as well as being able to identify issues prior to them resulting in help desk tickets.
Frequency: Daily
Users:
Estimated number of users that will encounter synchronized data that could be verified in advance is 21,000.
