Holistic data quality solution to improve patient care

Business Context

The hospital group’s vision to be a market-leading healthcare provider and with the mission to improve the lives of people is achieved through delivering high-quality and cost-effective care. Central to performance is the satisfaction of patients, which the hospital group measures through the SPC Quality Scorecard.

• The scorecard is used to track key metrics related to patient and employee safety and experience, however the quality of the underlying data is a concern.
• We worked with the client’s team to drive data quality improvements that enable greater trust in the data and confidence in decisions made.

Challenge

Executives lack trust in scorecard data due to poor quality, hindering decision-making and patient safety.

• The SPC scorecard is generated using data from the warehouse which is sourced from multiple enterprise systems and contains both system generated and manually captured data.
• The reconciliation between the systems present a challenge, there is a lack of central patients view. Making the report available on time for business consumption is challenging due to issues with the underlying data i.e., missing data, incomplete data and timeliness of capture.
• There are currently no defined standards that govern data quality rules, such as the accuracy, completeness, relevance, timeliness and reliability of the data across the landscape.
• This SPC use case highlights a broader need for improved data governance and quality management across the business.

Solution

An integrated data quality solution

We collaborated closely with the hospital group to tackle these challenges by deploying an encompassing Data Quality Solution. The solution included:

• Creation of a Data Quality Solution on Azure Databricks, including a robust ETL process that leveraged Great Expectations to generate input for the data quality report, comprehensively covering critical data tables within the hospital's data landscape.
• Development of a Data Quality Report, significantly enhancing granularity by offering the capability to drill down into various domains, regions, and individual hospitals. The report's delivery frequency was improved from a monthly basis to a more responsive daily update.

The integrated data quality solution, coupled with the revamped SPC scorecard reporting dashboard, effectively pinpoints potential data quality concerns. This facilitates streamlined collaboration between data engineers and hospital specialists, ensuring data integrity and quality enhancement.

A holistic data governance framework

• Establishment of a Data Catalogue within Azure Purview, providing a streamlined platform for data stewards within the hospitals to maintain and update data definitions, ensuring data consistency and quality.
• Formulation of precise Data Governance Rules and Standards specific to the SPC scorecard, reinforcing data accuracy, completeness, relevance, timeliness, and reliability for improved decision-making and reporting reliability.

By embedding data governance policies into technology, data stewards gain confidence that policies are consistently upheld.

First step in migration to the cloud

• Set up the foundation for the data team to use Azure and Databricks for advanced analytics and machine learning projects.