Holistic data quality solution to improve patient care
“This solution paves the way for a culture of enhanced data governance and quality management.”
Executive: Analytics and Product Development - Large Hospital Group
Case Study
Industry
Hospital Group
Company Size
5000+ employees
Problem Space
Data quality
Data governance
Data-driven insights
Data engineering
Data visualisation
Technology Used
Databricks
Azure Purview
Python
Great Expectations
Power BI
Business Context
The hospital group’s vision to be a market-leading healthcare provider and with the mission to improve the lives of people is achieved through delivering high-quality and cost-effective care. Central to performance is the satisfaction of patients, which the hospital group measures through the SPC Quality Scorecard.
- The scorecard is used to track key metrics related to patient and employee safety and experience, however the quality of the underlying data is a concern.
- We worked with the client’s team to drive data quality improvements that enable greater trust in the data and confidence in decisions made.
Challenge
Executives lack trust in scorecard data due to poor quality, hindering decision-making and patient safety.
The hospital group's commitment to delivering high-quality, cost-effective healthcare relied on the SPC Quality Scorecard to monitor patient and employee safety and satisfaction. However, concerns arose due to inconsistent data quality, affecting trust and decision-making. The following are identified as key challenges:- The SPC scorecard is generated using data from the warehouse which is sourced from multiple enterprise systems and contains both system generated and manually captured data.
- The reconciliation between the systems present a challenge, there is a lack of central patients view. Making the report available on time for business consumption is challenging due to issues with the underlying data i.e., missing data, incomplete data and timeliness of capture.
- There are currently no defined standards that govern data quality rules, such as the accuracy, completeness, relevance, timeliness and reliability of the data across the landscape.
- This SPC use case highlights a broader need for improved data governance and quality management across the business.
Solution
An integrated data quality solution
We collaborated closely with the hospital group to tackle these challenges by deploying an encompassing Data Quality Solution. The solution included:
- Creation of a Data Quality Solution on Azure Databricks, including a robust ETL process that leveraged Great Expectations to generate input for the data quality report, comprehensively covering critical data tables within the hospital's data landscape.
- Development of a Data Quality Report, significantly enhancing granularity by offering the capability to drill down into various domains, regions, and individual hospitals. The report's delivery frequency was improved from a monthly basis to a more responsive daily update.
The integrated data quality solution, coupled with the revamped SPC scorecard reporting dashboard, effectively pinpoints potential data quality concerns. This facilitates streamlined collaboration between data engineers and hospital specialists, ensuring data integrity and quality enhancement.
A holistic data governance framework
- Establishment of a Data Catalogue within Azure Purview, providing a streamlined platform for data stewards within the hospitals to maintain and update data definitions, ensuring data consistency and quality.
- Formulation of precise Data Governance Rules and Standards specific to the SPC scorecard, reinforcing data accuracy, completeness, relevance, timeliness, and reliability for improved decision-making and reporting reliability.
By embedding data governance policies into technology, data stewards gain confidence that policies are consistently upheld.
First step in migration to the cloud
- Set up the foundation for the data team to use Azure and Databricks for advanced analytics and machine learning projects.
Improved data quality builds trust in the scorecard, enabling confident data-driven decisions.
Through this collaboration, the client realised enhanced data quality and confidence in the SPC Quality Scorecard. This project addressed the immediate data quality challenges while laying the foundation for a broader culture of improved data governance and quality management across the organisation.
🌟
~30% improvement in data quality through rigorous governance rules, increasing trust in the SPC Quality Scorecard.
💼
Daily reporting enhanced decision-making agility and reducing operational latencies.
📈
Data governance rules and standards for the SPC scorecard resulted in a ~25% improvement in overall report reliability.
💪
Strengthened data cataloging empowered data stewards, ensuring data definitions are consistent and up-to-date.