Designing and Implementing an Automated Solution for Regulatory Compliance
“Melio is the shining example to the rest of the data team.”
Chief Data Officer - Middle East Digital Bank
Case Study
Industry
Banking
Company Size
201-500 employees
Problem Space
Insights
Automation
Data Engineering
Software Engineering
Regulatory Compliance
Technology Used
Python
Apache Airflow
PostgreSQL
Kubernetes
Flyway
Business Context
An emerging new digital bank in the Middle East enlisted Melio to help engineer an automated pipeline for their regulatory compliance needs. The pipeline needed to manage regulatory report generation while adhering to compliance requirements stipulated by the bank.
Several key challenges faced by the bank included:
- Lack of appropriate technical and regulatory skills available within the existing data team.
- Tight time frames to design, build and deploy the automated solution to meet the central bank’s launch requirements.
- Reduce manual effort required in compiling regulatory reports.
- Deploy a solution that could scale to over 250 reports required by the central bank.
Challenge
Overcoming critical business and technical challenges with the central bank
The digital bank needed to find an engineering partner that could help them to:- Create a data-driven solution that would adhere meticulously to regulatory protocols.
- Ensure that the solution was automated reducing manual intervention and errors.
- Address constant data challenges across multiple disparate financial systems in an environment where there is no “real world” data to work with.
The role that Melio stepped in to play was that of review and refactor. Our primary challenge was to salvage a failed solution implemented by another vendor. After completing a technical review of the existing solution, it was evident that the solution would require a complete rebuild from scratch.
The bank's technical and data teams were under immense pressure to deliver multiple projects for the bank and were unable to dedicate the time needed to rebuild the reporting solution. The team also lacked the required skills and capacity to develop an automated pipeline for processing the mandatory regulatory reports. With the bank's strict adherence to tight deadlines there was little room for up-skilling their technical teams within the available time frame.
Solution
Building a robust automated reporting solution
As our entry point into the project, we completed a technical review on the previous vendor’s solution. This was also our first opportunity to demonstrate our technical expertise and strategic advisory role we would continue to play through the duration of the project. The new solution was designed to scale and orchestrate the vast amount of required regulatory reports that would be enabled via the digital bank’s enterprise data warehouse.
Key components of the robust automated reporting solution
- Data sourcing and ingestion from the data warehouse
- Report orchestration and scheduling using Apache Airflow and python
- Data quality and validation utilising Great Expectations and Pydantic python libraries
- Data storage and reporting utilising Oracle Cloud, Postgres and MS Excel
- Deployment into Kubernetes to support scaled operations over time
The need to utilise MS Excel was driven by the central bank’s requirement that all banks supply weekly, monthly, quarterly and annual reports by populating their own Excel based template. The flexibility of python helped us to solve this challenge for the bank.
To overcome having little to no data to test with, our team designed and implemented a synthetic data generator that replicated financial transactions and accounts that was used to drive the process with business.
The bank’s data team was faced with many challenges throughout the duration of the project. This impacted the broader team as well as the Melio project team and deliverables. Melio ended up playing a strong advisory role to the Chief Data Officer. We helped the data team increase their maturity in developing and deploying data solutions using best practice software engineering and agile ways of working.
Automated end-to-end regulatory report solution
The automated reporting platform delivered the following key benefits for the bank:
💸
Time Savings: Significantly minimise manual effort and time needed for regulatory report generation, validation and delivery to business.
🚀
Built-in Data Quality Measures: Quality controls were built in to help identify and remedy data quality issues as they occur, enhancing the overall QA reporting process.
✅
Unified Issue Resolution: Effectively aid business and technical teams to swiftly address issues and errors affecting the data, platform or report process.
👫
Empowered Technical Teams: Enable technical teams to leverage new tools and technologies and apply stronger software engineering practices in the team.
🧱
Scale and Deploy: The bank required over 250 reports to be created. Our solution provided the platform and enablement for the data team to scale the solution to new regulatory reports.