Designing and Implementing an Automated Solution for Regulatory Compliance

Business Context

An emerging new digital bank in the Middle East enlisted Melio to help engineer an automated pipeline for their regulatory compliance needs. The pipeline needed to manage regulatory report generation while adhering to compliance requirements stipulated by the bank.

Several key challenges faced by the bank included:

• Lack of appropriate technical and regulatory skills available within the existing data team.
• Tight time frames to design, build and deploy the automated solution to meet the central bank’s launch requirements.
• Reduce manual effort required in compiling regulatory reports.
• Deploy a solution that could scale to over 250 reports required by the central bank.

Challenge

Overcoming critical business and technical challenges with the central bank

The digital bank needed to find an engineering partner that could help them to:

• Create a data-driven solution that would adhere meticulously to regulatory protocols.
• Ensure that the solution was automated reducing manual intervention and errors.
• Address constant data challenges across multiple disparate financial systems in an environment where there is no “real world” data to work with.

The role that Melio stepped in to play was that of review and refactor. Our primary challenge was to salvage a failed solution implemented by another vendor. After completing a technical review of the existing solution, it was evident that the solution would require a complete rebuild from scratch.

The bank's technical and data teams were under immense pressure to deliver multiple projects for the bank and were unable to dedicate the time needed to rebuild the reporting solution. The team also lacked the required skills and capacity to develop an automated pipeline for processing the mandatory regulatory reports. With the bank's strict adherence to tight deadlines there was little room for up-skilling their technical teams within the available time frame.

Solution

Building a robust automated reporting solution

As our entry point into the project, we completed a technical review on the previous vendor’s solution. This was also our first opportunity to demonstrate our technical expertise and strategic advisory role we would continue to play through the duration of the project. The new solution was designed to scale and orchestrate the vast amount of required regulatory reports that would be enabled via the digital bank’s enterprise data warehouse.

Key components of the robust automated reporting solution

• Data sourcing and ingestion from the data warehouse
• Report orchestration and scheduling using Apache Airflow and python
• Data quality and validation utilising Great Expectations and Pydantic python libraries
• Data storage and reporting utilising Oracle Cloud, Postgres and MS Excel
• Deployment into Kubernetes to support scaled operations over time

The need to utilise MS Excel was driven by the central bank’s requirement that all banks supply weekly, monthly, quarterly and annual reports by populating their own Excel based template. The flexibility of python helped us to solve this challenge for the bank.

To overcome having little to no data to test with, our team designed and implemented a synthetic data generator that replicated financial transactions and accounts that was used to drive the process with business.

The bank’s data team was faced with many challenges throughout the duration of the project. This impacted the broader team as well as the Melio project team and deliverables. Melio ended up playing a strong advisory role to the Chief Data Officer. We helped the data team increase their maturity in developing and deploying data solutions using best practice software engineering and agile ways of working.