Implementation of a Modern Data Platform
- Practice
- Team
1 Architect, 1 Sr Business Analyst, 1 Sr Data Engineer, 1 Data Engineer
- Technical environment
CHALLENGES
- Set up a centralized platform for data from the 15 banking subsidiaries in Africa (excluding Morocco).
- Reconcile all data from the central banking system as well as from additional source systems specific to each subsidiary.
- Set up a shared, robust platform capable of handling different data usage scenarios within the group.
SOLUTION
- Architecture study: A comprehensive study of architectural requirements enables us to select an ambitious target and the most appropriate components for strategic choices.
- Parametric development: The processes were developed in a fully parameterized way, enabling the reuse of developments in different subsidiaries.
- Data ingestion: Data ingestion and scheduling were transferred to Airflow via Azure Data Factory.
- Agile technical workshops: Technical workshops conducted in agile mode enabled us to find optimal solutions for resource utilization.
BENEFITS
- Centralized platform: A single platform centralizing all data from the 15 banking subsidiaries.
- Process optimization: Process optimization compared with the previous pilot architecture: 4 to 5 times improvement in processing times and 2 to 3 times improvement in data storage efficiency.
- Robust processes: Robust and flexible processes for data ingestion, transformation and recycling, facilitating maintenance.
- Aggregated data view: A new aggregated view of data across the African perimeter, opening up a multitude of new use cases accessible in a timely manner.
