Initialization of a new data platform

Data Platform

1 data architect & 2 data engineers

A data platform

that is both observable and accessible

A data warehouse

that is rich and well-documented

A data model

that is strong and future-ready

CHALLENGES

After migrating the the Google cloud (GCP), need to rationalize their data platform
Decommissioning of both « legacy» data warehouses (on Oracle and Snowflake), replaced by a new one built on BigQuery
Strong will of initializing an operating model and a change management plan allowing a wider audience of data citizens to understand and leverage the data
For the Phase 1, focused on the data foundation, reduce the impact on the reporting tools (Tableau and SAP BI4) as much as possible
Build a solid foundation, allowing the group to move toward its transformation ambitions

SOLUTION

Starting with a scoping phase, where we identified the scope, defined the high-level target model and documented the development best practices for dbt and BigQuery
Initialization of a data catalog, to make the full scope accessible to many
Development of a range of dbt models to build the bronze, silver and gold layers of the new datawarehouse
Development of a python toolbox to generate the bronze models and some of the silver ones from the data catalog
Set up elementary, to increase the observability of the platform

BENEFITS