A medallion warehouse on Microsoft Fabric
Building an end-to-end analytics platform: ingestion from operational sources, layered transformation in PySpark, and a reporting model analysts query directly. Work carried out within Itransition's commercial data engineering programme.
- Microsoft Fabric
- PySpark
- Power BI
- InfluxDB
- SQL
/The problem
Operational data arrived from several sources with inconsistent types, duplicated keys and no shared definition of the entities involved. Reporting on it directly meant every question started with a discussion about what the numbers meant.
/The approach
A medallion architecture: a bronze layer kept exactly as ingested and never edited, a silver layer where types are enforced and duplicates resolved, and a gold layer modelled as a star schema with declared grain. Transformations run in PySpark and are reproducible, so a bad result is fixed by rerunning a layer rather than patching rows. Time-series measurements from InfluxDB are aligned to the same dimensions so they can be reported alongside relational facts.
/The outcome
A warehouse where the reporting layer can be queried without prior knowledge of the source systems, and Power BI reports built on conformed dimensions rather than on individual extracts. Completed as part of Itransition's Commercial Software Development: Data Engineering programme, 2026.
/Services involved