MS Fabric Data Platform

Strategic discovery and implementation of a Microsoft Fabric architecture

Client: Call-centre operator, Asia-Pacific region

4

data marts live — the MVP and three straight after it

Medallion

architecture on Microsoft Fabric

Self-service

BI for business users, not a ticket queue

TypeScriptNode.jsPythonPostgreSQLRabbitMQAWSAmazon EC2AWS compute clusterAWS container servicesAmazon SQSAWS registryData centerAzure Kubernetes ServiceAzure Key VaultAzure DevOps servicesAzure Data Lake Storage Gen2Microsoft platformPower BICEPH S3-compatible storageTerraformPlaywrightSeleniumJasmineKarma
TypeScriptNode.jsPythonPostgreSQLRabbitMQAWSAmazon EC2AWS compute clusterAWS container servicesAmazon SQSAWS registryData centerAzure Kubernetes ServiceAzure Key VaultAzure DevOps servicesAzure Data Lake Storage Gen2Microsoft platformPower BICEPH S3-compatible storageTerraformPlaywrightSeleniumJasmineKarma

Challenge

The existing data landscape had to be understood before anything could be built on top of it — sources spread across on-premises systems and GCP, and a business that needed working data marts sooner than a full platform programme would deliver.

Solution

A target architecture on Microsoft Fabric following the Medallion pattern, delivered MVP-first: the first data mart and the core platform stood up together, with connectivity to the on-prem and GCP sources built in from the start.

Implementation

  • 01Discovery phase analysing the existing data landscape and business requirements
  • 02Target platform architecture on Microsoft Fabric, Medallion pattern
  • 03MVP for the first data mart alongside the core platform and source connectivity
  • 04Three further data marts delivered on the same foundation
  • 05PII masking, historical data migration, data quality pipelines, automated incident notifications and granular access security
  • 06Standardized pipeline templates, automated SDLC and BI self-service for business users

Business impact

Four data marts in production the MVP and three delivered straight after

Governance built in rather than retrofitted: masking, quality pipelines, granular access

Business users served by self-service BI and standardized pipelines

Technology stack

Platform
  • Microsoft Fabric
  • Azure
Data & Processing
  • Spark SQL
  • pySpark
  • SQL
  • Google BigQuery
DevOps
  • Terraform

Have a complex system to build or modernize?

Tell us what must change. We will bring the relevant domain and engineering leads into the conversation.

Explore all case studies