Lead Data Engineer (Microsoft Fabric)

We are looking for an experienced Microsoft Fabric Engineer to architect and deliver enterprise-scale data platforms using the Microsoft Fabric ecosystem. The role involves designing OneLake, Lakehouse, data pipelines, and data transformation solutions while modernising legacy data workloads.

Responsibilities:

- Lead the design and implementation of end-to-end Microsoft Fabric data solutions, including Lakehouse, Data Warehouse, Data Factory, and Real-Time Analytics.

- Architect and govern OneLake structures, workspace organisation, data domains, access controls, and platform standards.

- Build and maintain scalable data pipelines using Azure Data Factory and Fabric Data Pipelines.

- Develop and optimise PySpark transformation logic using Fabric Notebooks and Spark workloads.

- Design and optimise Delta/Parquet-based Lakehouse architectures, including medallion architecture and DirectLake integration with Power BI.

- Assess, migrate, and modernise legacy SSIS packages and SSRS reports using Fabric, ADF, and Power BI.

- Implement CI/CD, monitoring, governance, security, and infrastructure automation using Azure DevOps/GitHub Actions, Bicep, or Terraform.

- Lead technical discussions and reviews, mentor data engineers, and collaborate with analysts, data scientists, and business stakeholders.

Requirements:

- 8+ years of experience in Data Engineering, with at least 2 years of hands-on experience with Microsoft Fabric or closely related Azure data platforms.

- Strong hands-on expertise across Microsoft Fabric workloads, including Lakehouse, Data Warehouse, Data Factory, OneLake, and Real-Time Analytics.

- Deep experience with Azure Data Factory, including pipeline design, orchestration, linked services, triggers, parameterisation, monitoring, and migration to Fabric.

- Strong PySpark, Python, and SQL skills with experience in Spark optimisation, partitioning, caching, broadcast joins, and shuffle optimisation.

- Proven experience designing OneLake/Lakehouse architectures, medallion architecture, Delta/Parquet, data domain management, and DirectLake integration.

- Hands-on experience with SSIS and SSRS, including ETL development, report development, and migration to modern Fabric/Power BI solutions.

- Strong Azure fundamentals including ADLS Gen2, Azure Synapse, Key Vault, RBAC, CI/CD, and exposure to Terraform/Bicep.

- Strong technical leadership and communication skills; DP-600, DP-203, or equivalent certifications and experience with Purview, Eventstream, Power BI Premium, or data mesh are preferred.