Job Title : Senior Data Engineer

Experience : 7 - 9 Years

Location : Remote

Notice Period : Immediate Joiners Only

About the Engagement :

We are looking for experienced Senior Data Engineers to join a large-scale, multi-year Data Mapping and DataOps Platform modernisation programme for a major government ministry.

This strategic initiative focuses on transforming how data is managed, governed, and leveraged as a core enterprise asset.

Key Focus Areas :

- Data Engineering : Designing, building, and optimising scalable data pipelines to enhance stability, traceability, and refresh frequency

- Data Mapping & Metadata Capture : Onboarding critical financial systems into a centralised enterprise data catalogue and supporting governance initiatives

Work will be delivered in an Agile/Scrum model, collaborating closely with data product managers and cross-functional teams.

Platform Overview :

The platform is built on Microsoft Azure and Microsoft Fabric, with an existing in-house Finance Data Catalogue (FDC).

This role will contribute to scaling the platform into a production-grade DataOps ecosystem.

Key Responsibilities :

- Design, build, optimise, and maintain scalable data pipelines

- Develop and manage ELT pipelines, orchestration, and automation using Python

- Capture and onboard metadata into enterprise data catalogues

- Perform data analysis to identify patterns, anomalies, and quality improvements

- Contribute to data product design, including data models and structures

- Design and implement data models for analytics and reporting

- Build and maintain CI/CD pipelines for automated deployment

- Develop automation scripts using Python, Bash, and PowerShell

- Develop and maintain RESTful APIs for integration and interoperability

- Implement data quality checks and validation frameworks

- Set up monitoring, alerting, and performance tracking

- Develop dashboards and reporting solutions using Microsoft Fabric & Power BI

- Collaborate with stakeholders to translate business needs into technical solutions

- Ensure compliance with data governance, privacy, and security standards

- Maintain technical documentation and communicate with both technical and business teams

Mandatory Requirements :

Candidates must have recent (last 48 months) hands-on production experience in :

- Python (data pipelines, scripting, automation)

- SQL (data transformation, modelling)

- Microsoft Azure & Microsoft Fabric

- ELT Pipeline Development

- Git (version control & collaboration)

- Metadata Management & Data Catalogue onboarding

- Data Product Design

- Agile/Scrum methodologies

- Data Analysis & Data Quality Assessment