Senior Data Engineer

Job Description

We are looking for a highly skilled Senior Data Engineer with 6+ years of experience, specialising in data engineering and data migration projects. The ideal candidate should be proficient in Azure Data Factory (ADF), ETL/ELT processes, Python/PySpark, and have strong experience building scalable data pipelines across diverse environments.

Key Responsibilities

Design, develop, and implement end-to-end data pipelines across multiple data sources and destinations.
Lead and support data migration projects from legacy systems to modern data platforms.
Develop robust ETL/ELT workflows using Azure Data Factory (ADF).
Write clean, scalable code in Python or PySpark for data transformation and automation.
Perform complex querying, data manipulation, and optimization using SQL.
Collaborate with DevOps teams to manage CI/CD pipelines using Azure DevOps.
Deploy and manage infrastructure using Terraform and other IaC tools.
Ensure alignment with data governance, data warehousing, and dimensional modeling best practices.
Work with cross-functional teams to deliver high-quality and timely data solutions.
Mandatory Skills
6+ years of hands-on experience in data engineering.
Proven experience in data migration projects.
Strong expertise in Azure Data Factory (ADF).
Solid experience in designing and implementing ETL/ELT processes.
Proficient in Python/PySpark for data manipulation and automation.
Expert-level SQL knowledge.
Hands-on experience with Azure DevOps for CI/CD pipelines.
Experience using Terraform for deploying/managing Azure infrastructure.
Strong understanding of data warehousing, dimensional modeling, and data governance.