Job Summary :

We are seeking a highly skilled Senior Data Engineer with strong expertise in the Microsoft Azure Data Platform to design, develop, and optimize enterprise-scale data solutions. The ideal candidate will have hands-on experience in Azure Databricks, Lakehouse Architecture, Azure Data Factory, Microsoft Fabric, Synapse Analytics, and ADLS Gen2, along with advanced programming skills in PySpark, Scala, and Python. This role involves building scalable data pipelines, modernizing ETL processes, and enabling data-driven business decisions through robust data engineering practices.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines and ETL/ELT workflows on the Microsoft Azure ecosystem.

- Architect and manage enterprise-scale Databricks and Lakehouse solutions for analytics and reporting.

- Lead ETL modernization initiatives by migrating legacy data platforms to modern cloud-native architectures.

- Build high-performance data processing solutions using PySpark, Scala, and Python.

- Develop and optimize data ingestion, transformation, and integration processes using Azure Data Factory (ADF).

- Design and maintain enterprise data lakes using Azure Data Lake Storage Gen2 (ADLS Gen2).

- Implement data solutions using Microsoft Fabric and Azure Synapse Analytics to support business intelligence and advanced analytics.

- Optimize Spark workloads, data models, and query performance for large-scale datasets.

- Collaborate with data architects, analysts, and business stakeholders to understand requirements and deliver scalable solutions.

- Implement data quality, governance, monitoring, and security best practices.

- Develop reusable frameworks, documentation, and technical standards for data engineering projects.

- Troubleshoot production issues and continuously improve data platform reliability and performance.

Required Skills :

- 7+ years of experience in Data Engineering with a strong focus on Microsoft Azure technologies.

- Extensive hands-on experience with :

1. Azure Databricks

2. Azure Data Factory (ADF)

3. Microsoft Fabric

4. Azure Synapse Analytics

5. Azure Data Lake Storage Gen2 (ADLS Gen2)

- Strong expertise in designing and implementing Lakehouse Architecture.

- Proven experience leading enterprise-scale ETL modernization projects.

- Advanced programming skills in :

1. PySpark

2. Scala

3. Python

- Strong knowledge of SQL, data modeling, and data warehousing concepts.

- Experience building scalable batch and real-time data pipelines.

- Understanding of distributed computing, Spark optimization, and performance tuning.

- Experience working with cloud-native data engineering solutions and enterprise data platforms.

- Strong analytical, debugging, and problem-solving skills.