Job Description :

Key Responsibilities :

- Design, develop, and maintain scalable data engineering solutions using Azure Synapse Analytics.

- Develop robust data pipelines using Azure Data Factory (ADF) and other Azure data services.

- Build and optimize large-scale data processing solutions using PySpark and Python.

- Develop and maintain data processing workloads using Databricks.

- Work with Microsoft Fabric and modern cloud-based data platforms.

- Implement Medallion Architecture (Bronze, Silver, Gold) for scalable and governed data processing.

- Design and implement data ingestion, transformation, cleansing, validation, and aggregation processes.

- Develop efficient ETL/ELT pipelines for structured and unstructured data.

- Work with Azure Data Lake / Data Lake Storage for enterprise-scale data storage and processing.

- Develop and optimize data models for data warehousing, reporting, and analytics.

- Write complex and optimized SQL queries for data extraction, transformation, validation, and performance tuning.

- Optimize Synapse, Spark, and data pipeline workloads for performance, scalability, and cost efficiency.

- Participate in large-scale cloud data migration and modernization initiatives.

- Support the migration of legacy data platforms and workloads to modern Azure-based data architectures.

- Build data solutions supporting regulatory reporting and analytics platforms.

- Implement data quality, reconciliation, monitoring, and error-handling mechanisms.

- Collaborate with data architects, business analysts, reporting teams, and other engineering teams.

- Troubleshoot data pipeline and production issues and perform root-cause analysis.

- Follow data governance, security, compliance, and engineering best practices.

Required Candidate Profile :

- Strong hands-on experience in Data Engineering and cloud-based data platforms.

- Hands-on experience with Azure Synapse Analytics is mandatory.

- Strong experience with PySpark and Python.

- Hands-on experience with Azure Data Factory and data pipeline development.

- Experience with Databricks and Spark-based data processing.

- The ideal candidate should possess strong data engineering expertise with hands-on experience in Azure Synapse Analytics, Pyspark, Fabric, Databricks, and modern cloud-based data platforms, designing, developing, and optimizing scalable data solutions.

Key Skills : Pyspark, Azure Synapse Analytics, Azure Data Factory, Microsoft Fabric, Data Pipeline, Azure Data Lake, Data Lake, Data Warehousing, ETL, Data Bricks, Python, SQL.