Key Responsibilities:

- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF).

- Build and optimize data ingestion, transformation, and processing pipelines for structured and unstructured data.

- Develop scalable data solutions using Azure Databricks, Azure Data Lake Storage (ADLS), and Azure Synapse Analytics.

- Write efficient and optimized SQL, Python, and PySpark code for data transformation and processing.

- Design and implement analytical data pipelines for big data and relational database environments.

- Collaborate with cross-functional teams to understand business requirements and translate them into scalable data solutions.

- Ensure data quality, performance, scalability, and reliability across the data platform.

- Monitor, troubleshoot, and optimize data workflows and processing jobs.

- Implement best practices for data engineering, security, governance, and performance optimization.

- Support deployment automation using DevOps and CI/CD pipelines.

Required Skills & Qualifications:

Must-Have Skills:

- Strong experience in Azure Data Factory (ADF) for ETL pipeline development.

- Hands-on expertise with Azure Databricks and PySpark.

- Experience working with Azure Data Lake Storage (ADLS).

- Strong knowledge of Azure Synapse Analytics and Azure SQL Database.

- Proficiency in SQL, Python, and PySpark.

- Experience building data ingestion, transformation, and analytical pipelines.

- Strong understanding of big data processing and relational database concepts.

- Experience designing scalable, high-performance data engineering solutions.

- Strong analytical, debugging, and problem-solving skills.