Azure Data Engineer

Job Summary :

We are seeking a skilled Azure Data Engineer to join our growing data team. The ideal candidate will have hands-on experience with Azure Data Factory, Databricks, PySpark, and SQL, and will contribute to designing and implementing scalable, secure, and high-performance data solutions on Microsoft Azure.

Key Responsibilities :

- Design, develop, and manage data pipelines using Azure Data Factory (ADF) and Databricks.

- Implement and optimize data transformation using PySpark within Azure Databricks.

- Build and manage data integration and orchestration workflows across diverse sources.

- Work with Azure Synapse Analytics for data warehousing and analytics solutions.

- Write and optimize complex SQL queries for data extraction, transformation, and loading (ETL).

- Collaborate with DevOps teams to automate deployment using CI/CD pipelines.

- (Good to Have) Integrate real-time streaming data using Event Hub or similar technologies.

- Follow best practices for data governance, security, and compliance.

- Collaborate with stakeholders to gather requirements and deliver insights.

Required Skills :

Strong hands-on experience with :

- Azure Data Factory (ADF)

- Azure Databricks

- PySpark

- SQL

- Azure Synapse Analytics

- CI/CD and DevOps pipelines

Good to Have :

- Experience with Event Hub or other real-time data streaming platforms.

Certifications (Preferred) :

- Microsoft Certified : Azure Data Engineer Associate (DP-203)

- Databricks Certified Associate Developer

Preferred Attributes :

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration abilities.

- Ability to work independently and manage time effectively in a remote/hybrid setup.

- Experience working in Agile/Scrum environments.