Azure Data Engineer

Job Overview :

We are looking for an experienced Azure Data Engineer with strong expertise in building data pipelines, working with Azure data services, and developing scalable data processing solutions. The ideal candidate should have hands-on experience with PySpark, Microsoft Fabric, Python/Scala/Java, SQL, and big data technologies.

The candidate will be responsible for designing, developing, and maintaining reliable data pipelines and data platforms to support enterprise analytics and business requirements.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines on Microsoft Azure.

- Build and optimize data processing solutions using PySpark.

- Work with Microsoft Fabric for data engineering, processing, and analytics workloads.

- Develop ETL/ELT pipelines to ingest, transform, and load data from multiple sources.

- Work with Azure data stores and services to build scalable data solutions.

- Write complex and optimized SQL queries for data transformation and analysis.

- Develop reusable and maintainable code using Python, Scala, or Java.

- Perform data cleansing, transformation, validation, and quality checks.

- Work with large datasets using distributed and big data processing technologies.

- Optimize data pipelines and queries for performance, scalability, and reliability.

- Troubleshoot data pipeline failures and resolve data processing issues.

- Collaborate with data architects, data scientists, analysts, and business stakeholders.

- Implement data security, governance, and access-control best practices.

- Participate in code reviews, testing, deployment, and production support activities.

- Contribute to Agile ceremonies, sprint planning, and technical discussions.

Required Skills & Experience :

- Hands-on experience as an Azure Data Engineer.

- Strong experience developing data pipelines and working with Azure data stores.

- Hands-on experience with PySpark.

- Experience with Microsoft Fabric and its data engineering capabilities.

- Proficiency in at least one programming language: Python, Scala, or Java.

- Strong experience with SQL and relational databases.

- Good understanding of ETL/ELT processes and data warehousing concepts.

- Familiarity with big data technologies and distributed data processing.

- Strong troubleshooting, analytical, and problem-solving skills.

- Understanding of data engineering best practices and SDLC.