Job Description :

We are looking for an experienced Azure Data Engineer with strong expertise in Azure Data Factory, Azure Databricks, PySpark, and Snowflake to design, develop, and maintain scalable data pipelines.

The ideal candidate should have hands-on experience in ETL development, big data processing, SQL optimization, and integrating data from multiple sources.

Responsibilities :

- Design, develop, test, deploy, and maintain large-scale data pipelines using Azure Data Factory and Azure Databricks.

- Build and optimize ETL/ELT workflows to extract, transform, and load data from various sources into Snowflake.

- Use PySpark and Databricks for large-scale data processing and transformation.

- Develop complex SQL queries and optimize query performance in Snowflake.

- Troubleshoot data pipeline failures, database issues, and performance bottlenecks.

- Collaborate with cross-functional teams to gather requirements and design effective data engineering solutions.

- Ensure data quality, reliability, and consistency across data pipelines and processing workflows.

- Follow best practices for pipeline performance, code quality, and maintainability.

Requirements :

- 6 - 11 years of experience in data engineering, with hands-on experience building large-scale data pipelines.

- Strong experience with Azure Data Factory or similar Azure data integration tools.

- Proficiency in Azure Databricks and PySpark for big data processing and ETL development.

- Strong SQL skills, including query development and performance optimization in Snowflake.

- Experience integrating data from multiple sources into cloud-based data warehouses.

- Strong understanding of ETL/ELT processes and data pipeline design.

- Good analytical, troubleshooting, and problem-solving skills.

Compensation and Platform :

- CTC range: Competitive market salary based on experience.

- Cloud Platform: Must have extensive experience with Microsoft Azure.