Role : Azure Data Engineer

Experience : 5 to 12 yrs

Location : Pune

Key Responsibilities :

- Design and build ETL/ELT pipelines using Databricks, PySpark and SQL

- Implement batch and incremental data processing patterns for enterprise datasets

- Apply data engineering best practices for performance scalability and reliability

- Develop and optimize Spark jobs notebooks and workflows in Databricks

- Ensure data quality validation and reconciliation across pipelines

- Support data modeling and transformation for analytics and reporting use cases

- Integrate Databricks pipelines with upstream and downstream systems

- Participate in production support monitoring and incident resolution

Core Skills Required :

- Strong hands on experience with Databricks and Apache PySpark

- Programming skills in Python, PySpark and SQL for efficient data transformation and processing ETL

- Solid understanding of ETL/ELT patterns and data pipeline orchestration

- Big Data Processing Experience with big data frameworks specifically Apache Spark for processing large datasets

- Data Warehousing Modelling Understanding of data warehousing principles and scalable data modelling techniques

- Experience with data engineering best practices and optimization techniques

- Hands on exposure to enterprise data engineering workflows

- Experience supporting production data pipelines at scale

- Familiarity with analytics reporting or AI driven data consumption use cases

Good to have :

- Experience in working with Opensource storage formats such as Apache Delta Apache Parquet would be beneficial

- Awareness of compliance regulations relevant to data handling and processing

- Knowledge of data security practices including encryption access control and data governance

- Knowledge of other Azure services that integrate with ADF and ADB such as Azure Functions Azure Logic Apps and Azure Event Hubs