Description :

Exp - 6-10 Years

Location - Chennai ,Hyderabad , Bangalore

Notice : Immediate to 30 days only

Skill : Azure Data Engineer

Must have : Azure Databricks, Pysparks, SQL, Devops(CI/CD), Azure services(ADF, ADLS, Event Hub)

Good to have : Data modelling, Gen AI, Agentic AI, Streaming Analytics

Key Responsibilities :

- Design and implement scalable, metadata-driven frameworks for data ingestion, quality, and transformation across both batch and streaming datasets.

- Develop and optimize end-to-end data pipelines to process structured and unstructured data, enabling the creation of analytical data products.

- Build robust exception handling, logging, and monitoring mechanisms for better observability and operational support.

- Take ownership of complex modules and lead the development of critical data workflows and components.

- Provide guidance to data engineers and peers on best practices.

- Collaborate with cross-functional teams including business consultants, data architects, scientists, and application developers to deliver impactful analytics solutions.

EXPERTISE AND QUALIFICATIONS :

- 5+ years of overall technical experience, with a minimum of 2 years of hands-on experience with Microsoft Azure and Databricks.

- Proven experience delivering at least one end-to-end Data Lakehouse solution on Azure Databricks using the Medallion Architecture.

- Strong working knowledge of the Databricks ecosystem, including: PySpark, Notebooks, Structured Streaming, Unity Catalog, Delta Live Tables, Workflows, and SQL Warehouse.

- Advanced programming, unit testing, and debugging skills in Python and SQL.

- Hands-on experience with Azure-native services such as: Azure Data Factory, ADLS Gen2, Azure SQL Database, and Event Hub.

- Solid understanding of data modeling techniques, including both Dimensional and Third Normal Form (3NF) models.

- Exposure to developing LLM/Generative AI-powered applications.

- Must have excellent understanding of CI/CD workflows using Azure DevOps.

- Bonus: Knowledge of Azure infrastructure, including provisioning, networking, security, and governance.