Azure Databricks Engineer

Location : PAN India (Hybrid)

Experience : 6-9 Years

Must-Have :

- Experience with Azure Databricks , PySpark Notebooks , Spark SQL and Unity Catalog.

- Data Modelling experience with strong SQL knowledge.

Key Responsibilities :

- Develop, maintain, and optimize PySpark notebooks on Azure Databricks for data processing and analytics pipelines.

- Implement and manage Spark Structured Streaming solutions for real-time data processing.

- Work extensively with Spark SQL and Unity Catalog to manage and govern enterprise datasets.

- Design and implement robust data models supporting efficient querying and data integrity.

- Write complex SQL queries and optimize them for performance and scalability.

- Collaborate with cross-functional teams including Data Scientists, Analysts, and Architects to understand data requirements.

Must-Have Skills :

- Hands-on experience with Azure Databricks, PySpark, and Spark SQL

- Proficiency in Spark Structured Streaming

- Experience with Unity Catalog for data governance

- Strong SQL skills for data extraction, transformation, and analysis

- Solid understanding of Data Modeling principles

Good to Have :

- Experience in CI/CD pipelines for Databricks

- Knowledge of Delta Lake, Data Lakehouse concepts

- Familiarity with Azure Data Factory or other orchestration tools