Job Title : Azure Databricks Engineer

Experience : 6 to 8 Years

Location : Bangalore, Hyderabad

Job Summary :

We are looking for an experienced Azure Databricks Engineer to design, build, and maintain scalable data engineering solutions on the Azure cloud platform. The ideal candidate will have strong hands-on experience with Azure Databricks, PySpark, and Azure Data Lake Storage, along with exposure to Apache NiFi for data ingestion and flow management. You will work closely with IT teams, business stakeholders, and customers to deliver reliable, high-performance data pipelines.

Key Responsibilities :

- Design, develop, and maintain scalable ETL pipelines using Azure Databricks and PySpark

- Integrate Databricks with Azure Data Lake Storage (ADLS) as part of a modern data architecture

- Schedule, orchestrate, and monitor Databricks workloads using Databricks Workflows

- Build and manage data ingestion and transformation pipelines using Apache NiFi

- Perform incident restoration, regular application maintenance, and system health checks

- Optimize code performance and automate routine operational tasks

- Work with load-balanced, high-traffic data services ensuring reliability and scalability

- Collaborate closely with IT teams, business owners, data scientists, and business analysts

- Participate actively in Agile ceremonies including stand-ups, sprint grooming, demos, and solutioning sessions

- Engage directly with customers when required to understand requirements and explain technical solutions

Must-Have Skills :

- Strong hands-on experience with Azure Databricks, SQL, T-SQL, ETL development and big data pipelines

- Proficiency in Python, PySpark and Spark-based data processing

- Solid experience with Azure Data Lake Storage (ADLS) and cloud data architectures

- Familiarity with Apache NiFi for data ingestion, transformation, and routing

- Strong understanding of cloud-based data engineering and storage systems

- Excellent problem-solving skills to troubleshoot and optimize ETL workflows

Good-to-Have Skills :

Experience with additional Azure services such as :

1. Azure Synapse Analytics

2. Azure Data Factory (ADF)

3. Azure Functions

- Understanding of streaming data and real-time data integration

- Strong collaboration skills to work effectively across cross-functional teams

- Good communication skills for presenting and explaining technical solutions to both technical and non-technical stakeholders