Azure Databricks Engineer - Synapse Analytics

Job Role : Azure Databricks Engineer

Location : Gurugram, India

Employment Type : Contract-to-Hire

Role Overview :

We are looking for an experienced Azure Databricks Engineer to design, develop, and optimize scalable data pipelines and analytics solutions. The role requires strong expertise in Azure Databricks, Spark, ETL, and data integration workflows, with hands-on experience in performance tuning and cloud-native architectures.

Key Responsibilities :

- Design, develop, and maintain ETL/ELT pipelines and data workflows in Azure Databricks.

- Build and optimize Spark-based solutions for processing large-scale structured and unstructured data.

- Integrate Databricks with other Azure services (Data Lake, Synapse, Data Factory, Key Vault, Event Hub, etc.).

- Implement data ingestion frameworks using APIs, connectors, and streaming sources.

- Collaborate with data scientists, analysts, and application teams to deliver end-to-end data solutions.

- Ensure best practices for security, governance, version control, and CI/CD pipelines in Databricks.

- Conduct performance tuning, debugging, and job optimization for efficient resource utilization.

- Monitor, troubleshoot, and maintain Databricks clusters and jobs in production environments.

Required Technical Skills :

Mandatory : Strong proficiency in Microsoft Azure Databricks and Apache Spark.

- Hands-on experience with Azure Data Lake Storage (ADLS), Azure Synapse, and Azure Data Factory.

- Proficiency in PySpark/Scala/Spark SQL for data engineering.

- Exposure to cloud computing platforms (AWS, GCP) in addition to Azure.

- Knowledge of Delta Lake, DataBricks SQL, and MLflow for advanced analytics and machine learning workflows.

- Experience with data security, governance, and compliance frameworks.

- Understanding of real-time data streaming (Kafka/Event Hub).