Databricks Lead - Data Engineering

Description :

Job Title : Databricks Lead.

Location : Chennai/Hyderabad/Bangalore.

Experience : 8-10 years of overall experience and 5+ years of hands-on leadership in data engineering projects.

Budget : Open to discuss.

Notice Period : Immediate joiner/ Serving notice with less than 60 days/Notice is less than 60 days.

About the Role :

We are looking for a highly skilled Databricks Lead with 810 years of overall experience and 5+ years of hands-on leadership in data engineering projects.

The ideal candidate will lead teams in designing, building, and optimizing scalable data pipelines and Lakehouse solutions on Databricks.

Key Responsibilities :

- Lead the development and delivery of end-to-end data pipelines using Databricks, Spark, Delta Lake, and PySpark.

- Drive implementation of Lakehouse/Medallion architectures (Bronze/Silver/Gold).

- Guide and mentor data engineers, ensure best coding and architectural practices.

- Optimize Spark jobs, Databricks clusters, and workflows for performance and cost.

- Implement CI/CD pipelines, version control, and DevOps practices for Databricks (Repos, Workflows, Terraform).

- Collaborate with architects, data scientists, and business teams to deliver scalable data solutions.

- Ensure data quality, governance, security, and compliance standards are followed (Unity Catalog preferred).

Required Skills & Experience :

- 810 years in Data Engineering or Cloud Data Platforms.

- 5+ years leading data engineering teams or technical delivery.

- Strong hands-on experience with Databricks, Spark, PySpark, SQL, and Delta Lake.

- Experience with cloud platforms (AWS/Azure/GCP) and related data services.

- Solid understanding of data modeling, ingestion frameworks, orchestration, and performance tuning.

- Experience with CI/CD, Git, Terraform, Databricks Repos, and Workflows.

- Strong problem-solving, leadership, and communication skills.

Preferred Qualifications :

- Databricks certifications (Associate or Professional).

- Experience with streaming technologies (Kafka, Kinesis, Event Hubs).

- Familiarity with MLflow and data science integration.

What We Offer :

- Opportunity to lead cutting-edge data engineering initiatives.

- Collaborative, high-growth environment.

- Competitive compensation and professional development opportunities.