Key Responsibilities:

- Architect and implement enterprise-grade Lakehouse solutions using Databricks.

- Design and deliver scalable batch and real-time data pipelines using Apache Spark (PySpark/SQL).

- Build ETL/ELT pipelines, incremental data loads, and metadata-driven ingestion frameworks.

- Implement and optimize Databricks components: Delta Lake, Delta Live Tables, Autoloader, Structured Streaming, and Workflows.

- Design large-scale data warehousing solutions with 3NF and dimensional modeling.

- Establish data governance, security, and data quality frameworks, including Unity Catalog.

- Lead ML lifecycle management using MLflow and drive AI use cases (RAG, AI/BI).

- Manage cloud-native deployments on Microsoft Azure and integrate with enterprise systems (e.g., ServiceNow).

- Drive CI/CD, DevOps practices, and performance optimization of Spark workloads.

- Provide technical leadership, mentor teams, and ensure successful delivery.

- Collaborate with stakeholders to translate business requirements into scalable solutions.

Ideal Candidate:

- Databricks Lead.

- 10+ years of total experience, with 5+ years specifically in Databricks.

- Architecture-level expertise must understand end-to-end design, not just engineering execution.

- Hands-on with Delta Lake, Delta Live Tables, Spark (PySpark/SQL), and Azure Data Factory.

- Prior experience working on Azure or Azure DevOps environments.

- Has handled US/UK or international client-facing projects.

- Comfortable managing and leading a team.

- Basic understanding of AI/ML tools and services is a plus.

Perks, Benefits & Culture:

- The company provides free AWS and Azure certification training.

- Group medical insurance of ?5 lakhs is included in the benefits package, along with meal allowances.