Role Overview:

We are looking for an experienced Databricks Lead / Data Architect to design and implement enterprise-scale Lakehouse solutions on Azure. The ideal candidate will have strong expertise in Databricks, Apache Spark, Azure Data Platform, and Data Architecture, with the ability to lead end-to-end data engineering initiatives.

Key Responsibilities:

- Architect and implement scalable Lakehouse solutions using Databricks.

- Design and develop batch and real-time data pipelines using Apache Spark (PySpark/SQL).

- Build ETL/ELT pipelines, incremental data loads, and metadata-driven ingestion frameworks.

- Implement and optimize Delta Lake, Delta Live Tables, Autoloader, Structured Streaming, and Databricks Workflows.

- Design enterprise data warehouse solutions using 3NF and Dimensional Modeling.

- Establish data governance, security, and data quality using Unity Catalog.

- Manage ML lifecycle using MLflow and support AI use cases such as RAG and AI/BI.

- Deploy and manage cloud-native solutions on Microsoft Azure.

- Drive CI/CD, Azure DevOps, and Spark performance optimization.

- Lead technical teams, mentor engineers, and collaborate with stakeholders to deliver scalable data solutions.

Required Skills:

- 10+ years of overall experience with 5+ years in Databricks.

- Strong expertise in Databricks, Apache Spark (PySpark/SQL), Delta Lake, Delta Live Tables, and Azure Data Factory.

- Experience designing end-to-end data architectures and enterprise Lakehouse solutions.

- Hands-on experience with Microsoft Azure and Azure DevOps.

- Strong knowledge of Data Warehousing, ETL/ELT, Data Modeling (3NF & Dimensional).

- Experience with Unity Catalog, MLflow, and data governance frameworks.

- Prior experience working with US/UK or global clients.

- Proven experience leading teams and driving technical delivery.

Good to Have:

- Exposure to AI/ML services, RAG, and Generative AI.

- Experience with CI/CD pipelines and cloud-native data platforms.

Perks & Benefits:

- Free AWS & Azure Certification Training.

- Group Medical Insurance up to ?5 Lakhs.

- Meal Allowance.

- Opportunity to work on large-scale enterprise data and AI transformation projects.