We are looking for an experienced Databricks Architect Lead to design, build, and modernize enterprise-scale data platforms using the Databricks Lakehouse Platform. The ideal candidate will have strong expertise in cloud-based data engineering, distributed data processing, and modern data architecture while leading technical teams and driving large-scale data transformation initiatives. The role requires deep hands-on experience with Databricks, Apache Spark, PySpark, Delta Lake, cloud services, and enterprise data architecture.

Key Responsibilities:

- Design and implement enterprise-scale Databricks Lakehouse architectures.

- Lead the migration of legacy data warehouses and ETL platforms to Databricks.

- Build scalable and high-performance batch and real-time data pipelines.

- Develop data ingestion, transformation, and orchestration frameworks using PySpark and Spark SQL.

- Design Bronze, Silver, and Gold layer architectures following Medallion Architecture principles.

- Optimize Spark workloads for performance, scalability, and cost efficiency.

- Implement Delta Lake features including ACID transactions, time travel, schema evolution, and optimization.

- Architect secure and governed data platforms using Unity Catalog.

- Design enterprise data models for analytics, reporting, AI, and Machine Learning workloads.

- Build reusable data engineering frameworks and coding standards.

- Collaborate with business stakeholders to translate business requirements into technical solutions.

- Mentor data engineers, conduct architecture reviews, and establish engineering best practices.

- Lead proof-of-concepts and technology evaluations for modern data platforms.

- Work closely with DevOps teams to implement CI/CD pipelines for Databricks deployments.

- Ensure platform reliability, monitoring, observability, security, and governance across data workloads.

Required Technical Skills:

- Strong experience with Databricks Lakehouse Platform

- Apache Spark & PySpark

- Spark SQL

- Delta Lake

- Python

- SQL

- Data Engineering

- Data Architecture

- ETL & ELT Design

- Medallion Architecture

- Unity Catalog

- Databricks Workflows

- Databricks Repos

- MLflow (preferred)

Qualifications & Experience:

- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

- 10+ years of experience in Data Engineering, Big Data, or Data Architecture.

- 4+ years of hands-on experience with Databricks platform development and architecture.

- Strong expertise in distributed computing using Apache Spark.

- Experience designing enterprise data lakes and Lakehouse architectures.

- Strong understanding of cloud-native architecture patterns.

- Experience implementing secure and governed enterprise data platforms.

- Excellent analytical, troubleshooting, and stakeholder management skills.

- Experience leading technical teams and mentoring engineers.

- Databricks Certified Professional or Associate Certification.

- Experience with Data Mesh or Data Fabric architectures.

- Knowledge of Kafka, Event Hubs, or real-time streaming platforms.

- Experience with AI/ML pipelines and MLOps.

- Terraform or Infrastructure as Code.

- Kubernetes exposure.

- Experience with Snowflake migration or legacy data warehouse modernization.

- Strong architecture and solution design expertise.

- Leadership experience managing enterprise data engineering initiatives.

- Excellent communication and client-facing skills.

- Ability to work with cross-functional global teams.

- Passion for building scalable, cloud-native data platforms and driving innovation.