Posted on: 20/11/2026
Senior Databricks Architect
Work Location: Bangalore - Pune - Noida - Hyderabad - Chennai
Notice period: Immediate and Max 15 to 20 Days
Experience: 10 - 15 years
Role Overview:
Senior platform and solution architect responsible for enterprise-scale Databricks lakehouse architecture, modernization strategies, governance, migration roadmaps, engineering standards, performance and cost guardrails, and AI-ready data foundations. The role converts business and non-functional requirements into implementable platform patterns and guides engineering teams through delivery.
Core Responsibilities:
Lakehouse & Platform Architecture:
- Define target-state lakehouse architecture, workspace and environment strategy, Unity Catalog topology, storage and compute patterns, governed sharing, lineage and domain boundaries.
- Establish implementation standards for Medallion architecture, Lakeflow Declarative Pipelines / DLT, Workflows, Auto Loader, Delta Lake, system tables and Databricks Asset Bundles.
- Create reference architectures, architecture decision records, reusable blueprints and non-functional requirement guardrails.
Spark, Data Engineering & Streaming:
- Set engineering standards for PySpark, Spark SQL, Delta Lake, Structured Streaming, CDC, SCD, incremental processing, data contracts and schema evolution.
- Guide execution-plan analysis, AQE, join strategy, shuffle and skew management, partitioning, file sizing, caching and Photon adoption.
- Drive idempotent, auditable and recoverable pipelines with clear batch-versus-streaming decision rationale.
Migration & Modernization:
- Lead legacy warehouse, Hadoop and ETL modernization assessments and define migration waves, conversion patterns and dependency plans.
- Own validation, reconciliation, parallel-run, cutover, rollback and decommissioning approaches.
- Create reusable migration playbooks and accelerators while controlling delivery risk and business continuity.
Governance, Security & Data Sharing:
- Design Unity Catalog operating patterns covering catalogs, schemas, external locations, service principals, RBAC/ABAC, masking, row/column controls and auditability.
- Define lineage, data quality, privacy, retention and compliance-ready implementation standards.
- Guide Delta Sharing, Lakehouse Federation and governed producer-consumer patterns where appropriate.
Platform Engineering, DataOps & Reliability:
- Define Git-based CI/CD, testing, environment promotion, infrastructure-as-code and release controls for Databricks assets.
- Establish SLIs/SLOs, observability, alerting, runbooks, reliability scorecards and production-readiness reviews using platform telemetry and system tables.
- Improve developer experience through templates, reusable frameworks, guardrails and self-service patterns.
Cost Optimization & FinOps:
- Establish cluster and compute policies, autoscaling, workload sizing, serverless decision guidelines, tagging and chargeback/showback controls.
- Create cost and performance baselines and drive workload-level optimization with measurable outcomes.
- Balance performance, reliability and cost rather than optimizing one dimension in isolation.
AI-ready Data Foundations:
- Guide governed data foundations for RAG and AI use cases, including data quality, lineage, access controls and retrieval readiness.
- Demonstrate working awareness of Vector Search, model serving/lifecycle, MLflow, context engineering and AI governance.
- Assess emerging capabilities without weakening core data engineering and platform-accountability expectations.
Consulting & Stakeholder Leadership:
- Lead discovery and architecture workshops; articulate options, trade-offs, risks, roadmap and estimates to technical and business stakeholders.
- Explain the latest project with clear business problem, architecture choice, Databricks fit, alternatives considered, challenges, outcomes and lessons learned.
- Mentor engineers and leads, review designs, and support proposals, solution narratives and customer-facing architecture discussions.
Did you find something suspicious?
Posted by
Posted in
Data Engineering
Functional Area
Data Engineering
Job Code
1685099