Cloud Data Platform Engineer

Job Title : Cloud Data Platform Engineer / SRE (Azure | Databricks)

Experience : 6-12 Years

Location : Hybrid / PAN India (as applicable)

Role Type : Individual Contributor - Engineering Focus

Role Summary

We are seeking an experienced Cloud Data Platform Engineer / SRE with strong hands-on expertise in Azure, Databricks, and Microsoft Fabric. This is a pure engineering role focused on designing, building, automating, securing, and operating enterprise-grade cloud data platforms.

The ideal candidate will bring deep platform engineering experience, strong DevOps & Infrastructure-as-Code practices, and a reliability-first mindset aligned with SRE principles. You will work closely with Data Engineering, Analytics, Security, and Infrastructure teams to deliver scalable, secure, and cost-efficient data platforms.

Key Responsibilities :

1. Azure Data Platform Engineering :

- Design, deploy, and manage Azure data services including :

1. ADLS Gen2

2. Azure Data Factory (ADF)

3. Azure Synapse Analytics

4. Azure Key Vault

5. Virtual Networks (VNets), Private Endpoints

- Implement secure network architectures and private connectivity models.

- Manage storage accounts, lifecycle policies, and access controls.

2. Databricks Platform Administration :

- Provision and manage Databricks workspaces, clusters, and job orchestration.

- Implement and maintain :

1. Delta Lake architecture

2. Unity Catalog governance

3. Cluster policies and auto-scaling strategies

- Optimize Spark workloads for performance and cost efficiency.

- Support data engineering teams with best practices and platform standards.

3. Microsoft Fabric Enablement :

Deploy and manage Microsoft Fabric components :

1. Lakehouse

2. Data Pipelines

3. Warehouses

- Integrate Fabric with Azure data ecosystem.

- Establish governance and access frameworks within Fabric.

4. Infrastructure as Code & DevOps :

- Build reusable Infrastructure-as-Code (IaC) modules using :

1. Terraform

2. Bicep

- Design and implement CI/CD pipelines using :

1. Azure DevOps

2. GitHub Actions

- Automate environment provisioning and release management.

- Enforce GitOps and branching strategies.

5. Reliability Engineering (SRE Practices) :

- Implement monitoring and observability frameworks.

- Configure alerts, dashboards, and logging integrations.

- Drive incident management, root cause analysis (RCA), and postmortems.

- Establish SLOs, SLAs, and error budgets for platform reliability.

- Improve system resilience, availability, and fault tolerance.

6. FinOps & Cost Optimization :

- Monitor and analyze Azure and Databricks consumption.

- Implement cost governance strategies.

- Optimize compute clusters and storage utilization.

- Drive capacity planning and forecasting.

- Partner with finance and business teams for budget alignment.

7. Security & Governance :

- Implement RBAC, ACLs, and Unity Catalog permissions.

- Support zero-trust security models.

- Ensure compliance with enterprise security standards.

- Enable encryption, key management, and access auditing.

- Work with security teams for vulnerability assessments and remediation.

Required Skills & Qualifications :

- 6-12 years of experience in Cloud Data Engineering / Platform Engineering / SRE roles

Strong expertise in :

1. Azure Data Services

2. Azure networking & security

3. Databricks platform administration

4. Microsoft Fabric

- Hands-on experience with Terraform or Bicep

- Experience designing and maintaining CI/CD pipelines

Proficiency in :

1. Python

2. SQL

3. PySpark

4. PowerShell / Bash scripting

- Strong understanding of distributed systems and cloud architecture

Proven experience in :

1. Cost optimization (FinOps)

2. Platform reliability (SRE practices)

3. Monitoring & observability frameworks

Nice to Have :

- Azure / Databricks / DevOps / FinOps certifications

- Experience in regulated industries (BFSI, Healthcare, Retail)

- Exposure to Zero Trust Architecture

- Experience with containerization (Docker, Kubernetes)

- Knowledge of DataOps practices

Behavioral Competencies :

- Strong ownership mindset

- Problem-solving and analytical thinking

- Ability to work in high-availability production environments

- Strong communication and cross-team collaboration skills

- Continuous improvement mindset

What Success Looks Like in This Role :

- Stable, secure, and highly available data platform