Databricks Platform Engineer

Experience : 8 + Years

Work Timings : 5 : 00 PM - 2 : 30 AM IST

Notice Period : Immediate to 15 Days

Preferred Cloud Platform : AWS

Job Description :

We are looking for an experienced Databricks Platform Engineer to manage and optimize enterprise Databricks environments. The ideal candidate will have strong expertise in Databricks platform administration, workspace configuration, Unity Catalog, access management, environment governance, and platform monitoring. The role focuses on maintaining a secure, reliable, and cost-efficient Databricks ecosystem for data engineering and analytics teams.

Key Responsibilities :

- Platform Administration : Configure and administer Databricks workspaces, compute resources, cluster policies, runtime configurations, and workspace-level settings.

- Unity Catalog and Governance : Manage catalogs, schemas, permissions, data access policies, and metadata governance to ensure secure and controlled access to enterprise data.

- Databricks Workflows : Configure and manage job scheduling, task dependencies, workflow execution, retries, and failure handling.

- Delta Live Tables (DLT) : Support the configuration and administration of DLT pipelines, monitoring, pipeline reliability, and data quality expectations.

- Environment Management : Maintain development, staging, and production environments with consistent configurations, controlled access, and standardized release procedures.

- Infrastructure Automation : Use Terraform or equivalent Infrastructure as Code tools to automate workspace provisioning, configuration management, and platform changes.

- Version Control and Change Management : Use GitHub to manage platform configurations, track changes, maintain code repositories, and support controlled deployments.

- Monitoring and Incident Management : Monitor platform availability, job failures, compute utilization, and resource consumption. Investigate incidents and implement preventive measures.

- Security and Access Management : Configure identity integration, role-based access controls, service principals, and secure connectivity in coordination with cloud and security teams.

- Cost and Resource Optimization : Establish cluster policies, manage compute utilization, review usage patterns, and optimize platform costs without compromising reliability.

- Technical Documentation : Maintain platform configuration documentation, operational procedures, governance standards, and troubleshooting guidelines.

Required Skills :

- 8 + years of experience in platform engineering, cloud infrastructure, data platforms, or related technical roles.

- Strong hands-on experience administering and managing the Databricks platform.

- In-depth knowledge of Databricks workspaces, compute configurations, cluster policies, and runtime management.

- Practical experience with Unity Catalog, Databricks Workflows, and Delta Live Tables (DLT).

- Experience managing platform permissions, governance, and enterprise access controls.

- Knowledge of Terraform or equivalent Infrastructure as Code tools.

- Experience with GitHub, version control, and platform change management.

- Strong understanding of monitoring, troubleshooting, performance management, and cost optimization.

- Familiarity with AWS infrastructure, networking, IAM, and storage integrations is preferred.