Job Title : DataOps Technical Lead Databricks/Azure

Work Location : Bangalore / Chennai / Gurgaon/ Pune/ Kolkata / Hyderabad

Division/Department : Solution Support

Employment Type : Full-time

Job Description, Essential Duties and Responsibilities :

DataOps Lead is responsible for leading technical team and managing end-to-end project deliverables.

Should have strong experience and knowledge in DataOps support deliverables, KPIs and platform administration for Azure-Databricks DataOps project.

Data Ops & Support roles :

- Pipeline Monitoring & Incident Response

- Ticket/Incident Management

- Data Anomaly Investigations & Root Cause Analysis

- Hotfixes, Enhancements & Release Coordination

- Creating Runbooks and Knowledge Base documents

- Data Quality Tracking

Platform Administration :

- Databricks Platform Administration

- Azure Infrastructure Management

- Identity, Access & Governance

- Data Pipeline Architecture & Platform Standards

Tools & Technologies :

- Databricks

- Azure data services

- SQL Server

- Azure Synapse

- Azure Data Factory (ADF)

- Python and PySpark scripting using DLTs

- Github

- AI based tools (Claude, Co-Pilot etc)

- Linear (task management)

- ServiceNow

- Cosmos

- ADO

Key Responsibilities :

- Data pipelines : Monitor, identify issue, Debugging and Bug fixing of data pipelines. Should understand complex data pipelines and data engineering concepts. Mentoring and guiding junior team members.

- Data Quality & Data Observability : Identify and fix the Data quality issues by performing RCA. Should have a good understanding of data observability tools like Unravel etc.

- Data orchestration : Orchestration of jobs using Airflow.

- DevOps : Should have a good understanding of the Dev Ops process.

- Collaboration : Work closely with data engineering, data science, and IT teams to support data pipeline development, troubleshoot issues, and provide guidance on best practices.

- Incident Management : Lead troubleshooting and resolution of issues related to Databricks clusters and jobs, ensuring minimal downtime and impact on business operations.

- KPI : Responsible for managing MTTA, MTTR, IRR, response & resolution SLA and other key delivery matrix.

- Upgrades & Patches : Manage upgrades, patches, and new releases of Databricks, ensuring minimal disruption to ongoing operations.

- Documentation : Create and maintain detailed documentation of the Databricks environment, configurations, and procedures.

- SOP : Able to build SOPs and set up DataOps best practices.

- Continuous improvement and Innovation : Bringing continuous improvement and innovation for cost optimization, effort reduction, incidents reduction, platform stability and discuss with client, Development team for implementation.

- Team Leading : Lead the technical team and manage day-to-day technical deliverables.

- Escalation : Manage clients day-to-day technical escalation.

- Stakeholder management : Able to handle key stakeholders, including client leadership.

Skills, Education and Experience :

- Having 8+ years of Azure-Databricks DataOps experience.

- Team handling experience : Able to handle a team of 5-7 engineers.

- Open to work in UK & US Shift.

- Working in agile methodology and Sprint framework.

- Must have education degree of B. Tech/B.E/MCA