Azure DevOps Engineer

Job Title : Azure DevOps Engineer

Location : Remote (with 1 visit Bangalore office)

Experience : 4+ Years

Role Overview :

We are seeking a skilled Azure DevOps Engineer with hands-on experience in Azure Data Factory (ADF), Azure Databricks (ADB), and CI/CD pipeline management. The ideal candidate will work on deploying, automating, and optimizing data workflows and pipelines in a cloud environment, ensuring scalability, efficiency, and reliability.

Mandatory Skills :

- Azure Databricks (ADB) Data processing, orchestration, and optimization.

- Azure Data Factory (ADF) ETL, workflow design, and data movement automation.

- CI/CD Pipelines Azure DevOps, GitHub Actions, or equivalent tools.

- PySpark Writing and optimizing distributed data processing code.

- SQL Query optimization, data validation, and integration.

Key Responsibilities :

- Design, develop, and maintain CI/CD pipelines for Azure-based data platforms.

- Implement and optimize Azure Data Factory and Azure Databricks workflows for data processing and analytics.

- Collaborate with Data Engineering and ML teams to enable continuous integration and deployment of scalable data pipelines.

- Automate data orchestration processes using PySpark and SQL.

- Monitor, troubleshoot, and optimize pipeline performance and data flow.

- Maintain and enhance DevOps best practices, ensuring security and compliance in all deployments.

Good-to-Have Skills :

- Familiarity with Terraform or ARM templates for infrastructure automation.

- Experience with Azure Synapse, Delta Lake, or Data Lake Storage.

- Knowledge of monitoring tools (Azure Monitor, Log Analytics).

- Scripting knowledge in PowerShell or Python for DevOps automation.

Qualifications :

- Bachelors or Masters degree in Computer science, Information Technology, or related field.

- Minimum 4+ years of experience in Azure DevOps with a focus on data engineering workflows.

- Strong problem-solving, debugging, and collaboration skills.