The ideal candidate will play a critical role in designing, implementing, and maintaining cloud infrastructure and CI/CD pipelines to support scalable, secure, and efficient data and analytics solutions. This role requires a strong understanding of cloud-native technologies, DevOps best practices, and hands-on experience with Azure and Databricks.

Key Responsibilities :

Cloud Infrastructure Design & Management :

- Architect, deploy, and manage scalable and secure cloud infrastructure on Microsoft Azure.

- Implement best practices for Azure Resource Management, including resource groups, virtual networks, and

storage accounts.

- Optimize cloud costs and ensure high availability and disaster recovery for critical systems

Databricks Platform Management :

- Set up, configure, and maintain Databricks workspaces for data engineering, machine learning, and analytics workloads.

- Automate cluster management, job scheduling, and monitoring within Databricks.

- Collaborate with data teams to optimize Databricks performance and ensure seamless integration with Azure services.

CI/CD Pipeline Development :

- Design and implement CI/CD pipelines for deploying infrastructure, applications, and data workflows using tools like Azure DevOps, GitHub Actions, or similar.

- Automate testing, deployment, and monitoring processes to ensure rapid and reliable delivery of updates.

Monitoring & Incident Management :

- Implement monitoring and alerting solutions using tools like Dynatrace, Azure Monitor, Log Analytics, and Databricks metrics.

- Troubleshoot and resolve infrastructure and application issues, ensuring minimal downtime.

Security & Compliance :

- Enforce security best practices, including identity and access management (IAM), encryption, and network

security.

- Ensure compliance with organizational and regulatory standards for data protection and cloud operations.

Collaboration & Documentation :

- Work closely with cross-functional teams, including data engineers, software developers, and business stakeholders, to align infrastructure with business needs.

- Maintain comprehensive documentation for infrastructure, processes, and configurations.

Required Qualifications :

Education : Bachelors degree in Computer Science, Engineering, or a related field.

Must Have Experience :

- 6+ years of experience in DevOps or Cloud Engineering roles.

- Proven expertise in Microsoft Azure services, including Azure Data Lake, Azure Databricks, Azure Data Factory (ADF), Azure Functions, Azure Kubernetes Service (AKS), and Azure Active Directory.

- Hands-on experience with Databricks for data engineering and analytics.

Technical Skills :

- Proficiency in Infrastructure as Code (IaC) tools like Terraform, ARM templates, or Bicep.

- Strong scripting skills in Python, or Bash.

- Experience with containerization and orchestration tools like Docker and Kubernetes.

- Familiarity with version control systems (e.g., Git) and CI/CD tools (e.g., Azure DevOps, GitHub Actions).

Soft Skills :

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration abilities.