Celebal Technologies is a leading Solution Service company that provide Services the field of Data Science, Big Data, Enterprise Cloud & Automation. We are at the forefront of leveraging cutting-edge technologies to drive innovation and enhance our business processes. As part of our commitment to staying ahead in the industry, we are seeking a talented and experienced Data & AI Engineer with strong Azure cloud competencies to join our dynamic team.

Job Summary :

We are looking for a highly skilled Senior DevOps Engineer with deep expertise in Azure cloud and strong hands-on experience with Databricks. The role involves designing, implementing, and optimizing DevOps processes, ensuring seamless CI/CD, automation, and infrastructure scalability to support data and AI-driven projects.

Key Responsibilities :

- Design, implement, and maintain CI/CD pipelines for applications and data platforms.

- Deploy, manage, and optimize Azure infrastructure using IaC (Terraform/ARM/Bicep).

- Automate provisioning, monitoring, and scaling of Databricks clusters and jobs.

- Collaborate with data engineering and ML teams to streamline MLOps and DataOps workflows.

- Manage Azure DevOps/GitHub Actions/Jenkins pipelines for code deployments.

- Implement logging, monitoring, and alerting solutions (App Insights, Log Analytics, Prometheus, Grafana, etc.).

- Drive cost optimization and performance tuning across cloud and Databricks environments.

- Ensure best practices in security, compliance, and governance within Azure and Databricks.

- Mentor junior engineers and actively contribute to DevOps strategy and roadmap.

Key Skills :

- 8+ years of experience in DevOps, Cloud, and Infrastructure automation.

- Strong hands-on expertise in Microsoft Azure (AKS, ADF, Storage, Key Vault, Networking, Security, etc.).

- Proven experience in Databricks administration and automation (cluster management, job orchestration, integration with Azure services).

- Proficiency with Infrastructure as Code (Terraform, ARM, Bicep).

- Strong scripting skills (Python, PowerShell, Bash).

- Solid experience with CI/CD tools (Azure DevOps, GitHub Actions, Jenkins).

- Expertise in Docker, Kubernetes (AKS preferred) for containerized workloads.

- Familiarity with monitoring & logging tools (Azure Monitor, ELK, Prometheus, Grafana).

- Strong understanding of networking, security, and compliance practices in Azure.