Position Overview :

The MLOps Engineer will play a critical role in supporting the migration of Data Science models from GCP to Azure Databricks, building and optimizing MLOps workflows, and eventually contributing to broader data movement automation and self-serve capabilities across cloud environments.

The client is currently operating data workflows in GCP and is migrating only the Data Science workloads to Azure Databricks, while input data originates in GCP and model outputs are written back to GCP. This role requires someone who deeply understands Databricks internals, PySpark, CI/CD orchestration, and ML model operationalization, along with knowledge of GCP and Azure.

Key Responsibilities :

- Support migration of existing ML models from GCP to Azure Databricks

- Understand existing model architecture and replicate/optimize it in Azure Databricks

- Work closely with the Data Science team to operationalize migrated models

- Optimize models to reduce compute cost and increase test coverage

- Set up robust CI/CD pipelines using GitHub Actions for ML model deployments in Databricks

- Implement and manage MLflow for tracking, versioning, and managing model lifecycle

- Build efficient and scalable Data & ML pipelines using Databricks + PySpark

- Collaborate with the Data Engineering team regarding data movement between GCP - Azure Databricks

- Take over parts of cross-cloud data movement from the DE team and build self-serve automation for data flows

- Build pipelines where outputs from Azure Databricks must be transferred back to GCP

- Provide architectural inputs and workflow optimization guidance during and after migration

- Ensure scalable, cost-efficient, and reliable model execution in Databricks

- Improve testing, monitoring, and performance tuning for migrated and future ML models

Required Skills :

- 6- 8 years of experience in Data Engineering, ML Engineering, or MLOps roles

- Strong hands-on expertise in Databricks and deep understanding of how it works under the hood

- Proficiency in PySpark : writing scalable jobs, understanding execution plans, and optimization techniques

- Experience building CI/CD pipelines using GitHub Actions

- Experience with MLflow for tracking and operationalizing ML models

- Knowledge of integrating workflows between GCP and Azure ecosystems

- Strong debugging, optimization, and cost-efficiency mindset

Preferred (Bonus) Skills :

- Experience with cross-cloud data movement patterns

- Familiarity with DS model structures and ability to collaborate closely with DS teams

- Exposure to model monitoring and alerts in a distributed/cloud environment