AI/ML Developer - Azure Databricks

Description :

We are looking for an experienced AI/ML Developer with strong expertise in Azure Databricks, Python, PySpark, and SQL to design and implement scalable machine learning and data engineering solutions on the Azure cloud platform.

The candidate will work closely with data scientists, engineers, and business stakeholders to build and deploy AI/ML models and optimized data pipelines.

Key Responsibilities :

- Design and develop end-to-end AI/ML solutions using Azure Databricks

- Build and optimize data pipelines using PySpark and Spark SQL

- Develop, train, and deploy machine learning models in production

- Perform data preprocessing, feature engineering, and model tuning

- Work with large-scale structured and semi-structured datasets

- Optimize Spark jobs for performance and reliability

- Collaborate with teams to productionize ML models

- Maintain best practices in code quality, testing, and version control

Required Skills :

- Strong experience with Azure Databricks

- Proficiency in Python, PySpark, and Spark SQL

- Experience building Machine Learning models

- Hands-on experience with SQL and data transformation

- Experience with Azure Data Lake Storage (ADLS) and Azure Synapse

- Understanding of ETL/ELT processes and big data architecture

- Experience using ML libraries such as Scikit-learn, MLflow, TensorFlow, or PyTorch

- Strong analytical and problem-solving skills

Nice to Have :

- Experience with Kafka, Event Hubs, or streaming data

- Knowledge of Delta Lake and MLOps

- Experience with Docker or containerization