MLOps/AWS Data Engineer

Key Responsibilities :

- Design, build, and maintain scalable MLOps pipelines using AWS tools such as SageMaker, Lambda, Step Functions, and CodePipeline.

- Automate ML workflows including data preprocessing, model training, validation, deployment, and monitoring.

- Integrate CI/CD pipelines for ML models using AWS CodeBuild, CodeDeploy, or GitHub Actions.

- Containerize ML applications using Docker and deploy using ECS/EKS.

- Implement model performance tracking, drift detection, and automatic retraining triggers.

- Develop and manage ETL/ELT pipelines using AWS Glue, AWS Lambda, and Apache Spark (PySpark).

- Build robust and scalable data ingestion workflows from structured/unstructured sources (RDS, S3, APIs, etc.).

- Manage and optimize data lakes and data warehouses using Amazon Redshift, Athena, and Lake Formation.

- Implement data validation, quality checks, and lineage tracking.

- Use Terraform or CloudFormation to automate infrastructure provisioning.

- Implement logging, monitoring, and alerting for ML systems using CloudWatch, Prometheus, or ELK Stack.

- Ensure cloud cost optimization and security best practices across environments.

- Collaborate with Data Scientists, ML Engineers, and DevOps teams to understand requirements and implement efficient

solutions.

- Maintain comprehensive documentation of pipelines, systems, and processes.

- Participate in Agile ceremonies, sprint planning, and technical reviews.

Required Skills & Qualifications :

- 4 - 6 years of hands-on experience in data engineering, MLOps, or cloud-native ML/AI systems.

- Proficiency in Python with experience in writing production-grade code.

- Strong experience with AWS services : SageMaker, Glue, Lambda, ECS/EKS, CloudFormation/Terraform, CloudWatch, Step Functions, S3, Redshift, Athena

- Experience with CI/CD tools : Git, GitHub/GitLab, Jenkins, AWS CodePipeline.

- Hands-on with Docker and container orchestration.

- Experience working with Apache Spark / PySpark for large-scale data processing.

- Solid understanding of machine learning lifecycle (training, validation, deployment, monitoring).

- Strong SQL skills and experience working with large datasets.

Preferred Qualifications :

- Experience with Kubeflow, MLflow, or similar MLOps frameworks.

- Familiarity with Kafka, Airflow, or Apache NiFi for orchestration.

- AWS Certifications (e.g., AWS Certified Machine Learning Specialty, AWS Data Analytics, or Solutions Architect).

- Exposure to data governance, data privacy, and compliance frameworks.

- Prior experience in Agile/Scrum environment.