Senior Data Engineer - ETL

Key Responsibilities :

- Design, develop, and maintain high-performance ETL and real-time data pipelines using Apache Kafka and Apache Flink.

- Build scalable and automated MLOps pipelines for model training, validation, and deployment using AWS SageMaker and related services.

- Implement and manage Infrastructure as Code (IaC) using Terraform for AWS provisioning and maintenance.

- Collaborate with ML, Data Science, and DevOps teams to ensure reliable and efficient model deployment workflows.

- Optimize data storage and retrieval strategies for both structured and unstructured large-scale datasets.

- Integrate and transform data from multiple sources into data lakes and data warehouses.

- Monitor, troubleshoot, and improve performance of cloud-native data systems in a fast-paced production setup.

- Ensure compliance with data governance, privacy, and security standards across all data operations.

- Document data engineering workflows and architectural decisions for transparency and maintainability.

Requirements :

- 5+ Years of experience as Data Engineer or in similar role.

- Proven experience in building data pipelines and streaming applications using Apache Kafka and Apache Flink.

- Strong ETL development skills, with deep understanding of data modeling and data architecture in large-scale environments.

- Hands-on experience with AWS services, including SageMaker, S3, Glue, Lambda, and CloudFormation or Terraform.

- Proficiency in Python and SQL; knowledge of Java is a plus, especially for streaming use cases.

- Strong grasp of MLOps best practices, including model versioning, monitoring, and CI/CD for ML pipelines.

- Deep knowledge of IaC tools, particularly Terraform, for automating cloud infrastructure.

- Excellent analytical and problem-solving abilities, especially with regard to data processing and deployment issues.

- Agile mindset with experience working in fast-paced, iterative development environments.

- Strong communication and team collaboration skills.