Iris Software - Data Engineer Lead

Description :

Key Responsibilities :

- Design, build, and maintain scalable data pipelines and enterprise data platforms

- Develop robust ETL/ELT workflows and automate data engineering processes

- Work on distributed data processing using Spark and large-scale data frameworks

- Optimize and manage cloud-based data engineering solutions across AWS, Azure, or GCP

- Implement orchestration workflows using Airflow or similar tools

- Ensure data quality, governance, security, and compliance standards

- Collaborate with analytics, AI/ML, and business teams for data integration and delivery

- Support real-time and batch data processing requirements

- Troubleshoot data pipeline failures and optimize performance

- Contribute to data architecture modernization and engineering best practices

Required Skills & Experience :

- 8+ years of experience in Data Engineering

- Strong proficiency in Python and Advanced SQL

- Hands-on experience with Spark and distributed processing frameworks

- Experience working with cloud platforms such as AWS, Azure, or GCP

- Expertise in Airflow or other orchestration tools

- Strong understanding of data pipeline automation

- Experience with data governance and data quality practices

- Excellent analytical, troubleshooting, and problem-solving skills

Good to Have :

- Experience with Kafka or Kinesis streaming platforms

- Knowledge of Feature Stores and ML data infrastructure

- Exposure to LLM-ready data pipelines

- Understanding of MLOps concepts and workflows

Preferred Candidate Profile :

- Strong understanding of modern data architecture and scalable systems

- Ability to work in fast-paced and collaborative environments

- Experience supporting AI/ML and analytics use cases

- Strong communication and stakeholder management skills