As an AVP, you will lead the design, implementation, and governance of data pipelines for our feature stores. Feature stores are pivotal in enabling the building and deployment of machine learning models across our on-premise Cloudera big data lake and cloud environments (CML, CDP). Your role will be critical in ensuring data reliability, scalability, and performance in support of our data science initiatives.

Responsibilities :

- Design and implement scalable data pipelines for feature extraction, transformation, and loading (ETL) using technologies such as Pyspark, Scala, and relevant big data frameworks.

- Govern and optimize data pipelines to ensure high reliability, efficiency, and data quality across on-premise and cloud environments.

- Collaborate closely with data scientists, ML engineers, and business stakeholders to understand data requirements and translate them into technical solutions.

- Implement best practices for data governance, metadata management, and compliance with regulatory requirements.

- Lead a team of data engineers, providing technical guidance, mentorship, and fostering a culture of innovation and collaboration.

- Stay updated with industry trends and advancements in big data technologies and contribute to the continuous improvement of our data engineering practices.

Qualifications :

- Bachelors degree in Computer Science, Engineering, or a related field. Advanced degree preferred.

- 8+ years of experience in data engineering with hands-on experience in designing and implementing data pipelines.

- Strong proficiency in programming languages such as Pyspark and Scala, with experience in big data technologies (Cloudera, Hadoop ecosystem).

- Proven leadership experience in managing and mentoring a team of data engineers.

- Experience working in a banking or financial services environment is a plus.

- Excellent communication skills with the ability to collaborate effectively across teams and stakeholders.