Senior Data Engineer

Description :

- Implement ETL/ELT jobs for data transformation, cleansing, and loading into the data lake and data warehouse layers.

- Drive the migration of legacy data workflows (e.g., from Inferyx/MySQL) to modern, serverless AWS services.

- Establish automated monitoring, error handling, and alerting for all production pipelines.

2. Data Lake & Data Warehouse Management :

- Manage the structure and governance of the Amazon S3-based Data Lake, implementing best practices for partitioning, security, and cost efficiency.

- Optimize and administer our cloud data warehouse, primarily Amazon Redshift, focusing on performance tuning, resource allocation, and advanced SQL query optimization.

- Design and implement efficient dimensional and relational data models (e.g., Star Schema) to support BI and analytical requirements.

3. Technical Leadership & Collaboration :

- Act as a technical subject matter expert (SME) within the team, performing thorough code reviews and ensuring adherence to cloud engineering best practices.

- Mentor and guide junior data engineers on AWS services, PySpark development, and data modeling techniques.

- Collaborate directly with the Data Architect to implement the defined data strategy and work with Data Analysts to ensure data readiness and accuracy for reporting (Power BI).

4. Data Quality, Security, and Compliance :

- Implement rigorous data quality checks and validation logic directly within the ETL/ELT pipelines.

- Ensure all data solutions adhere to security protocols (IAM roles, S3 encryption) and meet the necessary regulatory compliance standards for the lending domain.

- Partner with DevOps to implement CI/CD practices and utilize Infrastructure as Code (e.g., CloudFormation or Terraform) for platform deployment.

Required Skills & Experience :

- 6 - 8 years of hands-on experience in a Data Engineer or Software Engineer role focused on data platform development.

Deep, proven expertise in the AWS Data Ecosystem :

- ETL/ELT : Expert proficiency with AWS Glue (PySpark) and Python programming for data transformation.

- Serverless Compute : Strong experience with AWS Lambda and Step Functions for orchestration.

- Data Warehousing : Experience with performance tuning, administration, and implementation on Amazon Redshift.

- Data Lake : Expertise with Amazon S3 (storage classes, partitioning, security).

- Advanced proficiency in SQL and deep experience designing data models for analytical

purposes.

- Experience integrating data from operational databases (e.g., MySQL, PostgreSQL, Amazon Aurora) and APIs.

- Hands-on experience with version control (Git) and continuous integration/continuous

deployment (CI/CD) practices.

- Experience or strong working knowledge of data governance and quality tools.

- Experience in the lending or financial services domain is highly desirable.

- Excellent problem-solving, communication, and technical documentation skills