Job Description :

As a Senior Data Engineer, you will play a key role in designing, building, and maintaining the data pipelines, models, and reporting structures that underpin risk-management platform. You will develop cloud-native data solutions that transform operational data into clean, scalable, and analytics-ready datasets that support reporting, dashboards, and cross-team consumption.

You bring strong expertise in Python-based data engineering, ELT design, data modeling, and building robust data flows in AWS. You enjoy solving complex data integration problems, designing efficient data structures, and working closely with product, engineering, and analytics teams to deliver high-quality datasets and reporting solutions.

While your primary focus will be on data engineering, you are comfortable contributing to .NET (C#) microservices when needed-particularly services that expose curated data through well-designed APIs. You operate independently on complex tasks while collaborating effectively with senior and staff-level engineers on architectural decisions.

Responsibilities :

- Design, build, and maintain Python-based data pipelines to process, transform, and enrich operational and analytical data.

- Implement ELT workflows, including ingestion, cleaning, transformation, aggregation, and historical modeling.

- Develop and maintain AWS-native data environments using S3, Redshift Spectrum, and related AWS services.

- Build Amazon QuickSight dashboards and reports, delivering actionable insights to customers and internal users.

- Integrate data from multiple, heterogeneous sources into unified, well-structured datasets.

- Implement and optimize schema designs, partitioning strategies, file formats, and metadata practices (including Apache Avro and Parquet).

- Develop or support the development of .NET (C#) microservices that expose curated data via RESTful APIs.

- Work with product managers, analysts, and engineers to understand data requirements, ensuring solutions meet performance and reliability needs.

- Enhance data quality, governance, and observability through validation, error handling, and lineage best practices.

- Troubleshoot and resolve data issues, improving performance and reliability across data pipelines and storage layers.

- Participate in code reviews, design discussions, and planning sessions to ensure engineering excellence.

- Contribute to continuous improvement of data processes, tooling, and development patterns.

Qualifications :

- Bachelor's degree in Computer Science, Engineering, Data Science, or equivalent experience.

- 10+ years of experience in data engineering or software engineering with strong exposure to cloud-native data pipelines.

- Strong experience building Python-based data workflows and ETL/ELT pipelines.

- Hands-on experience with AWS data services, especially:

1. S3 data lakes/marts

2. Redshift / Redshift Spectrum

- Solid understanding of Apache Avro, Parquet, ORC, or other structured/columnar formats.

- Strong proficiency in SQL, analytical query design, and performance tuning.

- Experience developing QuickSight dashboards or similar BI reporting solutions.

- Experience integrating multiple data sources into unified, query-efficient structures.

- Ability to work with .NET/C# for light API or microservice development.

- Strong understanding of data quality, pipeline reliability, and monitoring practices.

- Excellent problem-solving, debugging, and communication skills.

- Ability to work independently while collaborating effectively within a cross-functional engineering team.

Preferred Qualifications :

- Experience working with medium- to high-volume data systems.

- Familiarity with event-driven architectures (SNS/SQS, Kafka, Kinesis).

- Experience with CI/CD workflows for data pipelines or microservices.

- Exposure to infrastructure-as-code (Terraform, CloudFormation).

- Experience building internal automation tools, utilities, or frameworks for data engineering.

- Domain experience in risk management, analytics, or financial platforms.

- Experience optimizing cloud storage and compute costs in AWS.