Responsibilities :

Enterprise Data Office is looking for Senior Data Engineers with a strong track record in building high-performing data and analytical solutions that transform, integrate, and meet diverse business data needs. You will play a key role in designing and developing modern data pipelines and platforms, supporting both traditional analytics and emerging AI/ML and Generative AI use cases.

You will be responsible for building data products, contributing across architecture, design, build, test, and deployment phases, and serving as a core member of an Agile team driving user story analysis and delivery.

The Senior Data Engineer will be responsible for :

- Contributing to the design and implementation of data integration architecture and scalable data solutions on cloud-based platforms.

- Conducting source system analysis, data profiling, and data quality assessments to ensure accuracy and reliability of enterprise data.

- Building robust and scalable data pipelines (batch and real-time) to collect, process, and compute metrics from multiple financial data sources.

- Building scalable AI platforms to enable downstream use cases such as machine learning, predictive analytics, and Generative AI applications.

- Partnering with Data Science and AI/ML teams to support the development and deployment of Agentic AI systemsautonomous agents capable of decision-making, orchestration, and workflow execution.

- Designing and implementing Agents built using, and not limited to, Large Language Model integration, vector databases, embeddings, and retrieval-augmented generation (RAG) architectures.

- Assisting in the implementation of retrieval-augmented pipelines (RAG) by structuring and preparing enterprise data for semantic search and contextual retrieval.

- Contributing to data mapping and transformation design, ensuring alignment with models defined by Data Architects.

- Designing application architecture and articulating technical solutions for data pipelines to team members and stakeholders.

- Executing unit testing and validation of data outputs, ensuring high quality and adherence to data standards.

- Collaborating with business users during User Acceptance Testing (UAT) and supporting deployment into production environments.

- Ensuring adherence to change management, compliance, and regulatory requirements, particularly within financial services environments.

- Performing performance tuning, optimization, and code reviews to improve efficiency and reliability of data pipelines.

Required Qualifications :

- 8+ years of experience in data engineering building scalable data solutions in cloud or big data environments

- Hands-on experience with big data platforms (Hadoop ecosystem : HDFS, Hive, HBase, Pig, etc.)

- Experience in building microservice applications (FastAPI, Django)

- Proficiency in Python

- Working knowledge of OTEL and API Gateways

- Strong experience in ETL/ELT development, using tools such as :

1. DataStage, Informatica, Sqoop or modern cloud-native tools

- Strong programming expertise in :

1. SQL, PL/SQL

2. Python, Java, or Scala

3. Spark / distributed processing frameworks

- Experience with :

1. RDBMS (Teradata, Oracle, SQL Server, DB2, Redshift, Snowflake, etc.)

2. AWS cloud services (S3, Glue, Athena, RDS, etc.)

3. Scheduling tools (Autosys, Tidal, Oozie)

4. Source control tools (Bitbucket, Git, SVN, Jira)

- Experience working across Unix/Linux, Windows, or mainframe environments

- Strong understanding of data pipeline performance tuning, optimization, and reliability engineering

- Strong communication and collaboration skills with attention to detail.

Desired Qualifications :

- Experience supporting data engineering for Data Science and Machine Learning projects

- Hands-on or working knowledge of Generative AI concepts, including :

1. Preparing datasets for LLM-based applications

2. Prompt engineering fundamentals

3. Supporting retrieval-based AI systems

- Exposure to Agentic AI workflows, including :

1. Supporting orchestration of automated data-driven workflows

2. Enabling data pipelines for autonomous or semi-autonomous decision systems

- Familiarity with :

1. Vector databases (e.g., OpenSearch, Pinecone, FAISS exposure level)

2. Embeddings and semantic search concepts

- Experience handling :

1. Structured, semi-structured, and unstructured data

- Experience in financial services / banking domain, including regulatory and risk-sensitive data

- Experience with BI / reporting tools such as Tableau, Cognos, or SAS

- Contributions to open-source data tools or frameworks (preferred).

Education and Certifications :

- Required : Bachelors Degree in Analytics, Mathematics, Statistics, Computer Science, or related field

- Preferred :

1. Big Data / Data Engineering certifications

2. AWS / Cloud certifications

3. AI / ML or Generative AI certifications