Were looking for a Senior Data Engineer with deep expertise in RAG (Retrieval-Augmented Generation) and Vector Database design to build and manage the knowledge backbone for AI compliance and insights. This role focuses on modernizing archival data ingestion and enabling real-time contextual retrieval for AI-driven systems.

Key Responsibilities :

- Design and implement RAG Vector Databases (e.g., OpenSearch, Pinecone) using archival data from S3 / Glacier and overall data management via MS SQL Server.

- Modernize existing data ingestion pipelines, replacing legacy OCR-based processes with scalable ETL/ELT frameworks.

- Ensure data synchronization and consistency between RDS (MS SQL Server) and Vector DB for real-time AI context.

- Collaborate with AI, backend, and infrastructure teams to optimize retrieval performance and model access.

- Drive data integrity, schema evolution, and compliance readiness across systems.

Required Skills & Experience :

- Proven expertise in data engineering pipelines (Kafka / MSK, ETL / ELT).

- Hands-on experience with Vector Databases and RAG implementations (OpenSearch, Pinecone, FAISS, Chroma).

- Strong proficiency in SQL, data modeling, and Python / C# / Go.

- Experience with AWS data ecosystem (S3, RDS, Glue, Lambda and related technologies).

- 8 - 10 years of experience in data engineering or AI data platforms.