About the Role :

MantriAI is hiring for one of the largest players in the Banking & Financial Services industry. We are looking for an experienced Data Engineering professional to lead the design, architecture and delivery of scalable data pipelines and curated datasets that power advanced analytics, machine learning and technology innovation in the banking domain. The ideal candidate should have strong hands-on expertise in Databricks and PySpark, along with experience managing engineering teams, driving data architecture and delivering enterprise-grade data solutions.

Key Responsibilities :

- Lead the design and development of scalable data ingestion and transformation pipelines using Databricks, PySpark, Python and SQL.

- Architect and optimize large-scale Spark workloads for performance, reliability and cost efficiency.

- Build reusable transformation components and scalable analytical data models.

- Lead, mentor and manage a team of data engineers, ensuring high engineering standards and effective code reviews.

- Drive CI/CD deployments, production troubleshooting and pipeline maintenance.

- Translate complex banking business requirements into scalable technical data solutions.

- Implement data quality checks, reconciliation, exception handling and automated monitoring.

- Ensure data security, access controls, lineage and documentation standards.

- Drive end-to-end data integration strategies for analytics, reporting and machine learning use cases.

Required Skills & Experience :

- Strong hands-on experience in Databricks and PySpark.

- Advanced proficiency in Python and SQL.

- Experience designing complex ETL/ELT pipelines and working with Data Lakes/Delta Lakes.

- Strong understanding of distributed processing, workflow orchestration and Spark performance tuning.

- Banking domain experience, including banking data systems, structures and financial metrics.

- Experience implementing enterprise-grade data quality frameworks covering accuracy, completeness and consistency.

- Proven experience in technical leadership and managing engineering teams.

Good to Have :

- Experience with Azure-based data platforms.

- Knowledge of data governance and sensitive data handling (PII, GDPR).

- Experience preparing data for Power BI and machine learning applications.