Job Summary :

We are seeking an experienced and dynamic Engineering Manager Data Engineering to lead cross-functional data engineering teams and drive the design, development, and delivery of enterprise-scale data platforms.

This is a 50-50 blend of technical leadership and delivery management role where you will combine hands-on technical expertise with strategic people leadership.

You will be responsible for leading multiple squads (20+ members), ensuring exceptional delivery quality, and building scalable data solutions using Databricks on cloud platforms.

The ideal candidate will have a proven track record of end-to-end delivery, strong stakeholder management capabilities, and deep expertise in modern data engineering practices, particularly in the Banking/Financial Services sector.

Key Responsibilities :

Technical Leadership & Architecture (50%) :

- Lead the design and implementation of enterprise-scale data platforms using Databricks as the primary technology stack.

- Define and enforce technical architecture, coding standards, and engineering best practices across all data engineering initiatives.

- Drive adoption of modern data engineering patterns including Medallion Architecture, Delta Lake, and Delta Live Tables (DLT).

- Conduct comprehensive solution design reviews and provide technical guidance on complex data engineering challenges.

- Review code quality, performance optimization strategies, and ensure adherence to data security, governance, and compliance standards.

- Mentor team members on advanced Databricks features, PySpark optimization, and cloud-native data architectures.

- Stay current with emerging technologies and industry trends; champion innovation within the team.

- Ensure implementation of robust data modeling strategies (Star Schema, Snowflake, Data Vault) and ETL/ELT design patterns.

Delivery Management & Execution (50%) :

- Own end-to-end delivery accountability for multiple data engineering initiatives, from requirements gathering through production deployment and post-go-live support.

- Lead and manage 2+ squads (20+ members) with a strong focus on delivery excellence and quality outcomes.

- Develop comprehensive project plans, timelines, and resource allocations with accurate estimation, budgeting, and costing.

- Execute sprint planning and management; track progress against milestones and KPIs.

- Proactively identify, assess, and mitigate delivery risks and dependencies.

- Ensure production readiness through rigorous testing, validation, and release management processes.

- Drive continuous improvement in engineering processes, tools, and team productivity.

- Manage stakeholder expectations, communicate progress transparently, and prioritize business requirements effectively.

People Leadership & Development :

- Lead, mentor, and develop a high-performing team of Data Engineers and Technical Leads.

- Conduct regular one-on-one meetings, performance reviews, and career development discussions.

- Support hiring initiatives, onboarding programs, and capability development plans.

- Foster a culture of collaboration, innovation, accountability, and continuous learning.

- Identify and nurture talent; create pathways for career growth within the team.

Operational Excellence & Stakeholder Management :

- Ensure platform reliability, availability, and optimal performance in production environments.

- Drive root cause analysis for production incidents and implement preventive measures.

- Improve monitoring, alerting, observability, and incident response capabilities.

- Optimize cloud infrastructure costs and resource utilization across Azure, AWS, or GCP.

- Collaborate effectively with Product Owners, Solution Architects, Business stakeholders, and Platform teams.

- Communicate technical decisions, risks, and delivery status to senior leadership and business partners.

Required Skills & Competencies :

Databricks & Data Engineering (Must-Have) :

- Databricks Expertise: Databricks Workspace, Delta Lake, Delta Live Tables (DLT), Unity Catalog, Databricks Workflows, Auto Loader, Structured Streaming

- Programming: Python, PySpark, Spark SQL (SQL is mandatory; Python/PySpark is good-to-have)

- Data Engineering Patterns: ETL/ELT design, Medallion Architecture, Batch and Streaming data pipelines

- Data Modeling: Star Schema, Snowflake Schema, Data Vault methodologies

- Database Technologies: SQL Server, Oracle, Snowflake, PostgreSQL

Cloud Platforms (Must-Have) :

- Hands-on experience with Microsoft Azure (primary) and/or AWS

- Cloud Storage: ADLS (Azure Data Lake Storage), S3, or GCS

- Understanding of cloud-native architectures and infrastructure optimization

DevOps & CI/CD :

- Git and version control best practices

- Azure DevOps or GitHub for CI/CD pipeline management

- Infrastructure as Code (Terraform preferred)

- Release management and deployment automation

Leadership & Delivery Management (Must-Have) :

- Engineering leadership with proven experience leading multiple squads (20+ members)

- Agile delivery and project management expertise

- Strong planning, estimation, budgeting, and costing skills

- Stakeholder management and executive communication

- Risk management and conflict resolution

- Coaching, mentoring, and team development capabilities

- Solution and design thinking with end-to-end delivery ownership

- Excellent communication and interpersonal skills

Domain Expertise (Preferred) :

- Banking/Financial Services industry experience (highly preferred)

- Understanding of financial data governance, compliance (GDPR, SOX), and security requirements

Experience Requirements :

- Total Experience: 10-15+ years in Data Engineering and related roles

- Leadership Experience: 3-5+ years leading engineering teams or squads

- Databricks Experience: Hands-on, production-grade experience with Databricks on Azure, AWS, or GCP (mandatory)

- Enterprise-Scale Platform Development: Proven experience building and delivering enterprise-scale data platforms

- End-to-End Delivery: Demonstrated ability to own complete delivery lifecycle from requirements gathering, design, development, testing, production deployment, and post-go-live support

- Performance Tuning & Optimization: Experience optimizing data pipeline performance, cost management, and production support

- Solution Architecture: Strong background in designing and implementing complex data solutions

Preferred Certifications & Qualifications :

- Databricks Certified Data Engineer Professional

- Azure Data Engineer Associate or Azure Solutions Architect Expert

- AWS Certified Data Analytics Specialty

- Bachelor's degree in Computer Science, Engineering, or related field

Nice-to-Have Skills :

- MLflow and ML pipeline orchestration

- Databricks Asset Bundles (DABs)

- Apache Kafka or Azure Event Hubs experience

- AI/ML pipelines and Generative AI knowledge

- Data Mesh or Data Fabric architecture experience

- Databricks Genie or AI-assisted development tools

- NoSQL databases (MongoDB, Cassandra)

- Advanced monitoring and observability tools

Key Competencies :

- Technical Expertise: Deep hands-on knowledge of modern data engineering and cloud technologies

- Leadership: Ability to inspire and lead high-performing teams toward ambitious goals

- Delivery Excellence: Demonstrated track record of on-time, quality delivery of complex projects

- Strategic Thinking: Capability to align technical solutions with business objectives

- Communication: Exceptional verbal and written communication skills; ability to articulate complex concepts to diverse audiences

- Problem-Solving: Strong analytical and troubleshooting capabilities

- Adaptability: Comfortable in fast-paced, dynamic environments with evolving requirements

- Accountability: Takes ownership of outcomes and drives results

What We're Looking For :

- A hands-on technical leader who can code and architect solutions while managing teams

- Someone with proven expertise in Databricks and enterprise data platform development

- A delivery-focused professional with strong planning and estimation capabilities

- A leader who has successfully managed multiple squads and driven complex, multi-initiative programs

- Banking/Financial Services industry experience is a significant plus

- An individual with excellent communication skills who can engage effectively with all stakeholder levels

- A champion of quality, continuous improvement, and operational excellence