Overview :

We are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines.

The ideal candidate has deep expertise in modern data engineering, cloud-native services, real-time data ingestion & CDC frameworks.

This role involves leading complex data initiatives, guiding engineering teams, and collaborating with cross-functional stakeholders to deliver robust, scalable, and secure data solutions.

Key Responsibilities :

Data Architecture & Pipeline Development :

- Design and build scalable ETL/ELT pipelines using Python / PySpark.

- Architect and implement batch and real-time data pipelines using Kafka, Kafka Connect, and Debezium.

- Develop and optimize data ingestion workflows across multiple databases (MySQL, Postgres, MongoDB).

- Build and manage Apache Airflow DAGs for end-to-end pipeline orchestration.

Cloud Data Engineering :

- Lead data platform development on GCP (BigQuery, Data Proc, Data Stream ).

- Implement data ingestion, orchestration, and transformation at scale using Pub/Sub, Cloud Functions/Cloud Run, Dataflow, and GKE.

- Own data warehouse design and optimization across Snowflake and BigQuery.

Requirements :

Required Skills & Experience :

Core Technical Skills :

- Strong proficiency in Python, SQL, PySpark.

- Hands-on expertise with Kafka, Kafka Connect, Debezium, Airflow, Databricks.

- Deep experience with BigQuery, Snowflake, MySQL, Postgres, MongoDB.

- Solid understanding of vector data stores and search indexing.

- Knowledge of GCP services like Big Query, Cloud Functions, Cloud Run, Data Flow, Data Proc, Data Stream, etc.

Good to have :

Certifications :

- GCP Professional Data Engineer.

- Elastic Certified Engineer.

AI :

- Gemini Enterprise.

- Vertex AI Agent Builder.

- ADK.

Non-Technical & Leadership Skills :

- Communication : Exceptional verbal and written communication skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.

- Mentorship & Coaching : Proven experience in mentoring junior and mid-level engineers, fostering a culture of continuous learning and growth.

- Problem-Solving : Strong analytical and debugging skills, with a proactive approach to identifying and resolving technical roadblocks.

- Ownership & Accountability : Demonstrates a high level of responsibility for project outcomes, system reliability, and code quality.

- Agile Proficiency : Deep understanding and practical experience with Agile methodologies (Scrum/Kanban).

- Stakeholder Management : Ability to effectively manage expectations and build consensus across different teams.

Qualifications :

- Bachelors or Masters degree in Computer Science, Engineering, or a related field (or equivalent practical experience).

- Typically 7+ years of progressive experience in data engineering, with 2+ years in a technical leadership or lead engineer role.