Lead Data Engineer - Python

Responsibilities :

- Own end-to-end data architecture across ingestion, transformation, storage, and analytics

- Design and scale batch and streaming pipelines using Python, Airflow, Spark /PySpark, Kafka, dbt, and Debezium

- Architect and optimize data models across MongoDB, ClickHouse, and OpenSearch for high-volume ingestion and low-latency analytics

- Establish strong data quality, validation, and lineage practices using Great Expectations or similar frameworks

- Build clean, well-documented, analytics-ready datasets that power dashboards and AI workflows

- Mentor and guide a small team of 2 to 3 data engineers, setting standards and driving execution

- Partner closely with product, AI, and platform teams to enable new data use cases and faster experimentation

- Continuously improve performance, reliability, and cost efficiency on AWS analytics

- Requirements :

- 4 to 9 years of experience in data engineering or data platform roles

- At least 2 years of experience leading or mentoring engineers

- Strong hands-on expertise in Python, Apache Airflow, Spark / PySpark, and MongoDB

- Solid understanding of ETL / ELT patterns, streaming systems, CDC, and scalable data modeling

- Experience designing systems that balance flexibility, performance, and data quality

- Working knowledge of AWS and production performance tuning

Nice to Have :

- Exposure to supply chain, procurement, pricing, or marketplace data

- Experience with data contracts, observability, or data SLAs

- Familiarity with privacy and compliance concepts such as GDPR or India DPDP