Key Responsibilities :

1. Data Pipeline Ownership :

- Design, build, and maintain scalable ETL pipelines for tick, OHLC, fundamentals, and alternative data.

- Develop automated ingestion systems for both real-time and historical market data.

- Ensure fault-tolerant, high-availability data workflows.

2. Data Quality & Integrity :

- Detect and fix data anomalies: stale ticks, bad prints, missing data, timestamp issues.

- Implement automated QA, validation checks, and reconciliation workflows.

- Own daily data quality reporting.

3. Data Architecture & Storage :

- Manage storage layers.

- Optimize data retrieval for quants and live trading systems.

- Implement data versioning and point-in-time correctness for backtesting.

4. Vendor & Exchange Data Management :

- Integrate and manage feeds from Bloomberg, Refinitiv, FactSet, Tick data providers, NSE/BSE, global exchanges.

- Be primary contact for vendor issue resolution.

- Ensure compliance with vendor licensing and data usage policies.

5. Cross-Team Collaboration :

- Work closely with quant researchers to understand data needs and create feature-ready datasets.

- Support engineering teams in live feed integration, API access, and latency monitoring.

- Work with product and compliance on data governance and documentation.

6. Leadership & Mentorship :

- Lead a small team of data engineers/analysts.

- Review code, enforce best practices, and drive technical excellence.

- Build roadmap for data capabilities and automation.

Required Skills & Experience :

- 6 - 10 years of experience in market data, data engineering.

- Hands-on experience with :

1. Python (mandatory)

2. SQL and distributed databases

3. ETL frameworks

4. Message queues (Kafka/Pulsar a plus)

- Strong debugging skills for live and historical data issues