Job Title : Lead Data Engineer.

Location : Navi Mumbai / Mumbai.

Reporting To : Project Manager.

Education : Bachelor's degree in Engineering (CS, IT, EXTC, or equivalent).

Experience Required : 6-8 years.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines and analytical solutions using PySpark and Spark SQL.

- Write efficient, clean, and reusable Python code for data transformation, cleaning, and processing.

- Work with structured, semi-structured, and unstructured data to deliver business-ready datasets.

- Collaborate with Data Analysts and BI Developers to ensure delivery of clean, processed, and optimized data.

- Build and manage ETL processes and data integration workflows.

- Optimize complex SQL queries, functions, views, and indexing strategies for performance.

- Perform exploratory data analysis (EDA) and support data ingestion from various sources.

- Communicate with stakeholders to gather requirements and deliver effective data engineering solutions.

- Ensure adherence to best coding practices, documentation, and version control standards.

- Coordinate with cross-functional engineering and development teams for end-to-end solution delivery.

Required Skills :

- Strong experience with Apache Spark, PySpark, and Spark SQL.

- Proficient in Python and commonly used Python libraries for data processing.

- Deep understanding of SQL, query optimization, and data modeling.

- Experience with Big Data technologies and handling large datasets.

- Familiarity with any ETL tools is a plus.

- Strong analytical thinking and problem-solving skills.

- Excellent communication and collaboration abilities.