Job Summary :

We are hiring a Python PySpark Data Engineer to develop scalable data processing applications and high-performance data pipelines. The ideal candidate should have strong programming expertise in Python and PySpark along with experience in distributed data processing.

Key Responsibilities :

- Develop scalable data pipelines using Python and PySpark.

- Build batch and real-time data processing solutions.

- Design ETL workflows for large datasets.

- Optimize Spark jobs for performance.

- Develop reusable data engineering frameworks.

- Integrate multiple enterprise data sources.

- Perform unit testing and production support.

- Collaborate with Analytics, BI, and Data Science teams.

Required Skills :

- 6-11 years of Data Engineering experience.

- Strong programming expertise in Python.

- Hands-on experience with PySpark.

- Excellent SQL skills.

- Experience with Spark optimization and tuning.

- Knowledge of Hadoop ecosystem.

- Experience with Airflow or similar orchestration tools.

- Experience with Kafka is preferred.

Preferred Skills :

- AWS/GCP/Azure

- Docker

- Kubernetes

- Delta Lake

- Git & CI/CD

- Agile/Scrum methodology