About the Role :

We are looking for a Senior Python Developer with strong hands-on experience in Python and Apache Spark to design, develop, and optimize scalable, high-performance data applications. The role involves working across Spark Core, Spark SQL, Spark Streaming, and the Hadoop ecosystem while providing technical guidance to junior developers.

Responsibilities :

- Design and develop efficient, reusable, scalable, and reliable Python applications.

- Develop data processing solutions using Apache Spark, including Spark Core, Spark SQL, and Spark Streaming.

- Work with Python and Spark within the Hadoop ecosystem to process and analyze large-scale datasets.

- Use Pandas for data manipulation, transformation, and analysis, and Polars for efficient data processing where appropriate.

- Design and implement low-latency, highly available, and high-performance applications.

- Perform performance tuning, optimization, automation, and troubleshooting across applications and data processing workloads.

- Write clean, maintainable, and production-ready code following software development best practices.

- Conduct coding, testing, debugging, root-cause analysis, and production issue resolution.

- Collaborate with engineering teams and business stakeholders to understand requirements and deliver scalable solutions.

- Lead and mentor junior Python developers, provide technical guidance, and conduct code reviews.

- Contribute to continuous improvement of development processes, application performance, usability, and automation.

Tech Stack & Requirements :

- 4 - 9 years of experience in Python development, data engineering, or a closely related field.

- Strong hands-on proficiency in Python programming.

- Strong hands-on experience with Apache Spark and Spark programming.

- Good knowledge of Spark Core, Spark SQL, and Spark Streaming.

- Experience working with Spark and the Hadoop ecosystem.

- Strong experience with Pandas for data manipulation and analysis.

- Working knowledge of Polars for high-performance data processing.

- Experience developing and optimizing high-performance and scalable applications.

- Strong debugging, troubleshooting, and performance-tuning skills.

- Experience with testing, code reviews, and software development best practices.

- Ability to lead and mentor junior developers.

- Strong problem-solving, communication, and stakeholder collaboration skills.