Job Summary:

We are looking for a skilled Python / PySpark Developer building scalable data processing solutions. The ideal candidate should have strong expertise in Python and PySpark, with experience handling large datasets and developing efficient data pipelines.

Key Responsibilities:

- Design, develop, and maintain data pipelines using Python and PySpark

- Process and analyze large-scale structured and unstructured data

- Build and optimize ETL/ELT workflows

- Write efficient and optimized data transformation logic

- Ensure data quality, integrity, and performance

- Collaborate with data engineers, analysts, and stakeholders

- Troubleshoot and resolve data processing issues

- Participate in code reviews and follow best practices

Required Skills:

- Strong proficiency in Python (mandatory)

- Hands-on experience with PySpark / Apache Spark

- Experience in building data pipelines and ETL processes

- Strong SQL knowledge

- Understanding of distributed computing concepts

- Experience with version control tools (Git)

Preferred Skills:

- Experience with cloud platforms (AWS / Azure / GCP)

- Familiarity with big data tools and ecosystems

- Knowledge of data warehousing concepts

- Exposure to orchestration tools (Airflow)

- Basic understanding of Databricks

Soft Skills:

- Strong analytical and problem-solving skills

- Good communication and teamwork abilities

- Ability to work in an agile environment