AWS QA Engineer

Location : Bangalore / Pune, India

Experience : 5 - 10 Years

Employment Type : Full-time

Job Overview :

We are seeking a Senior QA Data Engineer with expertise in Python, PySpark, SQL, and data testing automation. The ideal candidate will be instrumental in ensuring the quality and reliability of our data pipelines through robust testing, and will collaborate closely with DevOps teams on CI/CD pipelines. Experience with AWS Cloud services is a significant plus for this role.

Key Responsibilities :

- Design, develop, and execute comprehensive test plans and strategies for data pipelines.

- Utilize Python, PySpark, and SQL for data validation, integrity checks, and testing automation.

- Implement and manage data testing automation frameworks to ensure the reliability and quality of data flowing through the pipelines.

- Collaborate effectively with DevOps teams to integrate automated tests into CI/CD pipelines.

- Identify, document, and track defects, working closely with development teams to ensure timely resolution.

- Conduct thorough analysis of data sources, transformations, and destinations to ensure data accuracy and completeness.

- Monitor data pipeline performance and proactively identify areas for optimization and improvement.

- Contribute to the continuous improvement of QA processes and best practices for data quality.

Required Skills & Qualifications :

- 5 - 10 years of experience as a QA Data Engineer or in a similar role focusing on data quality and testing.

- Strong proficiency in Python for scripting and automation.

- Hands-on experience with PySpark for testing data transformations in big data environments.

- Expert-level skills in SQL for data querying, validation, and analysis.

- Proven experience in designing and implementing data testing automation.

- Experience with AWS Cloud services, including AWS Glue, AWS Lambda, and S3.

- Familiarity with Jenkins for CI/CD pipeline integration.