Position : Big Data Tester

Experience : 5-9 Years

Location : Pune, India

Job Summary :

We are seeking an experienced Big Data Tester with 5-9 years of experience in software quality assurance, with a dedicated focus on data. The ideal candidate will have a strong background in Python, PySpark, and SQL, along with a proven track record of implementing best practices for data validation and ETL testing. You will be responsible for ensuring the accuracy, integrity, and reliability of our large-scale data systems. This role is perfect for a detail-oriented professional with a strong technical skillset and a passion for data quality.

Key Responsibilities :

Data Validation and Testing :

- Design, develop, and execute comprehensive test plans and test cases for Big Data applications.

- Perform data validation and integrity checks across various data sources and systems.

- Use Python and PySpark to write scripts and automate testing for ETL processes and data pipelines.

SQL & Data Querying :

- Write and optimize complex SQL queries to validate data transformations and business logic.

- Perform data analysis and use querying skills to identify discrepancies and anomalies.

Troubleshooting and Collaboration :

- Work closely with data engineers and developers to troubleshoot and resolve data quality issues.

- Participate in the entire software development lifecycle, ensuring quality is integrated from the start.

Required Skills & Qualifications

Core Experience : 5-9 years of experience in a testing or quality assurance role.

Mandatory Technical Skills :

- Expertise in Python and PySpark for data processing and automation.

- Extensive hands-on experience with SQL, including writing complex queries.

- Strong knowledge of ETL processes and best practices for data validation.

Professional Attributes :

- A strong, hands-on approach to testing and a meticulous attention to detail.

- Excellent analytical and problem-solving skills.

- Strong communication and collaboration skills to work effectively with cross-functional teams.

Preferred Skills :

- Experience with Unix environments.

- Hands-on experience with Big Data technologies such as Hadoop, Spark, Kafka, and NoSQL databases (MongoDB, Cassandra, Hive, etc.).

- Experience with both traditional and modern Data Warehousing solutions (Oracle, Teradata, SQL Server, Amazon Redshift, Google BigQuery, Snowflake).

- Prior development experience in AWS.