Description :

Job Title : Automation Engineer (ETL / Data Testing)

- Experience : 3+ Years

- Notice Period : Immediate to 30days only

- Location : Chennai

- Workmode : Hybrid

Job Summary :

We are looking for an ETL Data Tester / Automation Engineer to support a large-scale One Integrated Platform (OIP) data initiative. This role focuses on validating complex data processing and transformation logic involving consumer data. The candidate will be responsible for creating and executing test cases, validating business rules, and building automation frameworks primarily within Azure Databricks and Azure Data Factory. The ideal candidate should be passionate about data, possess strong analytical skills, and thrive in data-intensive environments.

Role & responsibilities :

Key Responsibilities :

Data & ETL Testing :

- Create and execute test scenarios to validate APIs, including request preparation and response analysis in JSON/XML formats.

- Validate end-to-end data flow from multiple source systems into Databricks, followed by transformations through Azure Databricks and Azure Data Factory (ETL) pipelines.

- Thoroughly test ETL rules, data transformations, and complex business logic for data consolidation and consumer merging.

- Validate source and target tables within Azure Databricks, including cleansed, transformed, joined, aggregated data sent to downstream systems.

- Execute data validation using SQL, Python, or PySpark based on user stories and technical specifications.

Automation & Framework Development :

- Automate recurring QA processes using Python, PySpark, or Java as required.

- Design, develop, and maintain automation frameworks using PyTest for data quality and validation scenarios.

- Enhance existing automation scripts and build new test coverage as needed.

Quality Assurance & Collaboration :

- Create and execute detailed manual test cases using functional and technical requirements.

- Log and track defects in Jira, and collaborate with development teams for defect validation and resolution.

- Create and manage test cases using TestRail.

- Participate in daily stand-ups and work closely with cross-functional teams in an Agile environment.

- Update Jira tasks regularly to communicate testing progress.

Minimum Qualifications & Requirements :

- 3+ years of hands-on experience with Azure Databricks.

- 1+ year of experience working with Azure Data Factory.

- 3+ years of strong experience writing complex SQL queries.

- 3+ years of experience with Python programming and/or PySpark.

- 3+ years of experience in test automation for ETL or data-intensive projects.

- Experience with code repository tools (branching, pull requests, code merges).

- Exposure to SonarQube and code quality practices is a plus.

- Good understanding of data file formats such as JSON, Parquet, Avro, etc.

- Working knowledge of Jira and Agile methodologies.

- Strong problem-solving skills and ability to manage multiple tasks in a fast-paced environment