Position: Data Quality & Test Engineer

Total Experience: 5 - 6 years

Location: Hybrid / Remote

Employment Type: Full-Time

Job Summary:

We are seeking a highly motivated Data Quality & Test Engineer with 5 - 6 years of experience in testing modern data platforms and cloud-based data engineering solutions. The ideal candidate will be responsible for validating data pipelines, ETL workflows, machine learning processes, and cloud infrastructure components. This role requires strong expertise in test automation, AWS services, data quality validation, CI/CD testing, and security verification to ensure high-quality and reliable data products.

Key Responsibilities:

Data Validation & Quality Assurance:

- Define, develop, and execute comprehensive test plans for data pipelines, ETL/ELT processes, Machine Learning workflows, and cloud infrastructure components.

- Validate data accuracy, completeness, consistency, and integrity across Raw, Curated, and Summarized/Consumption layers.

- Verify schema consistency, data mappings, transformations, and business rules.

- Conduct end-to-end data reconciliation and source-to-target validation.

Test Automation:

- Design, develop, and maintain automated test frameworks and scripts.

- Create and execute Unit, Integration, Regression, and Data Quality Tests.

- Develop automated test cases for AWS Glue, Lambda, Step Functions, and API integrations.

CI/CD & Release Validation:

- Validate CI/CD deployments across development, QA, and non-production environments.

- Collaborate with DevOps teams to perform deployment verification and release testing.

Observability & Monitoring Validation:

- Verify platform observability mechanisms including application logging, CloudWatch metrics, dashboards, and alerting.

Security & Compliance Testing:

- Conduct security validation including RBAC, IAM policies, and data access controls.

Required Skills:

- 5 - 6 years of experience in Data Testing, ETL Testing, Data Quality Engineering, or QA Automation.

- Strong understanding of Data Warehousing, Data Lakes, and modern Data Platforms.

- Hands-on experience with AWS services: Glue, Lambda, Step Functions, S3, CloudWatch, IAM.

- Proficiency in SQL, Python, and automated testing frameworks.

- Knowledge of CI/CD tools (Azure DevOps, GitHub Actions, Jenkins, or GitLab).

- Experience testing REST APIs and event-driven architectures.

Preferred Qualifications:

- Experience with Data Lakehouse architectures and Machine Learning workflow testing.

- Experience with Great Expectations, PyTest, or similar data validation frameworks.

- Knowledge of Infrastructure as Code (Terraform, CloudFormation).

- AWS Certifications (Cloud Practitioner, Developer Associate, or Data Engineer Associate).