Overview of Role :

As a Senior QA Engineer - Data, you will lead the QA effort for complex data engineering pipelines on Google Cloud Platform.

You will design and implement multi-layered testing strategies - integration, end-to-end, and data quality tests - across tools like dbt, Dataflow, Dataproc, BigQuery, AlloyDB, Cloud SQL, Cloud composer and Cloud Run, etc.

Your work ensures the accuracy, reliability, and performance of data systems at scale.

Responsibilities :

Testing Strategy & Test Design :

- Define and maintain testing methodologies for the full GCP data engineering stack : Dataflow (TestPipeline), Dataproc (spark-testing-base or pytest), Cloud Run container tests, and SQL-based data validation in BigQuery/dbt.

- Develop and execute data quality frameworks using dbt tests (schema, singular, freshness) and external tools like Great Expectations, Soda Core, and Dataplex.

Pipeline & Database Testing :

- Implement integration, and regression tests for ETL/ELT pipelines, including container-level and HTTP-triggered tests for Cloud Run.

- Use emulators or dedicated test instances to test Spanner, Cloud SQL, and AlloyDB.

- Validate stored procedures and database functions with sample data.

End-to-End (E2E) Pipeline Validation :

- Orchestrate comprehensive E2E tests via Cloud Composer/Apache Airflow or scripting.

- Simulate real-world data flows and validate intermediate and final outputs.

CI/CD & Automation :

- Embed QA in CI/CD pipelines (e.g., GitLab CI, Jenkins, GitHub Actions), automating test execution at all levels including data quality validations and dbt runs.

- Use IaC tools (Terraform, Deployment Manager) to provision reproducible test environments.

Collaboration & Stakeholder Engagement :

- Partner with data engineers and stakeholders to review design for testability.

- Mentor junior QA team members, champion QA best practices, and lead efforts to improve data quality KPIs and test effectiveness.

Monitoring & Observability :

- Utilize Cloud Logging, Monitoring, and observability tools (e.g., Elementary Data) to track pipeline health, test results, and identify anomalies.

Required Qualifications :

- 5+ years in QA or software testing, including focused on data pipelines/data warehouses.

- Proficient in complex SQL and writing data validation queries.

- Strong experience with GCP data tools : dbt, BigQuery, Dataflow, Dataproc, Cloud Run, Spanner, AlloyDB, Cloud SQL.

- Hands-on with data-quality frameworks : Great Expectations, Soda Core, dbt-utils, dbt-expectations.

- Familiarity with database emulators and test environment provisioning.

- CI/CD automation experience (Jenkins, GitLab CI, GitHub Actions).

- Skilled in scripting languages (Python, pytest).

- Excellent communication, mentoring ability, critical thinking, and attention to detail.

- API testing with PI Testing Postman REST APIs.

Preferred Qualifications :

- GCP Professional Data Engineer certification.

- Experience enhancing dbt tests with dbt-utils and dbt-expectations.

- Knowledge of BigQuery data-quality services like Dataplex.

- Proficient with observability tooling (Cloud Logging, Elementary Data).

- Familiarity with data governance, lineage, and compliance validation.

- Exposure enterprise-based data migration project.

- Exposure to Datawarehouse modernization.