Role: QA Lead for AI Project

Client Healthcare Domain

Location Bangalore (hybrid)

Duration 12 months

Start date ASAP (only immediate joiners)

1. need someone with global exposure, teams based in India, USA and The Netherlands

2. the visibility of this project is with the CEO and CTO , so need someone with excellent comms and AI exp

3. need someone who is okay working flexible hours

4. healthcare domain exp is required and data standards

5. they will be in the capacity of lead but IC role

6. need a senior resource

7. Exp in AWS/Azure mandatory

What is expected?

- Design, automate, and execute API test suites (functional, contract, performance, resilience)

- Define LLM evaluation strategies (performance testing, golden sets, guardrails, hallucination/consistency checks, relevance & latency KPIs); build automated eval pipelines tied to releases.

- Create PDLM deliverables: verification plans/protocols, test reports, risk-based coverage, V&V trace matrices to requirements and use cases.

- Integrate test infrastructure with Azure DevOps (CI/CD, test orchestration, quality gates, dashboards); enforce build-time gates for API and LLM quality.

- Partner with Product, Clinical, and R&D to refine requirements and acceptance criteria; ensure summaries render correctly for different user roles and workflows.

What is required from candidate?

- 8+ years in backend/API test automation (REST, JSON, auth), performance testing, and CI/CD.

- Hands-on with Python/TypeScript, Postman/Newman or REST-assured, PyTest/Jest, k6/JMeter, Docker.

- Experience validating ML/LLM systems (eval datasets, metrics, prompt/response testing, safety/guardrails).

- Strong documentation and traceability discipline for regulated product lifecycles (PDLM/QMS).

- Bonus: Azure DevOps pipelines, FHIR/HL7 familiarity, healthcare data privacy/security.

- Bonus: Frontend testing capabilities are preferred, but not part of the PDLM requirements.