QA Automation Engineer - GenAI / Agentic AI

We are looking for a QA Automation Engineer with 2+ years of relevant experience in test automation, including hands-on experience testing Generative AI and Agentic AI applications.

Key Responsibilities :

- Design, develop, and maintain automated test frameworks using Playwright or similar tools.

- Build automated UI, API, integration, and end-to-end test suites for GenAI and Agentic AI applications.

- Validate LLM responses, RAG workflows, agent/tool interactions, multi-step agentic workflows, and failure scenarios.

- Test AI applications for accuracy, consistency, hallucinations, response quality, latency, reliability, and regression.

- Automate testing of RAG pipelines, including retrieval quality, context relevance, citations, and response grounding.

- Validate multi-agent workflows, tool calling, retries, state transitions, guardrails, and human-in-the-loop scenarios.

- Integrate automated tests into CI/CD pipelines and support continuous quality monitoring.

Required Skills :

- 2+ years of relevant QA automation experience with exposure to GenAI / Agentic AI applications.

- Strong hands-on experience with Playwright and modern automation frameworks.

- Experience with API testing, UI automation, integration testing, and end-to-end testing.

- Good understanding of Generative AI, RAG, LLMs, Agentic AI, and multi-agent workflows.

- Experience testing non-deterministic AI outputs using appropriate evaluation and validation approaches.

- Strong programming skills in Python, JavaScript, or TypeScript.

- Experience with CI/CD, Git, test reporting, defect tracking, and debugging.

- Understanding of LLM evaluation, prompt testing, guardrails, performance testing, and AI application observability is preferred.