Location : Hyderabad/New Delhi

Mode : Hybrid

Shift : UK Shift

About the Role :

Are you passionate about ensuring Artificial Intelligence is reliable, safe, and truthful? We are looking for a QA Engineer to join our AI team to focus exclusively on testing the next generation of Agentic AI systems.

In this role, you will act as the gatekeeper of quality, ensuring our LLM applications, AI agents, and RAG systems produce accurate, logical, and reliable outcomes.

Key Responsibilities :

Design Evaluation Frameworks :

- Architect and implement comprehensive evaluation strategies for LLMs and Agentic AI systems to measure accuracy, safety, and reasoning quality.

Hallucination & Reliability Testing :

- Develop test cases specifically designed to detect hallucinations, factual errors, logical inconsistencies, tool misuse, and retrieval failures in RAG pipelines.

Benchmark Development :

- Create and maintain benchmark datasets and automated test suites to validate AI responses against ground truth data.

Multi-Step Workflow Testing :

- Evaluate the performance of AI agents performing complex, multi-step tasks and tool-using operations.

Cross-Functional Collaboration :

- Partner closely with AI engineers and data scientists to debug model behavior and iterate on prompt engineering based on test findings.

What You Bring (Requirements) :

Experience :

- 3+ years in Quality Assurance or Software Testing, with a strong focus on AI/ML systems in the last 1 - 2 years.

AI Expertise :

- Deep understanding of LLM behaviors (hallucinations, reasoning chains) and Agentic AI architecture (tool use, RAG).

Technical Proficiency :

- Experience using Python for test automation and API testing tools.

- Familiarity with LLM evaluation libraries (e.g., LangChain, DeepEval, or similar).

Analytical Mindset :

- A sharp eye for detail and the ability to classify ambiguous outputs (e.g., distinguishing between a 'creative' answer and a 'factually incorrect' one).

Nice to Have :

- Experience with prompt injection security testing or knowledge of ML pipelines