Role Overview :

We are seeking a detail-oriented QA Engineer with hands-on experience in testing AI/LLM-based applications. The ideal candidate should have a strong understanding of modern testing practices, along with exposure to evaluating Large Language Model (LLM) outputs using structured frameworks.

Key Responsibilities :

- Validate and test applications powered by Large Language Models (LLMs) for accuracy, consistency, and performance

- Design and execute test strategies specifically for AI-driven workflows and conversational systems

- Use evaluation tools and frameworks such as DeepEval or similar to assess model responses

- Identify edge cases, biases, and inconsistencies in LLM outputs

- Collaborate closely with developers, data scientists, and product teams to improve model quality

- Develop and maintain test cases, test plans, and automation scripts where applicable

- Track bugs, report issues, and ensure timely resolution

- Contribute to continuous improvement of QA processes for AI-based systems

Required Skills :

- 3-4 years of experience in Software Testing / QA

- Hands-on exposure to testing AI/ML or LLM-based applications

- Familiarity with LLM evaluation frameworks (e.g., DeepEval or similar tools)

- Strong understanding of manual and/or automated testing methodologies

- Ability to analyze model outputs and identify logical or contextual errors

- Experience with bug tracking and test management tools

- Good problem-solving and analytical thinking skills

Preferred Qualifications :

- Experience in testing NLP or conversational AI systems

- Basic understanding of prompt engineering concepts

- Exposure to Python or scripting for test automation

- Familiarity with Agile/Scrum environments

Desired Profile :

- Strong attention to detail and critical thinking ability

- Good communication and collaboration skills

- Proactive mindset with ownership of deliverables

- Ability to adapt in a fast-evolving AI/ML landscape