Description :

Job Title : Agentic QA Engineer

Experience Required : 2 to 4 years

Location : Bengaluru (HSR Layout)

About the Role :

We are looking for an Agentic QA Engineer who will be responsible for testing, validating, and ensuring the accuracy and reliability of our AI agents and automation workflows. The ideal candidate will have hands-on experience with GenAI-based systems, QA methodologies, and AI testing frameworks to guarantee high-quality performance of AI-driven solutions.

Key Responsibilities :

- Test and validate AI agents to ensure they perform tasks accurately and as intended.

- Design, develop, and execute test cases for conversational and task-based AI workflows.

- Analyze test results and identify functional issues, bugs, or performance gaps.

- Work closely with engineering, data science, and prompt engineering teams to resolve issues.

- Document test processes, results, and improvement recommendations.

- Ensure AI agents correctly interpret prompts, contexts, and multi-step actions.

- Contribute to the development of automated QA pipelines for AI systems.

Required Skills & Qualifications :

- 1- 5 years of experience in QA, software testing, or AI testing roles.

- Strong understanding of Generative AI, LLM agents, or autonomous AI systems.

- Familiarity with DeepEval, AI evaluation tools, or similar frameworks.

- Experience in manual testing; exposure to automation testing tools is a plus.

- Strong analytical thinking, problem-solving ability, and communication skills.

- Ability to collaborate effectively with cross-functional teams.

Good to Have :

- Exposure to frameworks like LangChain, CrewAI, AutoGen, or other LLM orchestration tools.

- Basic scripting/programming skills in Python or Java.

- Understanding of AI evaluation metrics, model behavior, and performance tuning.