Position: Agentic QA Engineer

Overall/Total Experience: 2 - 5 years

Location: Bengaluru

Working Days: 5 Days from Office

Notice Period Requirement: 30 Days (Maximum)

Client's Company Size: Startup / Small Enterprise

Roles & Responsibilities:

- Test and validate AI agents to ensure they perform actions accurately and as designed.

- Develop and execute test cases for different conversational and task-based AI workflows.

- Analyze test results, identify bugs or performance issues, and collaborate with the engineering team for fixes.

- Document testing procedures, outcomes, and recommendations for improvement.

- Ensure that AI agents handle prompts, contexts, and actions correctly.

- Contribute to the creation of automated QA processes for AI systems.

Candidate Requirements:

- Strong QA Engineer profile with AI / GenAI agent testing experience.

- 2+ years of experience in QA, software testing, or AI testing roles.

- Experience testing and validating AI agents to ensure they perform actions accurately and as designed.

- Ability to develop and execute test cases for conversational and task-based AI workflows.

- Ability to analyze test results, identify bugs or performance issues, and collaborate with engineering for fixes.

- Good understanding of Generative AI, LLM agents, or autonomous AI workflows.

- Understanding of DeepEval and other AI testing frameworks.

- Manual testing experience and familiarity with automation tools.

- Strong problem-solving, analytical, and communication skills.

- Ability to collaborate with developers, prompt engineers, and data scientists.

Preferred Skills:

- Exposure to LLM frameworks (LangChain, CrewAI, AutoGen, etc.).

- Basic programming or scripting knowledge (Java or Python).

- Understanding of AI evaluation metrics and model performance tuning.