Job Description : QA Engineer AI Testing & Automation

Location : Hyderabad / Chennai / Bengaluru

Experience : 5-10 Years

Employment Type : Full-Time, Permanent

Work Mode : Hybrid

Must-Have Skills (Non-Negotiable) :

- AI Testing / AI Product Validation

- Test Automation using Python & Selenium

Role Overview :

We are looking for an experienced QA Engineer with expertise in AI Testing, Test Automation, Functional Testing, and API Testing. The ideal candidate will have hands-on experience validating AI/ML models, designing automation frameworks, and ensuring the quality, reliability, and performance of AI-powered applications.

Key Responsibilities :

- Design and execute comprehensive test plans, test cases, and test strategies for AI models and AI-driven applications.

- Perform functional, regression, API, and automation testing to ensure application quality and stability.

- Validate AI models using industry-standard AI evaluation frameworks and methodologies.

- Evaluate AI systems using key metrics such as Accuracy, Precision, Recall, and other AI performance indicators.

- Develop and execute AI-specific test scenarios, including RAG (Retrieval-Augmented Generation) workflows and AI model validation.

- Work with AI evaluation frameworks such as DeepEval or similar tools to assess model performance.

- Analyze test results, identify defects, and provide actionable recommendations to improve AI model quality and application performance.

- Develop and maintain automation test scripts using Python and Selenium.

- Integrate automated test suites into CI/CD pipelines to support continuous testing.

- Participate in automation code reviews and contribute to the overall QA strategy.

- Collaborate closely with developers, AI engineers, and product teams in an Agile environment.

- Utilize Azure DevOps for test management, defect tracking, and release coordination.

Required Skills & Qualifications :

- 5+ years of experience in Functional Testing, API Testing, Test Automation, and AI Testing.

- Strong experience in Python scripting and Selenium automation.

- Hands-on experience with AI testing methodologies and AI product validation.

- Good understanding of AI evaluation metrics including Accuracy, Precision, Recall, and model performance analysis.

- Experience testing RAG-based applications and AI workflows.

- Hands-on exposure to DeepEval or similar AI evaluation frameworks.

- Experience with API testing tools such as Postman, Rest Assured, or equivalent.

- Familiarity with CI/CD pipelines and automation integration.

- Working knowledge of Azure DevOps.

- Experience working in Agile/Scrum development environments.

- Strong analytical, debugging, and problem-solving skills.

- Excellent communication and collaboration abilities.

Preferred Skills :

- Exposure to Machine Learning concepts and Generative AI applications.

- Experience with AI model performance optimization and validation frameworks.

- Knowledge of cloud platforms such as Azure, AWS, or GCP is an added advantage.

Education :

- UG : Any Graduate

- PG : Any Postgraduate (Preferred)

Hiring Criteria :

Candidates must have hands-on experience in both AI Testing and Test Automation. Profiles without practical AI testing experience will not be considered.