API Test Engineer - Automation Testing

Description :

Job Title :

AI Testing Engineer. YOE: 6+ Years. Location: Bangalore. Notice Period: Immediate to 15 Days. We are looking for an experienced AI Testing Engineer with strong expertise in AI/ML Testing, Python Automation Testing, and Large Language Models (LLMs).

The candidate will be responsible for validating AI-driven applications, ensuring model accuracy, reliability, performance, and ethical behavior.

Key Responsibilities :

- Perform functional, regression, and performance testing for AI systems.

- Validate LLM outputs for accuracy, relevance, bias, hallucination, and edge cases.

- Develop and maintain automated test frameworks using Python.

- Test APIs, data pipelines, and model integrations.

- Validate model behavior across different prompts, datasets, and scenarios.

- Collaborate with data scientists, ML engineers, and developers.

- Identify, log, and track defects related to AI behavior and performance.

- Prepare test reports and quality metrics.

Required Skills :

- Strong hands-on experience with Python Automation Testing.

- Experience in AI/ML Testing concepts and methodologies.

- Hands-on experience testing LLMs (any model such as OpenAI, Azure OpenAI, Gemini, LLaMA, etc.).

- Experience testing APIs and backend services.

- Good understanding of data validation and model evaluation metrics.

- Familiarity with prompt engineering and prompt validation.

Good to Have :

- Exposure to NLP, machine learning, or deep learning concepts.

- Experience with CI/CD pipelines and Git.

- Knowledge of ethical AI, bias testing, and model governance.