Role Overview :

As a Senior QA AI Testing Engineer, you will be at the forefront of ensuring the quality and reliability of our AI-powered products and services. Your day-to-day will involve designing, developing, and executing comprehensive test strategies for AI models, including Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. You'll collaborate closely with AI/ML engineers, data scientists, and product managers to understand requirements, identify potential risks, and deliver high-quality AI solutions. Your work will directly impact the user experience and business outcomes by ensuring the accuracy, robustness, and ethical considerations of our AI offerings.

Key Responsibilities :

- Develop and execute comprehensive test plans and test cases for AI/ML models, focusing on functionality, performance, scalability, and security, to ensure high-quality AI solutions.

- Design and implement automated testing frameworks and scripts using Azure Automation and other relevant tools, to improve testing efficiency and reduce manual effort.

- Perform rigorous testing of LLMs and RAG systems, including prompt engineering, response evaluation, and bias detection, to guarantee accurate and unbiased AI outputs.

- Collaborate with AI/ML engineers and data scientists to understand model architectures, training data, and evaluation metrics, to create effective testing strategies.

- Identify, document, and track defects using bug tracking systems, and work with development teams to resolve issues promptly, to maintain product quality.

- Contribute to the development of AI testing best practices and standards, to improve the overall quality assurance process.

- Analyze test results and provide detailed reports to stakeholders, to communicate testing progress and identify areas for improvement.

Required Skillset :

- Demonstrated ability to design and implement automated testing frameworks for AI/ML models, including experience with Azure Automation.

- Proven expertise in testing LLMs and RAG systems, including prompt engineering, response evaluation, and bias detection.

- Strong understanding of AI/ML concepts, algorithms, and evaluation metrics.

- Excellent analytical and problem-solving skills, with the ability to identify and resolve complex issues.

- Effective communication and collaboration skills, with the ability to work effectively in a team environment.

- Bachelor's or Master's degree in Computer Science or a related field.

- Ability to work effectively in a hybrid work environment based in Hyderabad.