Role Overview :

We are seeking a seasoned AI Testing Specialist to join our high-impact engineering team in Hyderabad. In this role, you will be at the forefront of validating complex AI systems, ensuring that our next-generation LLM-driven products are robust, reliable, and ethically sound.

You will collaborate closely with data scientists, product managers, and software architects to bridge the gap between model development and production readiness.

By designing sophisticated testing frameworks for Agentic AI and RAG architectures, you will directly influence the quality of user experiences, ensuring our AI solutions deliver accurate, safe, and high-performance outcomes for our global enterprise clients.

Key Responsibilities :

- Architect and execute comprehensive AI quality assurance strategies to validate the performance, accuracy, and safety of large-scale LLM deployments.

- Design and implement rigorous prompt testing protocols to evaluate model responses against edge cases, bias, and hallucination risks, ensuring high-fidelity outputs.

- Develop automated test suites for Agentic AI workflows to verify autonomous decision-making processes and multi-step task execution.

- Lead the end-to-end validation of RAG (Retrieval-Augmented Generation) pipelines, ensuring data retrieval precision and contextual relevance for end-users.

- Integrate advanced AI testing methodologies into existing CI/CD pipelines to maintain high-velocity regression testing without compromising model integrity.

- Partner with cross-functional stakeholders to define success metrics and quality benchmarks that align with business objectives and regulatory compliance standards.

Required Skillset :

- Demonstrated expertise in AI testing and quality assurance, with a deep understanding of the nuances involved in evaluating non-deterministic model outputs.

- Proven ability to design and maintain complex test automation frameworks that handle the unique challenges of LLM-based applications.

- Strong analytical mindset to interpret complex data, identify failure patterns in AI models, and communicate actionable insights to technical and non-technical stakeholders.

- Exceptional interpersonal skills, enabling effective collaboration within a hybrid work environment in Hyderabad, ensuring seamless alignment between engineering and product teams.

- A Bachelor's or Master's degree in Computer Science, Data Science, or a related quantitative field, complemented by a track record of delivering high-quality software solutions.

- High degree of adaptability to rapidly evolving AI technologies, with the ability to pivot strategies as new model architectures and testing standards emerge.