Description :

Job Summary :

Insight Global is seeking a highly skilled AI QA Engineer to support one of our medical device clients in India. This role is focused on designing and implementing robust quality assurance frameworks for complex, AI-driven systems, including Retrieval-Augmented Generation (RAG) pipelines, Large Language Model (LLM) applications, vector search platforms, and AI-enabled document processing workflows.

The ideal candidate brings strong hands-on experience testing AI systems across the full lifecyclefrom data ingestion and preprocessing through model inference, retrieval accuracy, and production deploymentwhile collaborating closely with AI Engineers, Data Engineers, and Product teams across cloud platforms such as Snowflake, AWS, and Azure.

Key Responsibilities :

AI & Data Quality Assurance :

- Design and execute comprehensive QA strategies for AI systems, including RAG pipelines, LLM-based applications, vector stores, and retrieval systems.

- Validate data ingestion, preprocessing, embedding generation, vector indexing, and retrieval accuracy.

- Test AI inference outputs for correctness, consistency, performance, and reliability across environments.

- Identify and document common failure modes in LLM systems, including hallucinations, retrieval mismatches, and data drift.

Test Automation & Framework Development :

- Build and maintain automated test frameworks using Python for APIs, AI services, data pipelines, and distributed systems.

- Develop automated validation for data transformations, embeddings, and vector search workflows.

- Implement regression, performance, and integration tests for AI-enabled services.

Cloud & Platform Testing :

- Test cloud-based data pipelines and workflows using Snowflake and at least one major cloud platform (AWS or Azure).

- Validate scalability, reliability, and performance of distributed AI systems.

- Ensure seamless integration across cloud services and data platforms.

OCR & Document Intelligence Validation :

- Validate AI-powered document processing pipelines using tools such as Azure Document Intelligence, AWS Textract, or similar OCR technologies.

- Ensure accuracy, completeness, and consistency of extracted data from structured and unstructured documents.

CI/CD & Quality Engineering Practices :

- Integrate automated AI and data tests into CI/CD pipelines.

- Collaborate with engineering teams to embed quality gates throughout the development lifecycle.

- Contribute to defining evaluation metrics, test coverage standards, and monitoring dashboards for AI systems.

Collaboration & Stakeholder Engagement :

- Partner closely with AI Data Engineers, AI Engineers, Product Managers, and Cloud teams to define quality requirements.

- Clearly communicate test results, risks, and improvement recommendations to technical and non-technical stakeholders.

- Advocate for QA best practices to improve overall quality maturity within the organization.

Required Skills & Qualifications :

Experience :

- 5+ years of experience in QA, quality engineering, or software testing within data-intensive or AI-driven environments.

- Hands-on experience testing AI systems, including RAG pipelines, LLM applications, and vector search platforms.

Technical Skills :

- Strong Python skills for test automation, framework development, and data validation.

- Experience testing cloud data pipelines using Snowflake and AWS or Azure.

- Experience with OCR and document extraction validation (Azure Document Intelligence, AWS Textract, or similar).

- Proficiency in API testing, data validation, and distributed system testing.

AI & Engineering Knowledge :

- Solid understanding of the end-to-end AI lifecycle and common LLM failure modes.

- Knowledge of software engineering best practices, including test design, debugging, CI/CD integration, and evaluation methodologies.

Soft Skills :

- Strong communication and collaboration skills.

- Ability to work cross-functionally in fast-paced, AI-focused product teams.

- Detail-oriented mindset with a passion for quality and reliability.

Nice-to-Have / Preferred Skills :

- Experience in regulated environments (e.g., medical devices, healthcare, or compliance-driven industries).

- Exposure to AI evaluation metrics, monitoring frameworks, or model observability tools.

- Experience testing large-scale, production AI platforms.