Job Title: QE Lead - AI & Analytics

Location: Pune / Nashik, India

Experience Required: 6+ Years

Employment Type: Full-Time

Job Summary :

We are seeking an experienced QE Lead - AI & Analytics to drive quality engineering initiatives for AI-powered applications, analytics platforms, and data-driven solutions.

The ideal candidate will be responsible for defining quality strategies, implementing test automation frameworks, and ensuring the reliability, accuracy, and performance of AI systems, including LLMs, RAG pipelines, and machine learning models.

Key Responsibilities :

- Define and lead the Quality Engineering strategy for AI, ML, and Analytics solutions.

- Design and execute testing approaches for AI systems, covering functional, non-functional, performance, reliability, and security aspects.

- Validate AI applications, including LLMs, RAG pipelines, predictive models, and analytics workflows.

- Establish AI evaluation frameworks using metrics such as accuracy, relevance, hallucination detection, drift monitoring, and robustness testing.

- Develop and maintain automation frameworks for APIs, data pipelines, analytics workflows, and distributed systems.

- Collaborate with Product Managers, Data Scientists, AI Engineers, and Development teams to ensure quality throughout the software lifecycle.

- Implement observability, monitoring, and quality measurement practices for AI systems in production.

- Mentor and guide Quality Engineers while promoting best practices across teams.

Mandatory Skills :

- 8+ years of experience in Quality Engineering, QA, or Software Testing.

- Hands-on experience testing AI/ML applications, Generative AI solutions, or LLM-based products.

- Experience working with Retrieval-Augmented Generation (RAG) systems.

- Strong experience in API Testing and Test Automation.

- Experience testing Data Pipelines, ETL processes, and Analytics platforms.

- Knowledge of AI evaluation metrics such as Accuracy, Relevance, Hallucination Detection, Model Drift, and Reliability.

- Experience with automation tools such as Selenium, Playwright, Cypress, Rest Assured, PyTest, or similar frameworks.

- Strong SQL and data validation skills.

- Experience working in Agile/Scrum environments.

- Team leadership and mentoring experience.

Preferred Skills :

- Programming experience in Python or Java.

- Experience with LangChain, LlamaIndex, Vector Databases, or MLOps tools.

- Exposure to cloud platforms such as AWS, Azure, or GCP.

- Experience with observability and monitoring tools such as Grafana, Kibana, Datadog, or Prometheus.

- Familiarity with CI/CD tools including Jenkins, GitHub Actions, or Azure DevOps.

Desired Candidate Profile :

- Strong analytical and problem-solving skills.

- Passion for AI technologies and continuous learning.

- Excellent communication and stakeholder management abilities.

- Ability to influence quality practices across teams.

- Proven track record of improving product quality and reliability.