About the job :

Job Title : Senior QA Engineer Web, Mobile & Conversational AI Testing

Location : Hybrid - Bangalore

Experience Level : 4+ years

Employment Type : Full Time

About the Role :

We are looking for a highly experienced Senior QA Engineer to lead and execute quality assurance efforts across our web, mobile, and AI-driven conversational products.

You will play a critical role in ensuring our applications deliver exceptional user experiences, high performance, and reliabilityespecially in LLM-based (Large Language Model) conversational systems.

Key Responsibilities :

Test Planning & Strategy :

- Develop and maintain comprehensive test plans and test cases for web, mobile, and conversational AI applications.

- Define QA strategy for end-to-end testing of features including functional, UI/UX, performance, and regression testing.

Execution & Automation :

- Perform manual and automated testing for mobile (iOS/Android) and web platforms. [Selenium, Pytest, Python Scripts]

- Design and maintain automated test scripts using tools like Selenium, Appium, or Playwright.

- Conduct conversational flow and response accuracy testing for LLM-based chat interfaces.

LLM & Conversational App Testing :

- Evaluate dialogue quality, prompt-response relevance, tone, and fallback behaviors in GPT-like systems.

- Test for hallucinations, safety, toxicity, and coherence in AI outputs.

- Validate integrations with RAG systems, vector stores, and API workflows.

Cross-Functional Collaboration :

- Collaborate with product managers, developers, and AI engineers to ensure feature clarity and testability.

- Provide feedback early in the development lifecycle to reduce bugs and enhance quality.

Monitoring & Reporting :

- Track and report bugs using tools like JIRA or Linear.

- Analyze and report on test results, coverage, and quality trends.

Required Qualifications :

- 4+ years of experience in QA for web and mobile applications.

- Hands-on experience with both manual and automation testing tools and frameworks.

- Strong knowledge of QA methodologies, testing types, and software development lifecycle.

- Automation testing using Pytests.

- Experience testing LLM-based or NLP-powered applications (ChatGPT, Rasa or custom LLMs).

- Proficient in writing SQL queries and working with Postman for API testing.

- Familiarity with vector databases and conversational testing tools (Langfuse, Phoenix, etc.) is a plus.

Preferred Skills :

- Experience in testing AI-driven personalization, intent recognition, or multi-turn conversation logic.

- Exposure to prompt engineering and evaluation frameworks for LLM outputs.

- Familiarity with CI/CD pipelines and version control systems like Git.

- Ability to build QA dashboards and automated quality reports