Real Page - Quality Assurance Engineer III - Artificial Intelligence

Job Title : QA Engineer III-AI

Experience : 6 Years to 9 Years

Location : Hyderabad

Notice Period : Immediate to 30 Days

Key Skills : Java, Selenium, Automation, AI, Rest assured

Role Summary :

Key Responsibilities :

AI & Compliance Quality Ownership :

- Own end-to-end quality strategy for the AI agent models/ Agentic AI platform, covering UI, backend services, and AI model outputs.

1. Decision correctness

2. Evaluation status

3. Confidence thresholds

4. Explainability / traceability of responses

- Define test approaches for nondeterministic AI behavior (golden datasets, tolerance-based assertions, confidence validation).

- Partner with Data Science teams to validate model inference results, regression drift, and version-to-version consistency.

Automation & Framework Development :

- Design, develop, and maintain robust Selenium WebDriver automation frameworks (Java preferred).

1. Web UI workflows

2. Compliance dashboards

3. Agent evaluation flows

4. Admin & configuration screens

- Ensure automation follows industry best practices : Page Object Model (POM), reusable utilities, proper wait strategies, and stable selectors :

1. Page Object Model (POM)

2. Datadriven and keyworddriven testing

3. Reusable libraries and utilities

4. Robust synchronization and wait strategies to reduce flakiness

- Integrate automation suites into CI/CD pipelines using :

1. Maven

2. Jenkins

3. GitHub

CI/CD & Quality Engineering Practices :

- Own Jenkins pipelines for automated test execution (smoke, regression, AI evaluation suites).

1. Pull request validation

2. Nightly regression

3. Release readiness checks

- Analyze test failures, improve flakiness, and ensure high signations ratio in automated results.

- Produce actionable test metrics (coverage, defect leakage, automation ROI).

Functional, Integration & Regression Testing :

- Drive functional, integration, and regression testing across multiple services and AI agents.

- Validate :

1. API integrations (REST)

2. Data flows across agent pipelines

3. Evaluation status lifecycle (queued processed evaluated completed).

- Perform exploratory testing on new AI capabilities and edge cases.

Leadership & Collaboration :

- Mentor junior engineers on automation, testing strategies, and AI validation approaches.

- Participate in requirement reviews and proactively identify quality risks early in the SDLC.

Required Skills & Experience :

Core Experience :

- Strong hands-on experience in test automation using Selenium WebDriver.

- Proficiency in Java (or similar OOP language) for automation.

- Solid experience working with :

1. Maven (build & dependency management)

2. Jenkins (CI pipelines)

3. GitHub (version control, PR workflows)

AI / DataDriven Systems (MustHave) :

- Experience testing AI/MLdriven applications, analytics platforms, or rule/decision engines.

- Understanding of :

1. Model inference pipelines

2. Confidence scores

3. Evaluation metrics

4. Handling nondeterministic outputs

- Ability to design validation strategies for AI agents, beyond simple pass/fail testing.

Testing & Tooling :

1. API testing (REST)

2. TestNG / JUnit

- Agile/Scrum methodologies

- Familiarity with test management and defect tracking tools (Azure DevOps TFS, etc.).

- Experience validating complex workflows across UI + API layers.

Nice to Have :

- Experience in compliance, audit, or regulatory systems.

- Exposure to performance testing (e.g., JMeter) for dashboards or AI pipelines.

- Knowledge of logging, monitoring, and observability tools for troubleshooting AI behavior.

- Experience validating data privacy and security aspects of AI systems.