Ladera Technology - Software Development Engineer - Test - AI/LLM Automation Testing

Description : AI/LLM Test Automation Engineer (SDET)

Location : Bangalore (Hybrid preferred)

Experience : 5-8 Years

Job Summary :

We are seeking a Senior Test Automation Engineer (SDET) with expertise in AI/LLM testing and Azure DevOps CI/CD to build robust automation frameworks for cutting-edge AI applications.

The role combines deep programming skills (Java/Python), modern DevOps practices, and specialized LLM testing to ensure high-quality AI product delivery.

Key Responsibilities :

- Design, develop, and maintain automation frameworks using Java/Python for web, mobile, API, and backend testing.

- Create and manage YAML-based CI/CD pipelines in Azure DevOps for end-to-end testing workflows.

- Perform AI/LLM testing including prompt validation, content generation evaluation, model behavior analysis, and bias detection.

- Write and maintain BDD Cucumber feature files integrated with automation suites.

- Execute manual + automated testing across diverse application layers with focus on edge cases.

- Implement Git branching strategies, code reviews, and repository best practices.

- Track defects and manage test lifecycle using ServiceNow or similar tools.

- Conduct root-cause analysis, troubleshoot complex issues, and drive continuous quality improvements.

Mandatory Skills & Experience :

- 5+ years SDET/Automation experience.

- Java/Python scripting for test frameworks (Selenium, REST Assured, Playwright).

- Azure DevOps YAML pipelines (CI/CD end-to-end).

- AI/LLM testing (prompt engineering, model validation, RAG testing).

- Cucumber BDD (Gherkin feature files + step definitions).

- Git (branching, PRs, GitFlow).

- ServiceNow/Jira defect tracking.

- Manual + Automation testing (web/mobile/API/backend).

Technical Stack :

Programming :

- Java, Python, JavaScript

CI/CD :

- Azure DevOps, YAML Pipelines

Testing :

- Selenium, Playwright, REST Assured, Postman

BDD :

- Cucumber (Gherkin), JBehave

AI/ML :

- Prompt validation, LLM APIs (OpenAI, LangChain)

Version Control :

- Git, GitHub/GitLab

Defect Tracking :

- ServiceNow, Jira, Azure Boards

Preferred Qualifications :

- Exposure to AI testing frameworks (LangSmith, Promptfoo).

- Experience with containerization (Docker) and Kubernetes.

- Knowledge of performance testing for AI workloads.

- AWS/GCP cloud testing experience.

- ISTQB or relevant QA certifications.

What We Offer :

- Work on next-gen AI/LLM products with global impact.

- Modern tech stack with Azure-native DevOps.

- Flexible hybrid/remote work model.

- Continuous learning opportunities in AI testing.