Posted on: 08/10/2025
About the Role :
We are looking for a highly skilled SDET Manager to spearhead the quality of AI application development at HighLevel.
This role is very critical in ensuring the relevancy, consistency, reliability, contextual recall and fairness of our AI platform.
Ideal candidates will have a combination of technical depth in AI/ML, Agentic AI along with deep expertise in test automation covering model validation, agentic workflows, and quality leadership.
Requirements :
- Overall 8 to12 years of experience in which minimum of 1 year in testing AI products.
- Good experience in Prompt Engineering, testing AI applications like conversations AI, voice AI, workflow AI.
- Architect and scale test automation frameworks for APIs, micro-services, ML pipelines, and agentic AI workflows.
- Experience in testing RAG based applications using frameworks like TruLens, RAGAs.
- Leverage AI to improve coverage and efficiency across testing lifecycle.
- Hands-on experience testing agentic AI frameworks (e.g., Agentic Studio, LangChain, AutoGen, crewAI).
- Strong programming skills in Python and Java or TypeScript/JavaScript.
- Proven expertise in test automation frameworks (Selenium, Playwright, Cypress etc.).
- Experience with cloud platforms (AWS, GCP) and CI/CD pipelines.
Responsibilities :
- Review testing strategies for AI application and fine tune strategy to minimize issues in production.
- Establish evaluation pipelines for detecting hallucination, context retention, latency in multi model environments.
- Generate right data set to simulate customer usage scenarios.
- Test for data, model and code issues.
- Define relevant KPIs for AI application testing.
- Non Functional testing Performance, Scalability, Reliability, Security.
- Implement A/B testing at appropriate levels to identify customer impact.
- Run down defect triaging for all customer issues to identify root cause and expand test coverage to mitigate gaps.
- Ensure quality deliverables from AI teams.
- Hire top talent and mentor SDET team on AI testing skills.
EEO Statement :
The company is an Equal Opportunity Employer.
As an employer subject to affirmative action regulations, we invite you to voluntarily provide the following demographic information.
This information is used solely for compliance with government record-keeping, reporting, and other legal requirements.
Providing this information is voluntary and refusal to do so will not affect your application status.
This data will be kept separate from your application and will not be used in the hiring decision.
Did you find something suspicious?