Description :

Experience : 3 to 5 Years

Job Type : Full-Time

Location : Sector - 142, Noida (Hybrid Working Model)

Position Summary :

We are looking for an innovative AI Quality Engineer to join our team. In this role, you will go beyond traditional software testing to ensure the accuracy, safety, and reliability of our Generative AI and LLM-based applications. You will combine your solid foundation in QA automation (Python/TS/Playwright) and API testing with emerging skills in Prompt Engineering and LLM Evaluation.

If you are a QA professional who is excited about how AI works, loves to "break" models, and wants to be at the forefront of AI testing, this role is for you.

Key Responsibilities :

AI & LLM Evaluation :

- Design and execute evaluation strategies for Large Language Models (LLMs), focusing on metrics like Accuracy, Hallucination, Tone, and Relevance.

- Perform Prompt Engineering and testing to optimize model outputs and ensure alignment with business requirements.

- Identify and document AI-specific defects.

Automation & Scripting :

- Develop and maintain automated test scripts using Python or JavaScript/TypeScript to validate AI pipelines.

- Utilize tools like Playwright or Selenium for UI testing of AI-integrated web applications.

API & Integration Testing :

- Perform robust API testing using Postman or programmatic methods (Rest Assured/Python Requests) to validate AI model endpoints.

- Ensure efficient token usage, latency standards, and proper error handling in API responses.

CI/CD & DevOps Integration :

- Integrate automated AI evaluation suites into Jenkins pipelines for continuous testing.

- Analyze logs and metrics using tools like CloudWatch or Datadog to identify performance bottlenecks

Skills :

- AI/LLM Exposure : Practical understanding of Generative AI concepts, Prompt Engineering, and LLM Evaluation techniques (RAG, Fine-tuning concepts).

- Programming : Hands-on experience with Python (highly preferred for AI tasks) or JavaScript/TypeScript.

- API Testing : Proficient in Postman (creating collections, assertions) and testing RESTful APIs.

- Core QA Tools : Experience with Git for version control, Jira for defect tracking, and Jenkins for CI/CD.

- Data Handling : Comfortable working with JSON/JSONL formats and handling data for test scenarios.

- Soft Skills : Strong analytical and problem-solving abilities, effective communication, an ownership mindset, and a willingness to learn and adapt.

Qualifications :

- Bachelors degree in Computer Science, Information Technology, or a related field with 70% or equivalent.

Desired Skills :

- Familiarity with AI Evaluation frameworks like DeepEval, Ragas.

- Experience with cloud platforms (AWS : EC2, Lambda, S3) for setting up test environments.

- Knowledge of Docker basics for containerized testing.

- Basic understanding of Performance Testing tools (JMeter/k6).

Benefits :

- Flexible Working Hours.

- Hybrid Working Style.

- Personal Accidental Insurance.

- Health Insurance to Self, Spouse and two kids.

- 5 days working week.