Maantic Inc - Senior Python Generative AI Tester - LLM/RAG

Job Summary :

Test generative AI solutions on AWS, validating LLMs, RAG, prompts, and agentic workflows with n8n and Python libraries like deepchecks and LangChain.

Key Requirements :

- Test LLM outputs on AWS Bedrock using boto3.

- Validate fine-tuned LLMs on AWS SageMaker with deepchecks.

- Verify langchain prompts in AWS environments.

- Test RAG pipelines with AWS OpenSearch and langchain.

- Validate AI agents (crewai/autogen) on AWS Lambda.

- Test n8n agentic workflows (e.g., n8n.io/workflows/6270).

- Ensure deployment stability on Amazon ECS/EKS with Docker.

- Monitor performance with Amazon CloudWatch and wandb.

Must-Have Skills :

- 5+ years of QA automation, 2 years- testing GenAI/LLMs using Python

- Expertise in AWS Bedrock, SageMaker, Lambda, boto3.

- Proficiency in deepchecks, langchain, crewai, autogen, wandb.

- Experience testing n8n workflows, RAG, prompts.

Preferred Skills :

- AWS certification (Machine Learning/Solutions Architect).

- Familiarity with llama-index, n8n templates