Role Summary :

We are looking for a Senior LLM Engineer with strong hands-on experience in agentic AI systems. The role focuses on designing and implementing multi-agent workflows, building LLM evaluation pipelines, and optimizing LLM behavior using prompt engineering. This is a core IC role requiring ownership of production-grade LLM systems.

1. Agentic Workflow Design (Primary - Core) :

- Design and implement agent-based workflows using LangChain and LangGraph.

- Build stateful, cyclic, and conditional agent graphs for autonomous decision-making.

- Manage agent memory, tool usage, retries, and execution control.

- Ensure reliability and scalability of agent pipelines in production.

2. LLM Evaluation & Quality Control :

- Implement LLM-as-a-Judge evaluation frameworks.

- Design evaluation prompts where stronger models validate :

a. Accuracy

b. Safety

c. Instruction adherence

d. Automate response scoring and regression testing for prompt and model changes.

3. Prompt Engineering & Model Usage :

- Perform advanced prompt engineering and prompt tuning to align outputs with product requirements.

- Manage and optimize calls to multiple LLM providers (OpenAI, Anthropic, Open-source).

- Optimize for latency, cost, and response quality.

4. Data Integrity & Validation :

- Enforce strict schema validation on LLM inputs and outputs.

- Analyze LLM responses to ensure compliance with business and technical requirements.

- Implement guardrails to reduce hallucinations and invalid outputs.

5. Ownership & Delivery :

- Operate as an independent contributor, owning features end-to-end.

- Collaborate with product and engineering teams on AI system design.

- Contribute to architectural decisions related to agentic systems.

Required Skills & Experience (Aligned) :

- 7+ years of strong Python development experience

- Hands-on experience building LLM-based applications

- LangChain (mandatory)

- LangGraph (highly preferred)

- Experience with LLM evaluation frameworks (LLM-as-a-Judge or equivalent)

- Strong prompt engineering and prompt tuning experience

Good to Have (Kept Optional, Not Core) :

- Exposure to PEFT / LoRA / adapters

- Experience with open-source LLMs

- Familiarity with RAG pipelines and vector database