Job Description : Lead AI Engineer (LLMs)

Location : Baner, Pune (Hybrid)

Experience : 5+ Years

Shift Timing : 2 : 00 PM 11 : 00 PM IST

Notice Period : Immediate to 15 Days

Role Overview :

We are seeking a Lead Generative AI Engineer to architect, build, and deploy production-grade GenAI systems using Large Language Models (LLMs). This role focuses on RAG pipelines, agent-based systems, evaluation frameworks, and scalable GenAI deployment.

You will lead GenAI initiatives end-to-end while mentoring engineers and driving best practices in LLM system design, evaluation, and MLOps.

Key Responsibilities :

GenAI Architecture & System Design :

- Design end-to-end Generative AI architectures (RAG, Agents, multi-step reasoning)

- Translate business use cases into scalable GenAI solutions

- Architect low-latency, reliable LLM inference systems

LLM & GenAI Development :

- Implement and optimize RAG pipelines (retrievers, vector DBs, embeddings)

- Work on prompt engineering, fine-tuning (LoRA/PEFT), and instruction tuning

- Build agent-based workflows (tool calling, memory, orchestration)

- Develop GenAI APIs using FastAPI / Flask

Evaluation & Reliability :

- Design LLM evaluation frameworks beyond accuracy

- Measure hallucination, faithfulness, relevance, latency, cost

- Use metrics such as BLEU, ROUGE, BERTScore, LLM-as-Judge

- Implement offline and online A/B testing for GenAI systems

MLOps & Deployment :

- Build CI/CD pipelines for GenAI workflows

- Handle model versioning, prompt versioning, monitoring, and drift

- Deploy GenAI services using Docker and cloud platforms

- Optimize cost, latency, and scalability of LLM systems

Leadership & Collaboration :

- Lead and mentor GenAI Engineers

- Own GenAI initiatives from POC ? production

- Work closely with Product, Data, and Engineering teams

Required Skills & Qualifications :

- Bachelors/Masters in Computer Science / AI / ML / Data Science

- 5+ years in AI/ML with strong recent focus on Generative AI

- Expert-level Python

- Hands-on experience with LLMs (OpenAI, Anthropic, LLaMA, Mistral, etc.)

- Strong experience in RAG, embeddings, vector databases

- Experience with prompt engineering and fine-tuning techniques

- Strong understanding of LLM evaluation and reliability

- Experience with MLOps, Docker, Git, CI/CD

- Cloud deployment experience (AWS / GCP / Azure)

Preferred Skills :

- Frameworks : LangChain, LlamaIndex, LangGraph

- Vector DBs : Pinecone, FAISS, Weaviate, Chroma

- Monitoring & evaluation : MLflow, Weights & Biases

- Experience building agentic or multi-agent systems

- Knowledge of cost optimization for LLM APIs