Senior Prompt Engineer - LLM

Description :

Role : Senior Prompt Engineer

Location : Pune (only local candidates)/ WFO

Interview : Virtual / Immediate joiners

Location : Pune (Only Locals)

Experience : 5+ years

We are seeking a Senior Prompt Engineer with 5+ years of experience in AI and LLM-driven application development. This role focuses on designing, testing, and optimizing enterprise-grade prompts for GenAI, RAG, and agentic systems. The ideal candidate will have hands-on experience with LLM APIs, vector databases, and multi-agent orchestration frameworks.

Strong expertise in prompt engineering, context design, and output control is essential.

Experience with Azure-based AI services, governance, security, and compliance is highly preferred. The role requires balancing accuracy, cost, performance and scalability in production AI systems.

In this Role, Your Responsibilities Will Be :

- Design, develop, test, and optimize advanced prompt strategies for LLMs (Open AI GPT, Claude, LLaMA, Mistral, Gemini) across enterprise use cases.

- Work closely with functional teams to engineer, test, and optimize prompts that translate business requirements into effective LLM-driven application behavior.

- Engineer multi-step prompt chains, system prompts, role-based prompts and meta-prompts for complex reasoning, planning, and decision-making tasks.

- Develop reusable prompt templates, prompt COT,ToT, Meta Prompting, Conversational Few-Shot and structured flows using frameworks like LangChain, LangGraph, and PromptLayer.

- Develop and maintain prompt orchestration pipelines for RAG, agentic AI, and multi-agent systems using LangChain, CrewAI, and MCP-based architectures.

- Engineer RAG-optimized prompts integrated with vector databases (PostreSQL, Redis, FAISS, Pinecone, Chroma) and re-ranking pipelines.

- Support agentic and multi-agent frameworks.

- Implement function-calling and schema-enforced prompting (JSON/YAML, tool/function APIs).

- Design hallucination-resistant, jailbreak-safe prompts with built-in guardrails and filters.

- Tune LLM parameters (temperature, top_p, penalties, max tokens) for accuracycostlatency balance.

- Develop reusable prompt templates, chains, and graphs (LangChain, LangGraph, PromptLayer).

- Perform A/B testing, regression testing, and prompt versioning for continuous optimization.

- Optimize token usage and context-window management to reduce inference cost.

- Align prompts with fine-tuning / LoRA / RLHF and embedding strategies.

- Implement PII/PHI masking, RBAC-aware prompts, and compliance-safe instructions (GDPR, HIPAA, ISO 27001).

- Monitor prompt drift, response instability, and output quality in production.

- Define and track prompt KPIs (quality, latency, cost, consistency).

- Ensure compliance with internal data governance and security policies.

Qualifications :

- Bachelor's or Master's degree in Computer Science, Data Science, Statistics, Mathematics, or a related field over 5+ years.

- Relevant Certifications/trainings on Gen AI, Prompt Engineering.

- Strong hands-on experience with LLM prompt engineering in production

- Experience integrating Prompts with RAG systems with vector databases.

- Hands-on Prompt Engineer with agentic frameworks and finetuning.

- Familiar with fine-tuning methods.

- Strong Python + API integration (FastAPI/Flask)

- Knowledge of LLM security, hallucination control, bias mitigation

- Experience/Knowledge in LLMOps/MLOps, MLflow, LangSmith, W&B, PromptLayer

- Understanding of PII protection, RBAC, compliance (GDPR/HIPAA/ISO)

- Excellent problem-solving, communication, and team collaboration skills.

Preferred Qualifications :

- Relevant certifications (Azure AI, Generative AI, LLMOps, Prompt Engineering)

- Experience with enterprise copilots, Chatbots, and agentic AI systems

- Hands-on with PromptOps / LLMOps governance and lifecycle management

- Expertise in semantic prompting, RAG, and context engineering

- Experience with LLM evaluation tools (RAGAS, TruLens, DeepEval, PromptLayer)

- Strong knowledge of LLM safety, guardrails and hallucination mitigation

- Familiar with hybrid model routing, fallback strategies and AutoPrompting

- Exposure to regulated domains and large-scale AI systems

- Certified in Azure AI Fundamentals (AI-900), Azure AI Engineer Associate (AI-102)

- Experience with big data technologies