LLM Specialist - NLP Pipelines

Description :

Key Responsibilities :

LLM Development & Optimization :

- Fine-tune and optimize large language models (GPT, Llama, Mistral, Falcon, etc.)

- Customize LLMs for domain-specific tasks : conversation, summarization, classification, content generation.

- Work on model evaluation, prompt design, and reinforcement feedback loops.

NLP Engineering :

- Build NLP pipelines for text processing, entity recognition, sentiment analysis, and retrieval augmented generation (RAG).

- Implement embeddings, vector search, and semantic similarity models.

Prompt Engineering & Model Interaction :

- Design effective prompts, system instructions, and multi-step workflows.

- Create reusable prompt templates for different use cases.

- Test, validate, and iterate prompts for accuracy and contextual alignment.

RAG Systems & Knowledge Integration :

- Develop RAG pipelines using vector databases (Pinecone, Chroma, Weaviate, FAISS).

- Implement document ingestion, chunking, embeddings, and retrieval workflows.

- Enhance AI responses using structured + unstructured knowledge sources.

AI Integration & Deployment :

- Integrate LLMs into backend systems, APIs, chatbots, and enterprise applications.

- Work with frameworks like LangChain, LlamaIndex, Haystack, or custom pipelines.

- Implement testing, monitoring, and performance optimization for deployed models.

Safety, Ethics & Compliance :

- Apply responsible AI practices, bias detection, and output safety checks.

- Ensure models comply with data privacy, PII handling, and compliance standards.

- Conduct model red-teaming and robustness evaluations.

Collaboration, Documentation & Research :

- Collaborate with product, engineering, and research teams to define AI features.

- Create documentation, model versions, datasets, and best practices.

- Stay updated with emerging LLM architectures, training techniques, and open-source tools.

Required Skills & Qualifications :

Technical Skills :

- Strong understanding of NLP, Transformers, embeddings, and LLM architectures.

- Experience with Python and libraries such as Hugging Face, Transformers, LangChain, Pydantic.

- Knowledge of vector databases (Pinecone, Chroma, FAISS, Weaviate).

- Ability to fine-tune and deploy models on GPU/Cloud setups.

- Familiar with ML frameworks : PyTorch, TensorFlow.

- Experience with API-based LLMs (OpenAI, Anthropic, Google, etc.)

- Understanding of evaluation metrics (BLEU, ROUGE, perplexity, accuracy).

Soft Skills :

- Strong analytical and problem-solving ability.

- Clear communication and documentation skills.

- Ability to work cross-functionally and handle fast-paced environments.

- Creative mindset for building AI-driven products.

Preferred Qualifications :

- Exposure to GPU environments, model quantization, and optimization techniques.

- Understanding of data engineering workflows.

- Prior work on chatbots, summarization systems, or enterprise AI tools