DemandBase - Senior Machine Learning Engineer

Description :

About the Role :

We are looking for a Senior Machine Learning Engineer to help architect and build next-generation Agentic AI systems at Demandbase. This role focuses on multi-agent orchestration, LLM-powered reasoning systems, evaluation frameworks, guardrails, and scalable GenAI architectures.

You will work at the intersection of advanced data science, generative AI research, and production-grade ML systems, shaping how intelligent agents operate reliably, safely, and effectively in enterprise environments.

This is not a platform infrastructure role - it is a deep AI systems engineering role centered around agent architecture, model evaluation, reasoning systems, and applied ML innovation.

Key Responsibilities :

Agentic AI & Multi-Agent Architecture :

- Design and implement multi-agent systems for complex enterprise workflows.

- Build agent orchestration frameworks (planner-executor, tool-using agents, retrieval-augmented agents, self-reflective agents).

- Develop architectures for reasoning loops, memory systems, tool integration, and contextual grounding.

- Design guardrails for hallucination mitigation, tool misuse prevention, and safe execution.

- Implement feedback-driven refinement loops and self-correction strategies.

GenAI Systems, Evals & Guardrails :

- Design and operationalize LLM evaluation frameworks (automated evals, LLM-as-judge, human-in-the-loop, adversarial testing).

- Build robust prompt engineering and prompt versioning strategies.

- Develop safety guardrails including content filtering, policy enforcement, and bias monitoring.

- Implement quality metrics for :

1. Factual accuracy

2. Groundedness

3. Latency and cost efficiency

4. Agent reliability

- Create structured evaluation pipelines to continuously improve agent performance.

Advanced Data Science & NLP :

- Apply advanced NLP techniques (transformers, embeddings, fine-tuning, RAG pipelines).

- Work deeply with unstructured and semi-structured data.

- Develop model experimentation frameworks for prompt optimization, fine-tuning, and retrieval strategies.

- Optimize data pipelines using Python, Pandas, Spark, and vector databases.

- Collaborate with data scientists to convert research prototypes into scalable AI systems.

AI System Design & Architecture :

- Architect modular, extensible AI systems for long-term maintainability.

- Design retrieval-augmented generation (RAG) systems with advanced chunking, embedding strategies, and re-ranking.

- Build memory architectures (short-term, long-term, vector-based).

- Optimize inference pipelines for performance, cost, and reliability.

- Define reusable patterns for enterprise-grade AI systems.

Operational Excellence for AI Systems :

- Implement evaluation-driven CI/CD for GenAI systems.

- Establish monitoring for :

1. Model drift

2. Agent failure modes

3. Tool misuse

4. Hallucination frequency

- Maintain reproducibility through experiment tracking and versioning.

- Ensure ethical AI practices and compliance with enterprise standards.

Technical Leadership & Mentorship :

- Define best practices for agentic AI architecture and LLM system design.

- Mentor engineers and data scientists in advanced GenAI methodologies.

- Drive internal thought leadership in Agentic AI and applied GenAI research.

- Contribute to building a high-performing AI engineering culture.

Basic Qualifications :

- 8+ years of experience in Machine Learning, AI Engineering, or Applied Data Science.

- Strong expertise in Generative AI systems and LLM architectures.

- Experience designing multi-agent or tool-using AI systems.

- Deep proficiency in Python and ML ecosystems (NumPy, Pandas, PyTorch/TensorFlow).

- Hands-on experience with :

1. RAG systems

2. Prompt optimization

3. Evaluation frameworks

4. Embedding models & vector databases

- Strong understanding of transformers, embeddings, and fine-tuning methods.

- Experience building production-grade AI systems from research to deployment.

- Strong system design and architectural problem-solving skills.

Preferred Qualifications :

- MS or PhD in Computer Science, AI, ML, or related fields.

- Experience with LLM orchestration frameworks (LangChain, LlamaIndex, custom agent frameworks).

- Experience with automated eval frameworks and benchmarking strategies.

- Familiarity with reinforcement learning, RLHF, or agent self-improvement loops.

- Experience with vector databases and retrieval systems.

- Published research or open-source contributions in AI/ML.

- Strong understanding of AI safety and responsible AI practices.