HamburgerMenu
hirist

Senior Manager - LLM Training

Gallop Research
Multiple Locations
8 - 12 Years

Posted on: 02/12/2025

Job Description

Job Description :


The ideal candidate will bring a blend of strong analytical skills, hands-on experience with AI model training, and the ability to collaborate effectively with cross-functional teams including Operations, Delivery, and AI Engineering.

Key Responsibilities :


1. Dataset Development & Training Preparation :


- Curate, prepare, and structure comprehensive training datasets for various Automation Operations (AO) use cases.

- Ensure datasets are robust, diverse, and aligned with the specific requirements of business processes and client needs.

- Develop data pipelines and documentation that support scalable model training workflows.

2. Annotation & Edge Case Identification :


- Identify, annotate, and document complex scenarios, exceptions, and edge cases to improve model behavior and adaptability.

- Create guidelines and annotation standards to maintain consistency and quality across all training data.

3. Model Evaluation & Supervision :


- Continuously monitor and evaluate AI model outputs to ensure accuracy, contextual relevance, and adherence to expected performance benchmarks.

- Provide actionable insights to the AI engineering team to support model refinement and optimization.

4. Data Gap Analysis & Enhancement :


- Conduct regular analysis of existing training datasets to identify gaps, inconsistencies, or improvement areas.

- Recommend data augmentation strategies, new training sources, and enhancements to boost model performance.

5. Cross-Functional Collaboration :


- Partner closely with Operations, Delivery, and AI teams to gather data feedback, understand business requirements, and validate model performance.

- Translate operational insights into data improvements and model training strategies.

6. Compliance & Quality Assurance :


- Ensure all data handling, labeling, storage, and usage practices adhere to internal quality standards and external compliance regulations.

- Maintain documentation, audit trails, and quality checks for all data training activities.

Qualifications We Seek :


Minimum Qualifications :


- Strong analytical skills with a proven ability to make data-driven decisions.

- Excellent communication and collaboration skills for effective cross-functional coordination.

- Ability to interpret business needs and translate them into technical data requirements.

Preferred Qualifications / Skills :


- Prior experience in training large language models or working with AI/ML technologies.


- Demonstrated ability to manage complex, multi-stakeholder projects and lead teams successfully.

- Familiarity with regulatory and compliance standards in data management (e.g., GDPR, data privacy protocols).

- Background in data science, machine learning, or related technical domains is a strong advantage.

info-icon

Did you find something suspicious?