On Device Machine Learning Engineer - LLM Fine Tuning

Job Description :

On-Device ML Engineer / LLM Fine-tuning Specialist

Key Responsibilities :

- Fine-tune and optimize small language models for domain-specific AI applications

- Build and manage multiple LoRA adapters for different AI health agents

- Implement dynamic adapter switching based on user context and application workflows

- Perform model quantization using INT4/INT8, GGUF, GPTQ, and related techniques

- Deploy optimized models on-device using frameworks such as CoreML, TFLite, llama.cpp, or MLC-LLM

- Curate training datasets and prepare instruction-tuning datasets for model enhancement

- Optimize model performance for low-latency and resource-efficient edge deployments

- Collaborate with product, AI, and mobile engineering teams to integrate AI capabilities into applications

- Evaluate model performance, inference efficiency, and deployment scalability

- Support experimentation and innovation around LLM optimization and alignment techniques

Required Skills & Experience :

- Strong hands-on experience with LoRA and QLoRA fine-tuning techniques

- Experience fine-tuning small LLMs (0.5B3B parameter range) such as :

i. Phi-3

ii. Gemma 2

iii. Llama 3.2

- Strong understanding of model quantization techniques including :

i. INT4 / INT8

ii. GGUF

iii. GPTQ

- Experience with on-device deployment using :

i. CoreML

ii. TFLite

iii. llama.cpp

iv. MLC-LLM

- Experience in training data curation and instruction dataset preparation

- Expertise in Multi-LoRA adapter management and dynamic adapter switching

- Strong Python and ML engineering skills

- Excellent analytical and problem-solving abilities

Nice to Have :

- Experience fine-tuning models on health, biomedical, or wellness datasets

- Familiarity with RLHF or DPO alignment techniques

- Android or iOS ML integration experience

- Exposure to edge AI optimization and mobile inference acceleration