Allstate - Data Scientist/Senior Consultant II - Python

Data Scientist - Senior Consultant II

About the Job:

At Allstate, great things happen when our people work together to protect families and their belongings from life's uncertainties. For more than 90 years, our innovative drive has kept us a step ahead of our customers' evolving needs.

We are seeking an experienced Data Scientist - Senior Consultant II to design, develop and productionise AI/ML and Generative AI solutions for the Scholar product's Insight Studio. In this role, you will transform large-scale customer conversations into actionable business insights by leveraging advanced data science, machine learning, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) and AI agent technologies.

The ideal candidate will have strong expertise in Python, machine learning, data science and Generative AI, with the ability to build, deploy and optimise scalable AI solutions in production environments.

Key Responsibilities:

- Translate complex business problems into end-to-end data science and AI/ML solutions aligned with business objectives.

- Design, develop and deploy machine learning and Generative AI solutions for conversational understanding, summarisation, classification and insight generation.

- Build and optimise LLM-powered applications using prompt engineering, model orchestration and foundation models.

- Design and implement RAG pipelines integrating structured and unstructured data sources to generate context-aware and reliable insights.

- Develop embedding-based solutions and vector search systems for semantic search and information retrieval.

- Build AI agent workflows using frameworks such as LangChain, LangGraph, LlamaIndex or similar.

- Perform data exploration, feature engineering, model development, evaluation and optimisation.

- Integrate foundation models using AWS Bedrock and leverage AWS SageMaker for machine learning workflows.

- Develop scalable data pipelines and distributed processing systems to support real-time and batch AI workloads.

- Implement model evaluation frameworks, LLM guardrails and responsible AI practices.

- Apply MLOps and LLMOps practices, including CI/CD, model versioning, automated evaluation, deployment and monitoring.

- Monitor and optimise production AI/ML solutions using Datadog, AWS CloudWatch or similar observability tools.

- Analyse model outputs, communicate findings and continuously improve model performance and business impact.

- Collaborate with product, business and engineering teams to translate requirements into effective AI-driven solutions.

Requirements:

- 7 - 10 years of experience in Data Science, Machine Learning, AI/ML Engineering or related fields.

- Strong proficiency in Python and hands-on experience developing production-grade AI/ML applications.

- Strong understanding of statistical modelling, machine learning algorithms, predictive analytics and data analysis.

- Practical experience developing Generative AI applications using LLMs, prompt engineering and model orchestration.

- Hands-on experience with RAG architectures, embeddings, vector databases and semantic search.

- Experience building AI agents and multi-step workflows using LangChain, LangGraph, LlamaIndex or similar frameworks.

- Experience with AWS Bedrock, foundation model integration and familiarity with AWS SageMaker.

- Strong understanding of data pipelines, distributed processing and scalable data architectures.

- Experience with model evaluation, performance optimisation, LLM evaluation and responsible AI practices.

- Knowledge of MLOps/LLMOps workflows, CI/CD pipelines, model deployment and production monitoring.

- Experience with AWS services such as Lambda, S3 and ElastiCache.

- Familiarity with monitoring and observability tools such as Datadog and AWS CloudWatch.

- Strong analytical, problem-solving and communication skills.

Nice to Have:

- Experience with Microsoft Fabric or similar analytics platforms.

- Exposure to fine-tuning foundation models and optimising LLM inference.

- Experience building conversational AI applications, enterprise AI assistants or customer insight platforms.

- Knowledge of Kubernetes, Docker, microservices and cloud-native architectures.

- Experience working with large-scale customer interaction and conversational datasets.