Data Scientist - LLM Models

Key Responsibilities :

- Utilize frameworks like Langchain for developing scalable and efficient AI solutions.

- Integrate vector databases such as Azure Cognitive Search, Weaviate, or Pinecone to support AI model functionalities.

- Work closely with cross-functional teams to define problem statements and prototype solutions leveraging generative AI.

- Ensure robustness, scalability, and reliability of AI systems by implementing best practices in machine learning and software development

- Exploring and visualizing data to gain an understanding of it, then identifying differences in data distribution that could affect performance when deploying the model in the real world.

- Demonstrable history of devising and overseeing data-centred projects

- Verifying data quality, and/or ensuring it via data cleaning

- Supervising the data acquisition process if more data is needed

- Finding available datasets online that could be used for training

- Defining validation strategies, feature engineering & data augmentation pipelines to be done on a given dataset

- Training models and tuning their hyperparameters

- Analysing the errors of the model and designing strategies to overcome them

- Deploying models to production

Preferred Skills :

- Proficiency in statistical techniques such as hypothesis testing, regression analysis, clustering, classification, and time series analysis to extract insights and make predictions from data.

- Proficiency with a deep learning framework such as TensorFlow, PyTorch & Keras

- Specialized in Deep Learning (NLP) and statistical machine learning.

- Strong Python skills.

- Experience with developing production-grade applications.

- Familiarity with Langchain framework and vector databases like Azure Cognitive Search, Weaviate, or Pinecone.

- Understanding and experience with retrieval algorithms.

- Worked on Big data platforms and technologies such as Apache Hadoop, Spark, Kafka, or Hive for processing and analyzing large volumes of data efficiently and at scale.

- Familiarity in working and deploying applications on Ubuntu/Linux system

- Excellent communication, negotiation, and interpersonal skills