AI Engineer - RAG/LLM/VectorDB

Job Overview :

We are seeking a highly skilled AI Engineer with expertise in Multimodal Retrieval-Augmented Generation (RAG), Vector databases, and Large Language Model (LLM) implementation. The ideal candidate will have a strong background in integrating structured and unstructured data into AI models and deploying these models in real-world applications. This role involves working on cutting-edge AI solutions, including the development and optimization of multimodal systems that leverage both text and visual data.

Key Responsibilities:

Multimodal RAG Implementation :

- Design, develop, and deploy Multimodal Retrieval-Augmented Generation (RAG) systems that integrate both structured (e.g., databases, tables) and unstructured data (e.g., text, images, videos).

- Work with large-scale datasets, combining different data types to enhance the performance and accuracy of AI models.

- Implement and fine-tune LLMs (e.g., GPT, BERT) to work effectively with multimodal inputs and outputs.

Vector Database Integration :

- Develop and optimize AI models using vector databases to efficiently manage and retrieve high-dimensional data.

- Implement vector search techniques to improve information retrieval from structured and unstructured data sources.

- Ensure the scalability and performance of vector-based retrieval systems in production environments.

LLM Implementation and Optimization :

- Implement and fine-tune large language models to handle complex queries involving multimodal data.

- Optimize LLMs for specific tasks, such as text generation, question answering, and content summarization, using both

structured and unstructured data.

- Integrate LLMs with vector databases and RAG systems to enhance AI capabilities.

Data Integration and Processing :

- Work with data engineers and data scientists to preprocess and integrate structured and unstructured data for AI model training and inference.

- Develop data pipelines that handle the ingestion, transformation, and storage of diverse data types.

- Ensure data quality and consistency across different data sources and formats.

Model Evaluation and Testing :

- Evaluate the performance of multimodal AI models using various metrics, ensuring they meet accuracy, speed, and robustness requirements.

- Conduct A/B testing and model validation to continuously improve AI system performance.

- Implement automated testing and monitoring tools to ensure model reliability in production.

Collaboration and Communication :

- Collaborate with cross-functional teams, including data engineers, data scientists, and software developers, to deliver AI-driven solutions.

- Communicate complex technical concepts to non-technical stakeholders and provide insights on the impact of AI models on business outcomes.

- Stay up to date with the latest advancements in AI, LLMs, vector databases, and multimodal systems, and share knowledge with the team.

Qualifications :

Technical Skills :

- Strong expertise in Multimodal Retrieval-Augmented Generation (RAG) systems.

- Proficiency in vector databases (e.g., Pinecone, Milvus, Weaviate, Chroma) and vector search techniques with recommender systems, vector search capabilities.

- Experience with LLMs (e.g., GPT, BERT) and their implementation in real-world applications. Experience with Mistral AI is a plus.

- Solid understanding of machine learning and deep learning frameworks (e.g., TensorFlow, PyTorch, MLflow etc).

- Experience working with structured data (e.g., SQL databases) and unstructured data (e.g., text, images, videos).

- Proficiency in programming languages such as Python, with experience in relevant libraries and tools.

Experience :

- 2+ years of experience in AI/ML engineering, with a focus on multimodal systems and LLMs.

- Proven track record of deploying AI models in production environments.

- Experience with cloud platforms preferably Azure, and MLOps practices is preferred.