Databricks Expert/AI Engineer

Databricks Expert / AI Engineer

Job Description :

We are looking for a highly skilled and passionate Databricks Expert / AI Engineer to join our fast-paced, innovation-driven startup. The ideal candidate should have hands-on experience with LLM models like LLAMA, Vector Database integration, PostgreSQL, and end-to-end application development using modern AI toolsets. Youll play a crucial role in building intelligent, data-driven applications that are production-ready and scalable.

Key Responsibilities :

- Work closely with data scientists and product teams to build and scale AI applications using Databricks.

- Fine-tune and deploy LLM models (LLAMA or similar) to solve real-world problems.

- Integrate and manage Vector Databases (e.g., FAISS, Pinecone, Weaviate) for efficient semantic search and retrieval tasks.

- Develop robust Python scripts and backend services to support AI workflows and data pipelines.

- Optimize and manage PostgreSQL databases for model data storage and feature engineering.

- Contribute to designing microservices and APIs for integrating AI capabilities into applications.

- Handle the entire ML lifecyclefrom experimentation to deployment using Databricks ML, MLflow, or SageMaker.

- Deliver MVPs and product enhancements quickly, adapting to changes in a fast-moving startup environment.

- Collaborate in an Agile development cycle and contribute to a DevOps-friendly stack.

Key Requirements :

- 5+ years of experience in data engineering, machine learning, or AI application development.

- Strong expertise in Databricks platform, Spark, and distributed data processing.

- Hands-on experience with LLMs (preferably LLAMA) and LangChain or similar frameworks.

- Proficiency in Vector Databases and retrieval-augmented generation (RAG) pipelines.

- Strong command of Python with experience in building scalable backend logic and automation scripts.

- Familiarity with PostgreSQL, data modeling, and query optimization.

- Knowledge of CI/CD, containerization (Docker/Kubernetes), and cloud platforms (AWS/Azure/GCP).

- Experience working in a startup or agile environment, with the ability to deliver rapid iterations.

Preferred Qualifications :

- Exposure to MLOps tools and practices.

- Experience with transformer-based models, Hugging Face libraries, or similar NLP stacks.

- Knowledge of streaming data pipelines (Kafka, Spark Streaming) is a plus.

- Prior experience in building AI-powered applications or tools in production.