OneMagnify - Databricks Engineer - Artificial Intelligence

Administers, maintains, develops and implements policies and procedures for ensuring the security and integrity of the company databases.

Implements data models and database designs, data access and table maintenance; resolves database performance issues, database capacity issues, replication, and other distributed data issues.

Job Description :

ESSENTIAL DUTIES AND RESPONSIBILITIES :

- Programming & Query Languages: Proficient in Python, and SQL for data analysis, modeling, and database management.

- Machine Learning & AI Models: Experience with classical ML (e.g., regression, decision trees, SVM), deep learning (CNNs, RNNs, Transformers), and advanced AI (GNNs, RL, GANs, LLMs like GPT-4, Claude, LLaMA).

- Natural Language & Vision AI: Skilled in NLP (NER, sentiment analysis, summarization) and computer vision (YOLO, ResNet, segmentation, OCR).

- Frameworks & Libraries: Hands-on with TensorFlow, PyTorch, Scikit-learn, Keras, Hugging Face Transformers, and Spark MLlib.

- Databricks Ecosystem: Proficient in Databricks Notebooks, MLflow, AutoML, Delta Lake, Unity Catalog, Feature Store, Model Serving, and Lakehouse AI.

- Data Engineering & Big Data: Experience with Apache Spark, Hadoop, Kafka, Airflow, dbt, and building scalable ETL pipelines.

- Cloud Platforms: Deploy and manage solutions on AWS (S3, SageMaker), Azure (ML, Synapse), and GCP (BigQuery, Vertex AI).

- Statistical Analysis & Experimentation: Apply A/B testing, hypothesis testing, Bayesian methods, and time series forecasting (ARIMA, Prophet, LSTM).

- Model Deployment & DevOps: Deploy models using Flask, FastAPI, Docker, Kubernetes, CI/CD pipelines, and integrate with APIs and Git for version control.

QUALIFICATIONS :

- To perform this job successfully, an individual must be able to perform each essential duty satisfactorily.

- The requirements listed below are representative of the knowledge, skill, and/or ability required.

- Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

EDUCATION AND/OR EXPERIENCE :

- Bachelors Degree (B.) from four-year college or university.

- In-depth understanding of all aspects of database technology with minimum five years experience in database administration or equivalent job experience.

- Experience with UNIX, Linux, and Microsoft Windows operating systems.