Job Description :

We are seeking skilled AI/ML QA Specialists with strong Databricks experience to ensure the quality, reliability, and regulatory readiness of AI/ML platforms. This role will focus on end-to-end testing of CCAR and ESG projects, covering data pipelines, feature engineering, model training, validation, deployment, and monitoring.

Key Responsibilities :

Model Development Platform QA :

- Validate data ingestion, feature engineering, and training pipelines built on Databricks (Spark, Delta, MLflow).

- Design and execute QA strategies for dataset quality, schema validation, lineage, feature consistency, drift checks, and reproducibility.

- Test MLflow experiments, model versioning, and artifacts for completeness and traceability.

- Ensure compliance with model risk management (MRM), audit, and documentation standards.

Model Execution / Production Platform QA :

- Test model deployment pipelines, including batch and real-time model execution.

- Validate model scoring accuracy, performance, data contracts, SLAs, error handling, and fallback logic.

- Perform regression, performance, and volume testing for production workloads.

Automation & Tooling :

- Build and maintain automated test frameworks for data and ML pipelines (Databricks notebooks, PySpark, Python).

- Implement data-driven QA checks (DQ rules, nulls, thresholds, statistical validation).

- Integrate QA into CI/CD pipelines for ML workflows.

Required Skills :

- 5 - 8+ years of QA or data validation experience.

- Hands-on experience with Databricks (Spark/PySpark, Delta Lake, MLflow).

- Strong Python experience for testing and automation.

- Solid understanding of the ML lifecycle.

- Knowledge of cloud platforms (Azure preferred).

Preferred Skills :

- Experience with model risk management (MRM) or regulated environments.

- Exposure to feature stores, model monitoring, and drift detection.

- Experience with performance testing at scale in distributed environments.

Education :

- Bachelors or Masters degree in Computer Science, Data Science, Engineering, or a related field.