Quantzig - Data Engineer

Job Role : Data Engineer

YoE : 2 to 7 years

Location : Bangalore

Date : Ongoing (from 29th April 2026)

About the Role :

We are seeking a highly skilled Data Engineer with expertise in Databricks and AWS Cloud to design, build, and optimize enterprise-scale ETL pipelines and reporting solutions. This role is ideal for someone who thrives at the intersection of data engineering and business intelligence, with a strong focus on transforming complex datasets into inputs for actionable insights. You will play a critical role in enabling spend analytics, patient access, and commercial reporting initiatives.

Key Responsibilities :

Databricks ETL Development :

- Design, develop, and maintain scalable ETL pipelines in Databricks using PySpark and SQL.

- Implement robust data ingestion, transformation, and validation processes to ensure high-quality datasets for analytics.

- Optimize workflows for performance, scalability, and reliability across large healthcare datasets.

Data Product Build :

- Transform AWS assets (on EC2 and Redshift) to Databricks by recreating the running ETL jobs and orchestrating via Airflow or Databricks Workflows.

- Hands-on working experience of catalog management using Unity Catalog.

- Conversant with AI capabilities within AWS and Databricks.

Requirements :

- 2 to 7 years of experience in Databricks data engineering roles.

- Strong hands-on proficiency in Databricks (PySpark, SQL) and AWS Cloud.

- Proven track record in ETL pipeline development and understanding of BI dashboards.

- Experience with US healthcare datasets.

- Strong SQL skills for data extraction, aggregation, and reporting.

- Excellent problem-solving abilities, with the capacity to work independently and in cross-functional teams.

- Detail-oriented with a commitment to delivering high-quality, reliable data solutions.

Good to Have :

Domain & Data Knowledge :

- Knowledge of IQVIA claims (standard, eLaaD, Remit, Rejection, NBRx, TRx, IGG4 Claims) and MMIT datasets to derive actionable insights.

End-to-End Production Deployment & Orchestration :

- Candidates with exposure to deploying and orchestrating data pipelines in production environments will be preferred. While not a core requirement, the following experience is a strong differentiator :

- Experience deploying ETL or ML pipelines end-to-end in a production environment, including environment promotion across dev, staging, and production.

- Familiarity with CI/CD tooling (GitHub Actions, Azure DevOps, or Jenkins) for automating pipeline deployment and release management.

- Exposure to Databricks Asset Bundles (DABs) or equivalent frameworks for version-controlled, repeatable job deployments.

- Working knowledge of Apache Airflow for DAG authoring, scheduling, dependency management, and monitoring of production workflows.

- Awareness of infrastructure-as-code practices (Terraform or AWS CloudFormation) for managing cloud resources supporting data pipelines.

- Basic understanding of containerization concepts (Docker) for packaging and deploying data pipeline components.

- Experience with pipeline monitoring and alerting tracking job health, SLA adherence, failure notifications, and data freshness in production.

- Familiarity with secrets and configuration management across environments (AWS Secrets Manager, Databricks Secrets, or equivalent).