Role Overview :

We are looking for a highly skilled with expertise in Kafka, Python, and Azure Databricks (preferred) to drive our healthcare data engineering projects.

The ideal candidate will have deep experience in real-time data streaming, cloud-based data platforms, and large-scale data processing.

This role requires strong technical leadership, problem-solving abilities, and the ability to collaborate with cross-functional teams.

Key Responsibilities :

- Lead the design, development, and implementation of real-time data pipelines using Kafka, Python, and Azure Databricks.

- Architect scalable data streaming and processing solutions to support healthcare data workflows.

- Develop, optimize, and maintain ETL/ELT pipelines for structured and unstructured healthcare data.

- Ensure data integrity, security, and compliance with healthcare regulations (HIPAA, HITRUST, etc.

- Collaborate with data engineers, analysts, and business stakeholders to understand requirements and translate them into technical solutions.

- Troubleshoot and optimize Kafka streaming applications, Python scripts, and Databricks workflows.

- Mentor junior engineers, conduct code reviews, and ensure best practices in data engineering.

- Stay updated with the latest cloud technologies, big data frameworks, and industry trends.

Required Skills & Qualifications :

- 4+ years of experience in data engineering, with strong proficiency in Kafka and Python.

- Expertise in Kafka Streams, Kafka Connect, and Schema Registry for real-time data processing.

- Experience with Azure Databricks (or willingness to learn and adopt it quickly).

- Hands-on experience with cloud platforms (Azure preferred, AWS or GCP is a plus).

- Proficiency in SQL, NoSQL databases, and data modeling for big data processing.

- Knowledge of containerization (Docker, Kubernetes) and CI/CD pipelines for data applications.

- Experience working with healthcare data (EHR, claims, HL7, FHIR, etc.) is a plus.

- Strong analytical skills, problem-solving mindset, and ability to lead complex data projects.

- Excellent communication and stakeholder management skills