Role overview :

We are seeking a Lead Backend Engineer with strong production-level Python experience and deep expertise in large-scale search platforms, preferably SolrCloud/Lucene, to help integrate and scale two large healthcare data platforms. The role will focus on enabling enriched emergency medical records from one platform to be consumed effectively by another. The platform currently manages approximately 500 TB of data and processes around 50 GB of new data daily, with the initial implementation focused on one data source and plans to onboard approximately ten additional sources. You will work closely with a small, senior engineering team and take ownership of complex technical challenges across data ingestion, enrichment, modeling, indexing, APIs and search architecture.

Key responsibilities :

- Own the lifecycle of data attributes across the technology stack, from definition and modeling through ingestion, indexing, APIs and presentation.

- Define scalable data models and attribute structures to support evolving healthcare data requirements.

- Develop and maintain data ingestion and enrichment workflows using Python.

- Work closely with Java/Spring services to expose and serve enriched data through APIs.

- Collaborate with front-end teams to ensure data is effectively presented and consumed.

- Design and implement a metadata-driven framework for managing data attributes across the technology stack.

- Create a single attribute definition that can generate the required configurations and artifacts currently maintained across multiple components.

- Design repeatable processes and frameworks for onboarding new data sources.

- Redesign the existing Solr architecture to support continued growth beyond the current fixed three-shard layout.

- Define and optimize SolrCloud sharding, routing, indexing and query strategies.

- Develop scalable, maintainable and production-ready backend components using Python.

- Contribute confidently to Java/Spring-based services and APIs.

Tech Stack :

- Python, Apache Solr/Lucene, SolrCloud 9, Java, Spring, SQL.

Required technical skills :

- Strong production-level experience with Python.

- Deep hands-on experience with Apache Solr/Lucene, preferably SolrCloud 9.

- Working knowledge of Java and Spring-based backend services.

- Strong SQL and data-modeling skills.

- Experience designing scalable data-ingestion and data-processing pipelines.

- Strong understanding of distributed systems and large-scale data platforms.

Good to have :

- Experience working with healthcare data, NEMSIS data, Databricks, and metadata-driven platforms.