About the Role :

We are seeking a Senior Software Engineer with 5 - 7 years of data engineering experience to lead the design and delivery of the data pipelines and services behind our platforms.

You will drive technical decisions on ingestion, transformation, and serving of data from diverse sources, mentor junior engineers, and work closely with product, platform, and programme teams to deliver data systems at scale.

This role also requires hands-on DevOps experience.

Required Skills & Qualifications :

- Experience : 5 - 7 years in data engineering or backend software engineering with a data focus, including experience leading or mentoring within a team.

- Education : Bachelor's/Master's in Computer Science or a related field.

- Programming : Expert-level Python (or Java/Scala) and advanced SQL, with a strong track record of clean, testable, production-grade code.

- Data Pipelines : Proven experience designing and owning batch and streaming ETL/ELT pipelines using tools such as Apache Airflow, Spark, Kafka, or dbt, including architecture decisions and platform choices.

- Databases & Storage : Solid knowledge of PostgreSQL or similar relational databases, plus at least one NoSQL, columnar, or time-series store; familiarity with data lake/warehouse concepts and formats such as Parquet, Avro, and JSON.

- APIs : Experience designing and consuming RESTful APIs and integrating data from varied sources and formats.

- Tools : Proficiency with Git, code reviews, automated testing, and CI workflows.

- Leadership : Experience mentoring engineers, leading design/code reviews, and driving technical decisions across a team.

- DevOps : Hands-on experience with Docker, Kubernetes, CI/CD (GitHub Actions, GitLab CI, or Jenkins), infrastructure as code (Terraform/Ansible), monitoring (Prometheus, Grafana), and cloud or Linux environments.

- Soft Skills : Problem-solving, clear communication and documentation, teamwork, and ability to work independently.

Key Responsibilities :

- Own the technical design of scalable batch and streaming data pipelines for ingesting data from APIs, databases, files, and event streams.

- Set standards for ETL/ELT workflows and data models that are reliable, testable, and easy to evolve, and review designs proposed by other engineers.

- Architect data storage and serving layers, and guide the team on exposing data through well-designed APIs.

- Define data quality, validation, lineage, monitoring, and alerting standards, and ensure the team implements them consistently.

- Lead performance tuning and troubleshooting of complex, production-scale data issues end to end.

- Partner with architects, product owners, and programme teams to shape requirements into technical solutions and roadmaps.

- Champion security, privacy, and data governance practices suited to government-facing systems.

- Write clean, documented, version-controlled code; lead code reviews and design discussions, and mentor junior and mid-level engineers.

- Own deployment, monitoring, and maintenance of the Data Exchange platform in production, ensuring stable, dependable operations.