Work Location: Guindy Olympia Tech

Work Mode: Hybrid - 3 days in the office.

Department: Product (AI) - Candidates are not expected to have AI experience.

Role: Senior Lead/Architect - Data Engineering.

Experience: 8 to 14 years.

Primary Skill sets:

- Databricks Lakehouse architecture: PySpark, Delta Lake, Delta Live Tables / Lakeflow, Unity Catalog, Workflows, Databricks SQL, CDC, Schema evolution, Security and governance.

- Azure Data Factory (ADF) pipelines: Copy activities, Mapping Data Flows, Metadata-driven pipelines, Dynamic pipelines, Integration Runtime, Self-hosted IR, Trigger mechanisms, Error handling, Retry frameworks, Dependency management, CI/CD.

- Enterprise Data Architecture: Modern Data Lakehouse, Data Warehouse, Medallion architecture, Dimensional modelling, Star/Snowflake schemas, Data Vault, Data mesh, Batch and streaming architecture, CDC, Event-driven pipelines, Master/reference data, Data governance, Data lineage, Data quality, Metadata management.

- Metadata-driven: Ingestion, Dynamic pipeline generation, Automated schema discovery, Automated table creation, Incremental load framework, CDC framework, Data quality framework, Pipeline monitoring, Exception handling, Automated reconciliation, Data validation, Deployment automation, Environment provisioning.

- Legacy ETL modernisation: Data warehouse migration, Data lake migration, ETL - ELT transformation, On-premises - Azure migration, Legacy database - Lakehouse migration, Batch - near-real-time modernisation, stored procedure modernisation, Legacy ETL code conversion.

Notice Period: Immediate Joiner / Early Joiner - 0 days to 15 days.