Azure Data Engineer - Databricks / PySpark / SAP S/4HANA

Experience: 6 - 12 Years

Location: Chennai (Hybrid)

Industry: FMCG / CPG

About the Opportunity:

We are looking for an experienced Azure Data Engineer to work on a large-scale enterprise data transformation initiative for a leading global FMCG / CPG organisation.

The role involves designing, developing and implementing a second enterprise data lake using Azure Databricks, Azure Data Lake Storage, Python, PySpark and SQL.

You will collaborate with Data Architects, SAP specialists, BI developers and business stakeholders to integrate SAP S/4HANA and other enterprise data sources into a scalable Azure-based data platform.

This is a hands-on engineering opportunity involving data pipeline development, data integration, performance optimisation, data quality and production support.

Key Responsibilities:

- Design, develop and maintain scalable ETL / ELT pipelines using Azure Databricks, PySpark, SQL and Azure services.

- Integrate data from SAP S/4HANA and other enterprise applications into Azure Data Lake.

- Develop curated data layers using Delta Lake and lakehouse architecture principles.

- Implement incremental data processing, pipeline orchestration, monitoring and error handling.

- Build reliable datasets for business reporting, analytics and downstream applications.

- Implement data quality checks, reconciliation and validation against source systems.

- Optimise data pipeline performance, processing efficiency and cloud resource utilisation.

- Apply version control, CI / CD, automated deployment and testing practices.

- Support data governance, access controls, lineage and technical documentation.

- Troubleshoot production issues and support ongoing data platform operations.

- Collaborate with cross-functional teams across SAP, Data Architecture, BI and business functions.

Required Skills and Experience:

- 6 - 12 years of relevant professional experience, including strong hands-on experience in Azure Data Engineering.

- Strong proficiency in Azure Databricks, Python, PySpark and SQL.

- Experience with Azure Data Lake Storage (ADLS).

- Hands-on experience with Azure Data Factory (ADF) or Databricks Workflows.

- Good understanding of Delta Lake, ETL / ELT pipelines and dimensional data modelling.

- Experience integrating or processing SAP S/4HANA data.

- Understanding of SAP business objects, data structures and enterprise data integration.

- Experience implementing incremental data processing, data quality controls and performance optimisation.

- Ability to collaborate effectively with technical and business stakeholders.

Preferred Skills:

- Experience in FMCG / CPG, retail, manufacturing or supply chain domains.

- Experience building enterprise data lakes or lakehouse platforms.

- Knowledge of SAP extraction approaches such as CDS Views, ODP or SLT.

- Experience preparing data for Power BI reporting.

- Exposure to CI / CD, infrastructure as code and data governance tools.

- Experience working with international or global teams.

Employment Details:

This is a full-time, permanent opportunity on the payroll of our staffing partner.

The selected candidate will work directly with the technology and data teams of a leading global FMCG / CPG organisation, contributing to enterprise-scale data engineering and analytics initiatives.

Please note: The employment relationship will be with the staffing partner, while the day-to-day project responsibilities will be aligned with the FMCG organisation.

Why Consider This Opportunity:

- Work on an enterprise-scale Azure Data Lake and modern data engineering initiative.

- Gain exposure to SAP S/4HANA integration and complex enterprise datasets.

- Collaborate with international Data Architecture, SAP and Analytics teams.

- Work with modern technologies including Azure Databricks, PySpark and Delta Lake.

- Contribute to data solutions supporting critical FMCG business functions such as supply chain, sales, inventory and finance.

Interested candidates with relevant Azure Data Engineering and SAP integration experience are encouraged to apply.