Lead Data Engineer - Adobe Experience Platform

Description :

We are seeking an experienced Lead Data Engineer - AEP with strong expertise in Big Data and Cloud technologies to design and build scalable data solutions for Adobe Experience Cloud customers. The ideal candidate will have hands-on experience in ETL development, distributed data processing, and AWS cloud-native architectures.

Key Responsibilities :

- Develop low-level design as per approved technical specifications.

- Build and deliver technology solutions aligned with functional and non-functional requirements.

- Develop scalable ETL pipelines involving Big Data.

- Design and develop data processing and analytics applications using PySpark and Python.

- Develop applications on AWS (Storage, Compute, ETL, DWH, Analytics, Streaming).

- Implement distributed storage and scalable data processing frameworks.

- Ensure solutions meet performance, scalability, maintainability, and reliability standards.

- Develop applications consuming REST APIs.

- Process deliverables through the complete development lifecycle.

- Maintain and support existing solutions and technology frameworks.

- Participate in Agile/Scrum ceremonies and provide regular updates.

- Work independently with minimal supervision.

Mandatory Skills and Experience :

- 8+ years of professional software engineering experience.

- 6+ years of relevant experience in Data Engineering.

- Strong experience developing ETL pipelines involving Big Data.

- Proficiency in PySpark, Python, and distributed data processing.

- Hands-on experience with AWS cloud services.

- Strong understanding of distributed systems and scalable architecture.

- Experience with SQL and NoSQL databases.

- Ability to write and optimize SQL, HQL, and NoSQL queries.

- Experience developing applications that consume REST APIs.

- Experience in Adobe Experience Platform (AEP).

Preferred Skills :

- Experience with Kubernetes or other container orchestration systems.

- Experience with enterprise-grade ETL tools.

- Knowledge of Adobe Experience Cloud solutions.

- Experience in Web Analytics or Digital Marketing domain.