Senior Data Engineer - PySpark/Azure Databricks

Description :

We are looking for an experienced Senior Data Engineer (L3) to join a high-performing Data Science team. The ideal candidate will be responsible for building scalable data infrastructure, developing robust data pipelines, and enabling advanced analytics and machine learning workflows. This role requires strong expertise in modern data engineering technologies, including DBT, Databricks, PySpark, Airflow, SQL, and cloud-based data systems. The candidate should be passionate about solving complex data challenges and working closely with Data Science, Analytics, and Engineering teams across global locations.

Responsibilities :

- Design, develop, and maintain scalable DBT models for analytics, reporting, and machine learning use cases.

- Build and optimize data pipelines using Databricks, PySpark, Delta/Iceberg tables, and SQL.

- Develop reliable orchestration workflows using Airflow and Prefect.

- Transform raw operational data into trusted, production-grade datasets.

- Collaborate with Data Scientists and ML Engineers to productionize feature pipelines and ML-supporting datasets.

- Improve data quality, reliability, testing, and documentation standards.

- Optimize Spark and DBT workloads for performance, scalability, and cost efficiency.

- Participate in peer code reviews and contribute to engineering best practices.

- Work closely with cross-functional global teams to support operational and business decision-making.

Requirements :

- 5+ years of experience in Data Engineering.

- Strong expertise in SQL and distributed data systems.

- Hands-on experience with DBT, Databricks, PySpark / Spark, Airflow, Prefect, and Delta Lake / Iceberg.

- Experience with Infrastructure-as-Code tools such as Terraform, AWS CDK, or Pulumi.

- Good understanding of data architecture, scalability, performance optimization, and fault tolerance.

- Experience working with cloud-based modern data platforms.

- Strong communication and collaboration skills.

Preferred Skills :

- Experience with streaming systems like Kinesis.

- Exposure to MLOps and AI/ML workflows.

- Familiarity with AI coding assistants such as Copilot, Cursor, Claude Code, etc.

- Experience working with globally distributed teams.

- Understanding of the supply chain or logistics domain is an added advantage.

What We're Looking For :

- Self-driven and proactive mindset.

- Strong problem-solving abilities.

- Ability to work in a fast-paced, collaborative environment.

- Passion for building scalable and reliable data systems.