We are seeking a seasoned Lead Data Engineer with deep technical expertise and leadership experience to join our team. The ideal candidate should possess a strong foundation in data engineering, a solid understanding of machine learning workflows, and hands-on experience with modern data platforms such as Databricks. This role will require close collaboration with client-side ML engineers, Product Owners, and other stakeholders to ensure seamless delivery and optimization of data and AI-driven solutions.

Key Responsibilities :

- Provide technical leadership and guidance to clients on the current data and analytics landscape.

- Optimize and manage large-scale data pipelines, particularly using Apache Spark.

- Collaborate with client ML engineers to define and implement MLOps and LLMOps processes.

- Assist in productionizing Generative AI (GenAI) workflows; prior experience in GenAI is a plus, but strong knowledge is essential.

- Work extensively with Databricks for data engineering and analytics tasks.

- Partner with the clients Product Owner to identify resource requirements, manage team allocations, and plan project milestones and release schedules.

- Lead and mentor a team of data engineers, ensuring high-quality delivery and adherence to best practices.

Required Skills & Qualifications :

- 9- 15 years of experience in Data Engineering, with strong hands-on technical skills.

- Proven experience with Apache Spark and performance optimization of data pipelines.

- Working knowledge or experience in MLOps, LLMOps, and GenAI workflows.

- Must have extensive hands-on experience with Databricks.

- Strong communication and stakeholder management skills.

- Experience in leading technical teams and handling project delivery in a collaborative environment.

Nice to Have :

- Exposure to cloud platforms like Azure, AWS, or GCP.

- Familiarity with data governance and security best practices.

- Experience working in Agile environments.