Role Overview:

As a Lead Data Engineer based in Pune, you will serve as the technical cornerstone for our data infrastructure, architecting robust systems that transform raw information into actionable business intelligence.

You will lead a high-performing team of engineers, collaborating closely with product managers, data scientists, and executive stakeholders to define the roadmap for our data ecosystem.

Your work will directly influence the scalability of our platform, ensuring that our data pipelines are not only performant and reliable but also capable of supporting complex analytical workloads that drive critical decision-making across the organization.

Key Responsibilities :

- Architect and oversee the development of scalable end-to-end data pipelines to ensure seamless data ingestion, transformation, and delivery for downstream analytical applications.

- Lead the design and implementation of complex data modeling strategies to optimize storage efficiency and query performance within our data warehousing environment.

- Mentor junior engineering talent by conducting rigorous code reviews and fostering a culture of technical excellence to elevate the team's overall output.

- Partner with cross-functional stakeholders to translate ambiguous business requirements into high-impact technical specifications that align with long-term organizational goals.

- Drive the migration and optimization of legacy data systems to cloud-native architectures to enhance system reliability and reduce operational overhead.

Required Skillset:

Mandatory to have at least one certification:

- Databricks Certified Data Engineer Associate OR Databricks Certified Data Engineer Professional

- Minimum 8 - 14 years in data engineering or related roles

- At least 4 - 8 years of hands-on experience with Databricks platform

- Min 2 - 3 years experience in team handling.

- SQL proficiency for data querying and transformation

- Strong Python programming skills for data processing and automation

- Proven hands-on experience developing data pipelines using PySpark, including DataFrames API, Spark SQL, and performance optimization

- Stakeholder management and mentoring experience

- Job stability - min 2yrs in an organization

Nice to Haves :

- Sprint Planning

- Estimation, Cross-functional Team Handling

- Delivery Ownership

- Release Management, Production Support