Lead Data Engineer - SQL/Python

Description :

We are seeking an experienced Lead Data Engineer to design, develop, and optimize scalable data pipelines and cloud-based data platforms. The ideal candidate will have strong technical expertise across modern data engineering tools, cloud technologies, and DevOps practices, along with the ability to collaborate with cross functional teams to deliver high-quality data solutions.

Experience : 6+ Years

Role : Lead Data Engineer

Education : BE/BTech/ME/MTech in Computer Science or Information Systems, or related field

Key Responsibilities :

- Develop, enhance, and maintain scalable data pipelines to support operational and analytical workloads.

- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver effective solutions.

- Manage and optimize cloud-based data infrastructure (Snowflake, Azure) for performance, scalability, and cost efficiency.

- Ensure data quality, accuracy, and reliability through comprehensive testing and validation practices.

- Support and contribute to data governance initiatives in coordination with Data QA and Data Governance teams.

- Monitor, troubleshoot, and resolve issues related to data pipeline performance and data flows.

- Participate in the design and implementation of data models and database schemas aligned with business requirements.

- Stay updated with emerging technologies and best practices in data engineering and recommend improvements to existing systems.

Required Experience & Technical Skills :

- 6- 10 years of experience in Data Engineering or related roles.

- Strong hands-on experience in designing and maintaining ETL/ELT pipelines and data integrations.

- Proficiency in :

1. Python

2. SQL

3. PySpark

4. Apache Airflow

5. DBT or SnowSQL

6. Cloud & DevOps experience with:

7. Snowflake

8. Azure DevOps, Terraform, Azure Pipelines

9. Docker, Kubernetes

10. Azure Databricks (preferred)

- Solid understanding of data modelling concepts and database management systems.

- Experience with big data technologies such as Apache Spark, Hadoop, and tools like dbt.

- Familiarity with CI/CD pipelines and version control systems (Git).

- Understanding of data governance, data quality frameworks, and cloud security best practices.