Job Description :

Position : Senior Data Engineer

Location : Chennai, Bangalore, Hyderabad

Job Type : Full-time

Interview Process : 4 Interview Rounds

Notice Period : Immediate to 30 days Only

Position Overview :

We are seeking a talented and experienced Senior Data Engineer to join our data engineering team. You will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure on cloud platforms. This role requires expertise in modern data engineering tools and a strong foundation in data transformation and optimisation.

Must Haves :

- 8+ years of hands-on experience in Azure Data Engineering.

- Strong expertise in Azure Data Factory (ADF) for building and managing ETL/ELT pipelines.

- Extensive experience with Azure Databricks, including Apache Spark and PySpark.

- Hands-on experience with Lakeflow Declarative Pipelines and enterprise-scale data platforms in Azure Cloud.

- Strong proficiency in SQL for complex data transformation and optimization.

- Experience designing and implementing scalable data pipelines and large-scale data processing solutions.

- Solid understanding of data warehousing, data modeling, and performance tuning.

Qualifications :

- Bachelor's degree in Computer Science, Engineering, Mathematics, or related field.

- 8+ years of professional experience in data engineering or related roles.

- 3+ years of hands-on experience with cloud-based data platforms.

- Proven track record of designing and implementing production-grade data pipelines.

- Strong problem-solving skills and attention to detail.

Additional Qualifications :

- Experience with cloud platforms (Azure, AWS, or GCP).

- Knowledge of data warehousing concepts and modern data architecture.

- Familiarity with version control systems (Git).

- Understanding of CI/CD pipelines and DevOps practices.

- Experience with data quality frameworks and testing methodologies.

- Knowledge of big data technologies and distributed computing.

Key Responsibilities :

- Design and develop robust, scalable data pipelines using Azure Data Factory (ADF).

- Build and optimize data processing workflows using Databricks and Apache Spark.

- Implement declarative pipeline solutions using Lakehouse architecture and best practices.

- Write and optimize complex SQL queries for data transformation and analysis.

- Develop PySpark applications for large-scale data processing.

- Collaborate with data scientists, analysts, and stakeholders to understand data requirements.

- Ensure data quality, security, and compliance across all pipelines.

- Monitor, troubleshoot, and optimize data pipeline performance.

- Document technical solutions and maintain best practices documentation.

- Mentor junior team members and contribute to team knowledge sharing.