Senior Data Engineer - ETL/Python

Description :

We are looking for an experienced Data Engineer to design, build, and optimize scalable data pipelines and analytics solutions on the Azure Data Platform. The ideal candidate will have strong hands-on experience with modern Azure data services and a solid foundation in SQL, Python, and Spark.

Key Responsibilities :

- Design, develop, and maintain end-to-end data pipelines using Azure Data Factory (ADF).

- Build and optimize data models and analytical datasets using Azure Synapse Analytics and Microsoft Fabric.

- Develop scalable data processing solutions using Databricks and Apache Spark.

- Ensure data quality, reliability, performance, and security across data platforms.

- Collaborate with analytics, BI, and business teams to understand data requirements and deliver insights.

- Optimize SQL queries and improve performance of large-scale datasets.

- Implement best practices for data engineering, monitoring, and documentation.

- Support production deployments and troubleshoot data pipeline issues.

Required Skills & Experience :

- 56 years of hands-on experience as a Data Engineer or in a similar role.

- Strong experience with Azure Data Platform, including:

- Azure Synapse Analytics

- Azure Fabric

- Azure Data Factory (ADF)

- Azure Databricks

- Strong proficiency in SQL for data modeling and query optimization.

- Hands-on experience with Python for data processing and automation.

- Solid understanding of Apache Spark and distributed data processing.

- Experience working with large datasets and performance tuning.

- Understanding of data warehousing concepts and ETL/ELT processes.

Nice to Have :

- Experience with CI/CD for data pipelines.

- Knowledge of data governance, security, and compliance on Azure.

- Exposure to Power BI or other reporting/analytics tools.

- Azure certifications related to data engineering.

- Experience working in other Public clouds like AWS & GCP.

- Understanding of Cloud Infrastructure & Networking.