Job Title : Senior Data Engineer Azure & PySpark

Experience : 6+ Years

Location : Pune / Bangalore / Hyderabad (Hybrid)

Employment Type : Full-time

About the Role :

We are looking for an experienced Senior Data Engineer who can design, build, and optimize scalable data pipelines and workflows across the Azure ecosystem. The ideal candidate will bring strong expertise in Azure data services, PySpark, Airflow, SQL, and Python, with the ability to solve complex data integration challenges and deliver high-performing data solutions.

Key Responsibilities :

- Design, build, and maintain ETL/ELT pipelines using Azure Data Lake, ADF, Databricks, and PySpark.

- Develop and orchestrate data workflows using Apache Airflow.

- Optimize PySpark jobs for performance and scalability across large datasets.

- Write efficient and complex SQL queries for data validation, transformation, and reporting.

- Collaborate with data architects, analysts, and business teams to translate requirements into scalable solutions.

- Implement best practices in data modeling, data governance, and security on Azure.

- Monitor, troubleshoot, and optimize pipelines for performance and cost efficiency.

- Mentor junior engineers and contribute to code reviews, design discussions, and technical leadership.

Must Have Skills :

- 6+ years of professional experience in data engineering.

- Strong hands-on experience with Azure Data Services (ADF, ADLS, Synapse, Databricks).

- Proficiency in PySpark for large-scale data processing.

- Expertise in Apache Airflow for workflow orchestration.

- Strong programming skills in Python.

- Advanced knowledge of SQL (complex queries, optimization, performance tuning).

Good to Have :

- Experience with Azure DevOps / CI-CD pipelines.

- Knowledge of Delta Lake and data lakehouse architectures.

- Exposure to Snowflake or other cloud data warehouses.

- Familiarity with Kubernetes (AKS) for containerized data workloads.

Why Join Us?

- Opportunity to work on end-to-end Azure cloud data engineering projects.

- Be part of a high-performance engineering team driving cloud data solutions.

- Hybrid work flexibility and exposure to modern data stack.

- Continuous learning culture with opportunities to upskill in cloud-native and big data technologies.