Description :

Job Title : Senior Data Engineer

Location : Pune

Experience : 5.5 Years

Employment Type : Full-Time

Job description :

Role : Senior Data Engineer (Azure/Databricks)

Overview :

We are seeking a Senior Data Engineer with 5+ years of hands-on experience in building and scaling data solutions within the Azure ecosystem. The ideal candidate will have deep expertise in Databricks, PySpark, and modern data lake architectures, and be adept at designing robust ETL pipelines and data workflows.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines using Azure and Databricks.

- Build efficient data ingestion and transformation workflows with PySpark and Delta Lake.

- Develop, optimize, and maintain ETL/ELT pipelines using Python and SQL.

- Implement data modeling, data quality, and governance best practices.

- Work with relational databases (e.g., PostgreSQL) and lakehouse architectures (e.g., Delta Lake).

- Collaborate with cross-functional teams to troubleshoot, optimize, and support production workloads.

Required Skills & Experience :

- 5+ years of data engineering experience.

Strong proficiency in the Azure ecosystem, including :

- Azure Databricks

- Azure Data Lake Storage (ADLS)

- Azure Functions

- Azure Data Factory (preferred)

- Advanced PySpark and SQL expertise, including performance tuning.

- Deep understanding of ETL/ELT design patterns and data warehousing concepts.

- Experience with Delta Lake (ACID transactions, schema evolution, time travel).

- Hands-on experience with PostgreSQL or similar RDBMS.

- Strong analytical, problem-solving, and communication skills.

- Self-motivated and comfortable in fast-paced, agile environments.

Preferred Skills :

- Familiarity with CI/CD pipelines (Azure DevOps, Git).

- Knowledge of Infrastructure-as-Code tools (Terraform).

- Exposure to real-time streaming technologies (Kafka, Event Hub).

- Awareness of data governance tools (e.g., Microsoft Purview).

Technical Stack :

Languages & Tools : Python, PySpark, SQL

Platforms : Azure Databricks, ADLS, Azure Functions, ADF

Databases : PostgreSQL, Delta Lake

DevOps & Governance : Azure DevOps, Git, Terraform, Purview