Job Role : Senior Data Engineer.

Experience : 6+ Years.

Location : South Bangalore, KA.

Key Responsibilities :

- Design, develop, and optimize ETL/ELT pipelines using Azure Databricks and Azure Data Factory.

- Build scalable batch and streaming data processing solutions using Python and PySpark.

- Implement and maintain Delta Lake and Lakehouse architectures.

- Design and manage data models, including Dimensional Modeling and SCD Type 1 & Type 2 implementations.

- Develop data storage solutions using Azure Data Lake Storage (Gen2, Delta Tables, and Parquet formats).

- Integrate external systems through REST APIs and secure authentication mechanisms.

- Build and maintain CI/CD pipelines using Azure DevOps and Git.

- Implement data governance, security, and access control best practices.

- Optimize SQL queries, data processing performance, and workload efficiency.

- Collaborate with cross-functional teams to deliver scalable and reliable data solutions.

Required Skills :

- 6+ years of experience in Data Engineering.

- Strong hands-on experience with Azure Databricks, Python, and PySpark.

- Experience with Azure Data Factory (ADF) and Azure Data Lake Storage.

- Strong knowledge of Azure DevOps, Git, and CI/CD implementation.

- Experience with REST API integrations and authentication mechanisms.

- Strong SQL skills including query optimization, CTEs, and window functions.

- Experience with Delta Lake concepts including ACID transactions, Time Travel, Vacuum, and Z-Ordering.

- Understanding of distributed Data Processing, Data Governance, and security best practices.

- Experience with Databricks Asset Bundles (DABs).

- Exposure to event-driven architectures and streaming data solutions.