Role: Azure Data Engineer

Job location: Trivandrum (Interested for this location only apply)

Key Responsibilities:

- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks.

- Build robust ETL/ELT solutions using PySpark and Python for large-scale data processing.

- Implement and manage Unity Catalog for data governance, security, and access management.

- Develop and maintain Delta Live Tables (DLT) pipelines to support reliable and efficient data ingestion and transformation.

- Integrate data from multiple enterprise sources, including TIBCO, On-Premises SQL Server, SharePoint, and Shared Folders.

- Collaborate with business stakeholders and technical teams to understand data requirements and deliver high-quality solutions.

- Implement CI/CD pipelines and automate deployments using Azure DevOps.

- Support Azure cloud infrastructure, release management, and production deployments.

- Optimize data pipelines for performance, scalability, reliability, and cost efficiency.

- Ensure adherence to enterprise security, governance, and compliance standards.

- Troubleshoot production issues and provide ongoing support for data platform operations.

Required Technical Skills:

1. Azure Technologies:

- Azure Databricks

- Unity Catalog

- Delta Live Tables (DLT)

- Azure Data Factory (ADF)

- Azure Data Lake Storage (ADLS)

- Azure DevOps (CI/CD, Repos, Pipelines, Release Management)

- Strong Azure Infrastructure and Release Engineering experience

2. Programming:

- Python

- PySpark

- SQL

3. Data Integration:

- Experience working with: TIBCO, On-Premises SQL Server, SharePoint, and Shared Folder/File-based integrations.

Required Experience:

- Azure Databricks: 5+ years

- Azure Data Factory: 5+ years

- Azure DevOps: 5+ years

- Unity Catalog: 2+ years

- Strong experience with enterprise-scale Azure data platform implementations.

- Experience designing secure, scalable, and high-performance data solutions.