CarbyneTech - Data Engineer

Description :

- Design, develop, and maintain scalable data pipelines using Azure Data Factory, Databricks, and SQL Server.

- Implement ETL/ELT processes to ingest, transform, and load data from various sources (on-prem, cloud, APIs).

- Optimize data storage and retrieval using Azure Data Lake, Delta Lake, and SQL-based solutions.

- Collaborate with tech-leads to define data models, schemas, and follow standards.

- Monitor and troubleshoot data workflows, ensuring reliability and performance.

- Work with DevOps to Implement CI/CD pipelines for data engineering solutions using Azure DevOps.

- Ensure data security and compliance with enterprise and regulatory standards.

Required Skills & Qualifications :

- 7+ years of experience in data engineering, with at least 3 years in Azure cloud environments.

- Strong proficiency in Azure Data Factory, Azure Databricks, Azure SQL Server, and Azure Data Lake.

- Solid understanding of SQL, Python, and Spark for data processing and transformation.

- Experience with Delta Lake.

- Familiarity with CI/CD practices and tools (e. Azure DevOps).

- Knowledge of data governance, security, and compliance best practices.

- Excellent problem-solving and communication skills