We are looking for a skilled Data Engineer with strong hands-on experience in Azure and Databricks to build scalable data pipelines and optimize data workflows.

The ideal candidate will be responsible for designing ETL/ELT processes, transforming and enriching data, handling structured and unstructured datasets, and supporting business analytics through efficient data modeling and performance tuning.

Key Responsibilities :

- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks.

- Extract, transform, and load data from multiple data sources (structured & unstructured).

- Clean, validate, and enrich data at various processing stages.

- Develop and optimize data models (dimensional modeling, star schema, etc.).

- Monitor, troubleshoot, and improve performance of data pipelines and SQL queries.

- Ensure data quality, consistency, and governance standards.

- Collaborate with BI, analytics, and business stakeholders to understand requirements.

- Implement best practices for performance tuning and cost optimization in Azure.

Required Skills & Qualifications :

- Strong hands-on experience with Azure Databricks.

- Good understanding of Azure Data Lake, Azure Data Factory, and related Azure services.

- Strong SQL skills (complex joins, aggregations, query optimization).

- Experience with PySpark / Apache Spark.

- Experience building and maintaining ETL/ELT workflows.

- Knowledge of data modeling concepts.

- Experience handling unstructured textual data.

- Strong analytical and problem-solving skills.

Good to Have :

- Experience with Delta Lake.

- CI/CD experience for data pipelines.

- Knowledge of data governance and security best practices.

- Exposure to cloud data warehousing solutions.