We are looking for an experienced Azure Data Engineer with 8 to 12 years of experience in designing, developing, and maintaining scalable data solutions on Microsoft Azure. The ideal candidate should have strong expertise in Azure Databricks, Azure Data Factory, ADLS, SQL, Python/PySpark, and ETL processes, along with an understanding of ERP data and financial data environments.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines and ETL/ELT workflows on Azure.

- Develop data transformation and processing solutions using Azure Databricks, Python, PySpark, and SQL.

- Build and optimize Databricks notebooks for data transformation, cleansing, aggregation, and table creation.

- Develop and maintain data ingestion pipelines using Azure Data Factory (ADF).

- Work with Azure Data Lake Storage (ADLS) for scalable storage and management of structured and unstructured data.

- Integrate data from multiple sources, including ERP systems, databases, APIs, and enterprise applications.

- Understand ERP data structures, business processes, and data flows and translate them into scalable data solutions.

- Implement data quality, validation, reconciliation, and error-handling processes across data pipelines.

- Optimize data pipelines and queries for performance, scalability, reliability, and cost efficiency.

- Work with Alteryx and Power BI to support data analysis, reporting, and visualization requirements.

- Collaborate with business, analytics, finance, risk, and technology teams to understand requirements and deliver data solutions.

- Support data governance, security, audit, and compliance requirements.

- Troubleshoot data pipeline and production issues and perform root-cause analysis.

- Create technical documentation covering data pipelines, transformations, data flows, and operational procedures.

- Contribute to cloud modernization and data platform improvement initiatives.

Required Skills & Experience :

- 8 to 12 years of experience in Data Engineering, ETL, or related data technologies.

- Strong hands-on experience with Microsoft Azure data services.

- Strong expertise in Azure Databricks and Databricks Notebooks.

- Proficiency in SQL and strong programming experience in Python and/or PySpark.

- Hands-on experience with Azure Data Factory (ADF).

- Strong experience with Azure Data Lake Storage (ADLS).

- Strong understanding of ETL/ELT concepts, data integration, data transformation, and data warehousing.

- Experience designing and implementing data pipelines using multiple data sources.

- Good understanding of Azure cloud services, security, networking, and data platform architecture.

- Experience working with ERP data and ERP data flows.

- Hands-on experience with Alteryx and Power BI for data analysis and visualization.

- Strong understanding of data quality, reconciliation, validation, and performance optimization.

- Good analytical, problem-solving, and stakeholder-management skills.