Roles & Responsibilities :

Design and Implement Data Pipelines :

- Develop and maintain robust data pipelines for ingesting, processing, and storing data from various sources.

Azure Services Expertise :

- Utilize Azure services like Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure Data Lake Storage, and Azure SQL Database.

Data Modeling and Architecture :

- Design and implement data models, ensuring data quality, consistency, and performance.

ETL Processes :

- Develop and optimize ETL (Extract, Transform, Load) processes using tools like Azure Data Factory or similar.

Programming Skills :

- Proficiency in Python, SQL, and potentially Scala or R for data manipulation and processing.

Big Data Technologies :

- Experience with big data technologies such as Spark, Hadoop, or similar.

- Cloud Infrastructure :

- Understanding of Azure cloud infrastructure services like networking, storage, and security.

Collaboration and Communication :

- Work effectively with cross-functional teams, including data scientists, analysts, and other engineers.

Continuous Improvement :

- Identify opportunities to optimize and improve existing data pipelines and processes.

Mentorship :

- Guide and mentor junior data engineers.

Key Technical Skills Required :

- 5+ years of experience as a data engineer or similar role, with a focus on Azure.

- Proficiency in Azure Data Factory, Azure Synapse, and Azure Databricks.

- Strong programming skills in Python and SQL.

- Experience with big data technologies like Spark.

- Experience with data modeling and ETL processes.

- Understanding of cloud infrastructure and security best practices.

- Excellent communication and collaboration skills.