Role Overview :

We are seeking a highly skilled and experienced Azure Data Engineer to join our dynamic team in Bangalore. In this role, you will be responsible for designing, building, and maintaining robust and scalable data pipelines and data warehousing solutions on the Azure platform.

You will collaborate closely with data scientists, analysts, and business stakeholders to understand their data requirements and translate them into efficient and reliable data solutions. Your work will directly impact our ability to derive valuable insights from data, enabling data-driven decision-making and improved business outcomes.

Key Responsibilities :

- Design and implement data ingestion pipelines using Azure Data Factory to efficiently extract, transform, and load data from various sources into Azure data storage solutions.

- Develop and maintain data processing workflows using Azure Databricks with Python to perform complex data transformations, cleansing, and enrichment.

- Build and optimize data models and data warehouses in Azure Synapse Analytics and Azure SQL DB to support business intelligence and reporting needs.

- Implement real-time data streaming solutions using Azure Stream Analytics to capture and process data from streaming sources for immediate insights.

- Automate data pipeline orchestration and scheduling using Apache Airflow to ensure timely and reliable data delivery.

- Monitor and troubleshoot data pipeline performance, identify bottlenecks, and implement optimizations to improve efficiency and scalability for all stakeholders.

- Collaborate with data scientists and analysts to understand their data requirements and provide them with the necessary data infrastructure and tools to perform their analyses.

- Ensure data quality and integrity by implementing data validation and monitoring processes throughout the data pipeline to maintain trust in the data.

Required Skillset :

- Demonstrated ability to design, build, and maintain data pipelines and data warehousing solutions on the Azure platform.

- Proven expertise in using Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure SQL DB, and Azure Stream Analytics for data engineering tasks.

- Strong proficiency in Python for data manipulation, transformation, and automation.

- Experience with Apache Airflow for data pipeline orchestration and scheduling.

- Excellent communication and collaboration skills to effectively work with data scientists, analysts, and business stakeholders.

- Ability to understand business requirements and translate them into efficient and scalable data solutions.

- A Bachelor's or Master's degree in Computer Science, Data Science, or a related field is preferred.

- Ability to work effectively in a hybrid work environment, collaborating with both on-site and remote team members.