DevOn Software Services - Azure Data Engineer

Role : Azure Data Engineer

Experience Range : 5- 8 years

Location : Bangalore

Job Description :

Role & Responsibilities :

Data Management and Storage :

- Design and implement data storage solutions using Azure SQL Database, Azure Data Lake Storage and Azure Blob Storage.

- Ensure data storage systems are scalable, performant, and cost-effective.

Data Integration and ETL :

- Develop and manage ETL (Extract, Transform, Load) processes using Azure Data Factory.

- Ensure efficient data cleaning, transformation, and loading into Data Lakes.

Big Data and Analytics :

- Utilize big data technologies like Azure Databricks and Apache Spark for data processing and analytics.

- Create data processing workflows and pipelines to support data analytics and machine learning applications.

Data Pipeline Architecture :

- Design, construct, and maintain data pipelines and combine large, complex datasets that meet business requirements.

Optimization and Monitoring :

- Implement effective metrics and monitoring processes to ensure data quality and system performance.

- Identify and implement internal process improvements, such as automating manual processes and optimizing data delivery.

Collaboration and Support :

- Work with stakeholders to address data-related technical issues and support data-driven decision-making.

- Collaborate with data and analytics experts to enhance data system functionality.

ML Use Cases :

- Collaborate with data scientists to create datasets for variety of use cases .

- Implement data annotations, curation techniques to enhance data readiness for ML Applications.