Azure Data Engineer - Python/SQL

Job Title : Azure Data Engineer

Responsibilities :

1. Data Solutions Design and Implementation :

- Design and implement modern data architectures using Azure Data Services such as Azure SQL Database, Azure Data Lake, Azure Data Factory, and Azure Databricks.

- Develop, construct, test, and maintain architectures such as databases and large-scale processing systems.

- Ensure solutions are scalable, resilient, and efficient to meet the needs of the business.

2. Data Integration and Orchestration :

- Utilize Azure Data Factory to create data pipelines for data transformation and integration.

- Implement data flows to connect operational systems, data for analytics and BI systems.

- Design and implement event-driven architectures using Azure Event Hubs and Azure Stream Analytics.

3. Data Storage and Optimization :

- Manage and optimize data storage using Azure Blob Storage, Azure Data Lake Storage, and other Azure storage options.

- Implement enterprise data warehouses in Azure Synapse Analytics to support business intelligence tools.

- Optimize data retrieval and perform tuning on SQL databases to ensure performance and scalability.

4. Security and Compliance :

- Implement security measures to protect data at rest and in transit.

- Ensure compliance with data governance and security policies.

- Manage and monitor data permissions and access within Azure environments.

5. Collaboration and Support :

- Work closely with data scientists, business analysts, and other stakeholders to ensure their data needs are met.

- Support migration of data from legacy systems to Azure cloud.

- Provide training and support to data users on new solutions.

Skills and Qualifications :

Education :

- Bachelors degree in Computer Science, Engineering, or a related field.

Technical Skills :

- Strong proficiency with Azure cloud services related to data and analytics (Azure SQL, Data Lake, Data Factory, Databricks, etc.).

- Experience with SQL and data modeling, as well as familiarity with NoSQL databases.

- Proficiency in programming languages such as Python or Scala.

- Knowledge of data warehousing and data lake concepts and technologies.

Analytical Skills :

- Ability to analyze complex data structures and derive insights.

Communication Skills :

- Excellent communication skills to effectively collaborate and convey complex solutions.

Problem-Solving :

- Strong problem-solving abilities to address data-related challenges.

Experience :

- Proven experience as a Data Engineer, preferably with a focus on Azure cloud technologies.

- Hands-on experience designing and delivering scalable and secure data solutions in Azure.

Work Environment :

- Office setting, with potential for remote work depending on company policy.

- Collaborative environment with frequent project team interactions.

- Requires staying updated with continuous innovations in cloud data technologies.