Data Engineer - Azure Databricks/PySpark

Description :

Desired Competencies (Technical/Behavioral Competency)

Must-Have :

- Must have experience in Object oriented programming, data structures, dependency injection and multi-threading or background in ETL, RDBMS

- Experience in Cloud Data Warehouse tools (Azure Data Factory & Azure Databricks)

- Experience working with GIT as source control and should understand the concepts around pull requests, branches, repositories.

- Experience with Microsoft Azure (Azure DATABRICKS , SQL, U-SQL, T-SQL, Storage, Web apps, web roles, worker roles, Service Fabric. purview)

- Data warehousing concepts

- Executed, hands on azure migration or development

Good-to-Have :

- Nice to have hands on ETL knowledge

- Ability to write clear and concise technical documents

- Banking domain experience will be an advantage

- Ability to analyze data and develop strategies for populating data lakes

Responsibility of / Expectations from the Role :

1. Understand business requirement & banking domain

2. Responsible for preparing application design document

3. Responsible for developing code & unit testing

4. Good Analytical & Debugging Skill

Responsibilities :

- Design, develop, and deploy data solutions using Azure Data Factory, Azure Databricks, and related technologies.

- Collaborate with cross-functional teams to understand business requirements and deliver customized solutions.

- Develop and maintain data pipelines, data models, and data visualizations using Azure Databricks, Azure Synapse Analytics, and other tools.

- Implement best practices for data engineering, including data security, data quality, and data governance.

- Troubleshoot and resolve technical issues, ensuring high availability and performance of data solutions.