Description
:

Job Summary
:

We are looking for an experienced Senior Python Developer with strong expertise in designing, developing, and deploying scalable applications. The ideal candidate should have hands-on experience in Python-based frameworks, backend development, APIs, and cloud technologies, along with a solid understanding of software engineering best practices.

Key Responsibilities
:

- Design, develop, and optimize scalable data pipelines and ETL processes using Azure Data Services and Databricks.

- Implement data transformations and analytics using PySpark and Python.

- Configure and manage Databricks Unity Catalog for secure data governance and access control.

- Orchestrate workflows using Apache Airflow for scheduling and automation.

- Collaborate with cross-functional teams to deliver high-quality data solutions in an Agile environment.

- Set up and maintain CI/CD pipelines for data workflows and deployments.

- Manage version control and branching strategies using Git.

- Write and execute unit tests to ensure code quality and reliability.

- Monitor, troubleshoot, and optimize data processes for performance and cost efficiency.

Role Requirements and Qualification
:

- Azure Knowledge
:
Strong experience with Azure Data Factory, Azure Data Lake, Azure Synapse, or related services.

- Programming
:
Proficiency in Python and PySpark for data engineering tasks.

- Databricks
:
Hands-on experience in building and managing workflows in Databricks, including Unity Catalog.

- Orchestration
:
Experience with Apache Airflow for workflow scheduling and automation.

- CI/CD
:
Familiarity with tools like Azure DevOps or Jenkins for automated deployments.

- Version Control
:
Expertise in Git for code management.

- Testing
:
Strong understanding of unit testing frameworks for data pipelines.

- Agile Methodology
:
Experience working in Agile teams and using tools like Jira or Azure Boards.

- 5+ years of relevant experience in Python development.

Mandatory skills
:

- Python, Databricks, git

Desired skills
:

- Deep expertise in python

- Experience with Databricks, pyspark and Git.

- Experience with Azure cloud platform