Responsibilities :

- Establish scalable, efficient, automated processes for data analysis, data model development, validation, and implementation.

- Work closely with analysts/data scientists to understand impact to the downstream data models.

- Write efficient and well-organized software to ship products in an iterative, continual release environment.

- Contribute to and promote good software engineering practices across the team

- Communicate clearly and effectively to technical and non-technical audiences.

Minimum Qualifications :

- University or advanced degree in engineering, computer science, mathematics, or a related field

- Strong hands-on experience in Databricks using PySpark and Spark SQL (Unity Catalog, workflows, Optimization techniques)

- Experience with at least one cloud provider solution (GCP preferred)

- Strong experience working with relational SQL databases.

- Strong experience with object-oriented/object function scripting language : Python.

- Working knowledge in any transformation tools, DBT preferred.

- Ability to work with Linux platform.

- Strong knowledge of data pipeline and workflow management tools (Airflow)

- Working knowledge of Git hub /Git Toolkit

- Expertise in standard software engineering methodology, e.g. unit testing, code reviews, design documentation

- Experience creating Data pipelines that prepare data for ingestion & consumption appropriately.

- Experience in maintaining and optimizing databases/filesystems for production usage in reporting, analytics.

- Working in a collaborative environment and interacting effectively with technical and non-technical team members equally well.