Data Engineer - Python/PySpark

Description :

- 7-12 years technical experience (within financial services industry preferred)

- Solid experience in Data Engineering, BI/software development such as ELT/ETL, data extraction and manipulation in Data Lake/Data Warehouse/Lake House environment.

- Hands on programming experience in writing Python, SQL, Unix Shell scripts, Pyspark scripts, in a complex enterprise environment

- Hands on cloud-based solution design, configuration and development experience with Azure and AWS

- Hands on experience of using AWS Services - S3,EC2, EMR, SNS, SQS, Lambda functions, Redshift

- Hands on experience Of building Data pipelines to ingest, transform on Databricks Delta Lake platform from a range of data sources - Data bases, Flat files, Streaming etc..

- Quality engineering development experience (CI/CD Jenkins, Docker)

- Experience in Terraform, Kubernetes and Docker Experience with Source Control Tools Github or BitBucket