Big Data Engineer - Python/Spark

Description :

Role & responsibilities :

- Lead the design and development of new Data & Analytics solutions in various parts of the product

- Modernize existing system components by re-designing them according to the new

architecture paradigm.

- Integrate developed solutions with other components of the system

- Working closely with the Architecture team to influence new technologies choice

- Ensure the non-functional requirements (ex. stability, scalability, and performance) of services and applications are met

- Mentor and guide other team members in best practices and technical decisions

- Being a part of a scrum team collaborating with other cross-functional teams to identify and solve complex technical problems

- Understand business domain, customers needs, and business use cases

Preferred candidate profile :

Overall working experience in Data Engineering 5+ years :

Strong knowledge of Python, Apache Spark, and Airflow with a minimum of 3+ years of practical experience :

- Solid understanding of SQL

- Strong experience with relational and non-relational databases

- Solid understanding of data models and data pipeline design (batch and real-time)

- Strong Experience developing solutions in a Cloud environment

- Strong understanding of DevOps methodologies

- Working experience in an Agile environment

- A natural interest in modern data processing technologies and the ability to learn new things

- Passion to get into the development process quickly and deliver good-quality code

- English level at B2 or higher

Nice to have :

- Experience with Snowflake

- Understanding of distributed architecture features and challenges

- Experience working in a containerized environment (Docker, Kubernetes)

- Experience in implementing projects & solutions using GCP Stack (Dataproc, Composer, GCS)

- Knowledge of real-time streaming platforms (Kafka, RabbitMQ and etc)

- Have experience with AI, machine learning & statistical tools