Senior Data Engineer - AWS/Python

Job Description :

We are looking for a highly skilled Senior Data Engineer with strong expertise in AWS, Python, and PySpark to join our team.

Key Responsibilities :

- Build scalable PySpark applications using Spark DataFrames in Python.

- Optimize Spark jobs for large-scale data processing.

- Work with AWS analytics services : EMR, Athena, Glue.

- Work with AWS compute/storage services : Lambda, EC2, S3, SNS.

- Collaborate with cross-functional teams to design and implement efficient data pipelines.

- Maintain codebase using Git or other version control systems.

- Ensure data quality, performance tuning, and process improvements.

Required Skills :

- 5+ years of IT experience, with hands-on in Big Data technologies.

- Strong expertise in Python and PySpark.

- Practical experience with data compression & storage formats : Parquet, Avro, ORC, Snappy, Gzip.

- Good understanding of data warehousing concepts dimensions, facts, schemas (snowflake, star).

Good to Have :

- Knowledge of AWS databases such as Aurora, RDS, Redshift, ElastiCache, DynamoDB.