- Python Or PySpark.

Roles and Responsibilities :

- Design, develop, and maintain large-scale data pipelines using AWS services such as AWS Glue, Lambda, EMR, and Redshift.

- Develop scalable and efficient data models using PySpark and Spark SQL to process massive datasets.

- Architect and deploy scalable data pipelines using AWS Glue and PySpark to ensure high-quality data ingestion and transformation for downstream analytics.

- Design and maintain high-performance data warehouses in Amazon Redshift to provide stakeholders with reliable, real-time insights for strategic decision-making.

- Optimize serverless computing workflows using AWS Lambda and ECS to reduce operational overhead and improve the efficiency of our data processing tasks.

Desired Candidate Profile :

- 7- 12 years of experience in Data Engineering with expertise in Python programming language.

- Strong understanding of AWS ecosystem including AWS Glue, Lambda, EMR, Redshift, EC2, ECS, S3 etc.

- Experience with big data technologies like Hadoop/Hive/Pyspark/SparkSQL.