Senior Developer - Data Engineering

Key Responsibilities :

- Develop and optimize data processing pipelines using Apache Spark / PySpark.

- Build and automate ETL workflows using AWS Glue Studio, Glue ETL, and Glue Catalog.

- Manage and maintain data storage solutions using Amazon S3.

- Monitor systems using AWS CloudWatch and ensure high performance and reliability.

- Use AWS DevOps practices to streamline deployments and improve operations.

- Manage AWS IAM roles and policies for secure data access.

- Troubleshoot data issues and perform Spark performance tuning.

- Ensure data quality, accuracy, and integrity across all workflows.

- Work closely with cross-functional teams to integrate data solutions into business processes.

- Stay updated on AWS best practices and emerging technologies.

Required Skills :

• Python
• PySpark / Spark optimization
• SQL
• ETL development

• Glue Studio, Glue ETL, Glue Catalog
• S3
• CloudWatch
• IAM

- Experience with data lake or data warehouse solutions.