Data Engineer - AWS Platform

Description :

We are looking for a skilled AWS Data Engineer to design, build, and maintain scalable data pipelines and cloud-based data solutions on AWS.

The ideal candidate should have hands-on experience with AWS data services, ETL processes, data warehousing, and big data technologies.

Key Responsibilities :

- Design and develop scalable ETL/ELT data pipelines using AWS services.

- Build and manage data lakes and data warehouses on AWS.

- Develop data integration solutions using tools like AWS Glue, Lambda, and Step Functions.

- Work with structured and unstructured data from multiple sources.

- Optimize data processing workflows for performance and cost efficiency.

- Implement data quality, governance, and security best practices.

- Collaborate with Data Analysts, Data Scientists, and Business teams for data requirements.

- Monitor and troubleshoot production data pipelines and workflows.

- Automate deployment and infrastructure using CI/CD and Infrastructure as Code tools.

Required Skills :

- Strong experience with AWS services such as :

1. Amazon Web Services S3

2. Redshift

3. Glue

4. Lambda

5. EMR

6. Athena

7. Kinesis

8. IAM

9. CloudWatch

- Proficiency in SQL and Python/Scala/Java.

- Experience with ETL tools and data pipeline development.

- Knowledge of big data technologies like Spark and Hadoop.

- Hands-on experience with data warehousing concepts and dimensional modeling.

- Experience with orchestration tools such as Airflow.

- Familiarity with DevOps and CI/CD practices.

- Understanding of data security and compliance standards.

Preferred Qualifications :

- Bachelors degree in Computer Science, Engineering, or related field.

- AWS Certification such as :

1. AWS Certified Data Engineer

-

2. AWS Certified Solutions Architect

- Experience with Snowflake or Databricks is an added advantage.

- Good communication and stakeholder management skills.