Role Overview :

We are seeking a highly skilled and motivated AWS Kafka Data Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and maintaining our data streaming infrastructure on AWS, leveraging Kafka and PySpark.

Key Responsibilities :

- Design and implement scalable and fault-tolerant data pipelines using Kafka, PySpark, and other AWS services to ingest, process, and transform large volumes of data for various analytical and operational needs.

- Develop and maintain Kafka topics, producers, and consumers, ensuring optimal performance and reliability for real-time data streaming to support time-sensitive business applications.

- Build and deploy PySpark jobs on AWS EMR or other suitable platforms to perform complex data transformations and aggregations, enabling data scientists and analysts to derive valuable insights.

Required Skillset :

- Demonstrated ability to design, develop, and maintain data streaming pipelines using Apache Kafka and related technologies.

- Proven expertise in developing and deploying PySpark applications for data processing and transformation at scale.

- Strong understanding of AWS services such as EC2, S3, EMR, Lambda, and Kinesis, and the ability to leverage them to build robust and scalable data infrastructure.

- Excellent problem-solving and analytical skills, with the ability to identify and resolve performance bottlenecks and data quality issues.

- Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams and stakeholders.

- Bachelor's or Master's degree in Computer Science or a related field.

- Ability to work in a hybrid work environment.