Job Title : AWS Spark Scala Data Engineer

Job Summary :

We are seeking a highly skilled AWS Spark Scala Data Engineer to design, develop, and optimize scalable data processing solutions on AWS. The ideal candidate will have strong expertise in Scala, Apache Spark, AWS cloud services, and ETL development, with experience building high-performance batch and streaming data pipelines for enterprise-scale applications.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines using Scala and Apache Spark.

- Build, optimize, and maintain ETL/ELT processes for large-scale data processing.

- Develop Spark applications using Spark Core, Spark SQL, and DataFrames/Datasets.

- Work with AWS services such as S3, EMR, Glue, Redshift, and Athena.

- Optimize Spark jobs for performance, scalability, and resource utilization.

- Develop and manage workflow orchestration using Apache Airflow or equivalent scheduling tools.

- Implement data quality, validation, and monitoring processes.

- Collaborate with Data Architects, Data Analysts, and cross-functional teams to deliver business requirements.

- Troubleshoot production issues and perform root cause analysis.

- Follow coding standards, version control, CI/CD, and best practices for data engineering.

Mandatory Skills :

- Strong hands-on experience with Scala

- Expertise in Apache Spark (Spark Core, Spark SQL, DataFrames/Datasets)

- Strong experience with AWS (S3, EMR, Glue, IAM, Redshift, Athena)

- Advanced SQL and query optimization

- Experience in ETL/Data Pipeline Development

- Experience with Apache Airflow or similar workflow orchestration tools

- Experience with Hive, Parquet, ORC, Delta Lake, or Apache Iceberg

- Strong understanding of distributed computing concepts

- Experience with Git and Linux/Unix environments

Preferred Skills :

- Experience with Apache Kafka or Spark Streaming

- Knowledge of Docker and Kubernetes

- Experience with CI/CD pipelines

- Python knowledge is a plus

- Familiarity with Agile/Scrum methodologies

- Experience with data modeling and data warehousing concepts

Required Qualifications :

- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.

- 58+ years of experience in Data Engineering.

- 3+ years of hands-on experience with Apache Spark and Scala.

- 3+ years of experience working on AWS cloud services.

- Strong analytical, problem-solving, and debugging skills.

- Excellent communication and collaboration skills.

Nice to Have :

- AWS Certified Data Analytics Specialty

- AWS Certified Solutions Architect

- Databricks experience

- Delta Lake implementation experience

Experience :

- 58+ Years

Location :

- [Location]

Employment Type :

- Full-Time

Notice Period :

- Immediate to 30 Days Preferred