Specialist - Data Engineering

Job Description

Senior Specialist - Data Engineering

Job Summary :

We are seeking a highly skilled Senior Specialist - Data Engineering with strong Spark Scala knowledge, mandatory for a significant and complex engagement. An AWS background is welcome.

As a Big Data Full Stack Developer, you will work with various underlying tools such as Spark/Scala, Oozie, Postgre, and Couchbase. The role requires collaborating with architects to implement the application solution successfully.

Job Responsibilities :

- Modify and enhance the underlying Scala framework, as required.

- Work with Spark Streaming jobs, handle in-memory Spark state, and process messages for near-real-time systems.

- Optimize Spark queries for improved performance.

- Integrate with SQL databases, especially Postgre and Couchbase.

- Collaboratively solve technical problems within specified timelines with minimal guidance.

- Implement various engineering tasks, including gap analysis, data analysis, and impact analysis.

- Independently perform low-level design configuration, code customization, refactoring, and code reviews.

- Demonstrate expertise in design, development, testing, build, and deployment stages of the software lifecycle.

- Ensure thorough documentation and provide reliable engineering support to all stakeholders.

Required Skills :

- Mandatory expertise in Spark Scala.

- AWS background and familiarity with Big Data technologies.

- Proficiency in modifying and enhancing Scala frameworks.

- Experience with Spark Streaming and in-memory processing.

- Strong SQL database integration skills, particularly with Postgre and Couchbase.

- Problem-solving abilities and the ability to meet project deadlines.

- Effective collaboration with team members and architects.

- Documentation skills for maintaining comprehensive project records