Spark Core/Streaming with programming language Scala also in Machine Learing OR AI, Good Understanding of Data Structures, Algorithms, Data Transformation, Data Ingestion, Optimization mechanism/techniques, Good understanding of Big Data (Hadoop, MapReduce, Kafka, Cassandra) Technologies

Job Summary : We are looking for a candidate with strong experience into Spark Core/Streaming, Spark RDD API, Spark Dataframes API with Any programming language Scala/ Java/ Python, with Machine Learning and AI, Should have hands-on programming experience and Good Understanding of Data Structures, Algorithms, Data Transformation, Data Ingestion, Optimization mechanism/techniques, Good understanding of Big Data (Hadoop, MapReduce, Kafka, Cassandra) Technologies.

Key Responsibilities :

- Design and develop scalable and high-performance data processing applications using Apache Spark (Core, Streaming) with Scala.

- Work with Spark RDD and DataFrame APIs to build robust data pipelines.

- Apply machine learning or AI techniques to solve real-world data problems.

- Perform data ingestion from various structured and unstructured sources.

- Optimize Spark jobs for performance and resource usage.

- Collaborate with cross-functional teams including data scientists, data engineers, and architects.

- Ensure high code quality through unit testing and code reviews.

- Design and implement data transformation, data ingestion, and ETL pipelines.

Required Skills and Experience :

- 4+ years of experience in Big Data technologies with strong hands-on development in Apache Spark (Core/Streaming).

- Strong proficiency in Scala (preferred), with knowledge of Java or Python.

- Solid understanding and experience with Spark RDD API and DataFrame API.

- Working experience with Machine Learning or AI frameworks and use cases.

- Strong understanding of data structures, algorithms, and performance optimization techniques.

- Experience with data ingestion, transformation, and processing pipelines.

We deal with huge amounts of data at a massive scale, so we are looking for engineers who love solving challenging problems through conducting independent research and collaborating with teams across our product teams to help improve the overall product experience.