Big Data Engineer

Job Description :

The successful candidate will be responsible for working with a range of big data technologies, including Apache Spark, Hadoop, and NoSQL databases. They will be expected to have strong programming skills in languages such as Python and Java, as well as experience with workflow orchestration tools and containerization.

Key Responsibilities :

- Apache Spark, Hadoop, Hive, HDFS.

- SQL & NoSQL databases (Postgres, Cassandra, MongoDB, HBase).

- Strong programming skills in Python, pySpark (Spark Core) & Java.

- Experience with workflow orchestration tools (Airflow, Oozie).

- Proficiency in containerization & CI/CD (Docker, GitLab CI/CD).

- Good understanding of data modelling, governance, and security practices.

- Strong problem-solving and performance tuning skills.

- Hands-on with cloud platforms (Azure preferred).

- Worked on Kafka / Flink / Storm (real-time streaming).

Required Skills :

- Strong programming skills in Python, pySpark, and Java

- Experience with Apache Spark, Hadoop, Hive, and HDFS

- Knowledge of SQL and NoSQL databases, including Postgres, Cassandra, MongoDB, and HBase

- Understanding of data modelling, governance, and security practices

- Experience with workflow orchestration tools and containerization