Job Title: Data Engineering Specialist

Experience: 8+ Years

Location: Pune

Role Summary:

We are looking for an experienced Data Engineering Specialist to design, develop, and optimize scalable data ingestion and ETL pipelines across the Hadoop ecosystem. The ideal candidate should have strong hands-on experience with Apache NiFi, Spark/PySpark, Hadoop/HDFS, Apache Airflow, Cloudera Data Engineering (CDE), Hive, SQL, and Python.

Mandatory Skills:

- Apache NiFi

- Spark / PySpark

- Hadoop / HDFS

- Apache Airflow

- Cloudera Data Engineering (CDE)

- Hive & SQL

- Python

Key Responsibilities:

- Design, develop, and maintain scalable ETL and data ingestion pipelines.

- Build and optimize data processing solutions using Spark, PySpark, and NiFi.

- Develop and manage Airflow workflows for scheduling and orchestration.

- Work with Hadoop, HDFS, Hive, and CDE for enterprise data processing.

- Write efficient Hive SQL and Python code for data transformation and automation.

- Monitor, troubleshoot, and optimize production data pipelines.

- Collaborate with cross-functional teams to deliver reliable data solutions.

Required Skills:

- 8+ years of experience in Data Engineering and Big Data technologies.

- Strong expertise in NiFi, Spark/PySpark, Hive SQL, Python, and Airflow.

- Good understanding of Hadoop, HDFS, distributed data processing, and ETL best practices.

- Hands-on experience with Cloudera Data Engineering (CDE) or similar enterprise data platforms.

- Experience supporting and optimizing production data pipelines.

- Strong analytical, problem-solving, and communication skills.