Hadoop Developer - Big Data Technologies

Job Title : Hadoop Developer (Python / PySpark / Scala / Advanced SQL)

Experience : 5 to 8 Years

Notice Period : Immediate to 15 Days

Location : Any Brillio Location Bangalore, Pune, Hyderabad, Chennai, Gurugram

Role Overview :

We are looking for an experienced Hadoop Developer with strong programming skills and a deep understanding of Big Data ecosystems. The ideal candidate will be proficient in Python, Java, or Scala, have hands-on experience with Spark and Hadoop, and be highly skilled in advanced SQL.

This role involves developing and maintaining large-scale data processing pipelines, collaborating with multiple stakeholders, and ensuring data quality, performance, and compliance.

Key Responsibilities :

- Develop, maintain, and enhance Big Data ETL pipelines using Hadoop ecosystem tools.

- Perform data analysis, profiling, and quality checks to support business and analytics teams.

- Collaborate with data scientists, business analysts, and cross-functional teams to align engineering solutions with business goals.

- Automate data ingestion, transformation, and validation workflows to improve efficiency.

- Monitor, troubleshoot, and optimize data pipelines for performance, scalability, and reliability.

- Ensure data security, governance, and compliance with Mastercard and industry standards.

Must-Have Skills :

- Programming : Python / Java / Scala

- Big Data Frameworks : Apache Spark, Hadoop

- Database Skills : Advanced SQL (including performance tuning and query optimization)

Good-to-Have Skills :

- Experience with Databricks for collaborative data engineering.

- Familiarity with AWS cloud services for big data workloads.

Qualifications :

- Bachelors degree in Computer Science, Information Technology, or related field.

- 5 to 8 years of relevant experience in Big Data development.

- Proven ability to handle large datasets and optimize ETL workflows.

- Strong problem-solving and communication skills with the ability to work in a fast-paced environment.