Position Overview:

We are looking for a Data Engineer with hands-on experience in Apache Airflow, Python, and Jupyter Notebook to build, orchestrate, and maintain robust data pipelines that power our analytics and business decisions. In this role, you will work closely with data teams to clean, transform, and move data reliably across our platforms while ensuring pipeline performance and data integrity at scale.

Key Responsibilities:

Pipeline Architecture:

- Design, build, and maintain scalable ETL/ELT data pipelines using Apache Airflow and Python.

Orchestration & Workflow:

- Schedule, monitor, optimize, and troubleshoot Airflow DAGs to ensure reliable, high-availability data flow.

Data Processing:

- Perform data extraction, cleaning, transformation, and validation across structured and unstructured multi-source environments.

Prototyping & Analysis:

- Use Jupyter Notebook for exploratory data analysis, pipeline testing, data profiling, and rapid prototyping.

Cross-Team Collaboration:

- Partner with data analysts, data scientists, and product teams to translate business requirements into reliable data structures.

Quality & Performance:

- Ensure high data quality, strict data integrity, and pipeline execution speed at scale.

Documentation:

- Maintain clean, clear documentation for all data pipelines, lineage, and operational workflows.

Required Qualifications:

Experience:

- 1 to 2 years of hands-on, production experience in data engineering.

Language Proficiency:

- Strong proficiency in Python for data manipulation, ETL scripting, and automation.

Workflow Orchestration:

- Working experience with Apache Airflow (DAG creation, task dependencies, custom operators, and scheduling).

Prototyping Tools:

- High comfort level working in Jupyter Notebook for data prototyping and interactive analysis.

Data Foundations:

- Solid understanding of SQL, relational/non-relational databases, and data modeling concepts.

Soft Skills:

- Clear communication, analytical problem-solving, and attention to detail.

Preferred Qualifications:

Education:

- B.Tech / B.E. in Computer Science, Data Science, or a related field, preferably from a premier institution (NIT or IIT).

Cloud Infrastructure:

- Familiarity with major cloud environments (AWS, GCP, or Azure).

Big Data Technologies:

- Exposure to distributed computing frameworks like Apache Spark, streaming platforms like Kafka, or cloud data warehouses (e.g., Snowflake, BigQuery, Redshift).

Shift Timing:

- 10:00 AM 7:00 PM IST