Job Description :

As a Data Engineer, you will own the architecture and maintenance of our data pipelines.

You will be responsible for orchestrating complex workflows using Apache Airflow, ensuring that data flows seamlessly from our various on-site sources (IoT devices, ERPs, project management tools) into our central data warehouse.

You will play a critical role in transforming raw construction data into actionable insights that drive project timelines and efficiency.

Key Responsibilities :

- Pipeline Orchestration : Design, build, and maintain robust data pipelines using Apache Airflow. You will write and optimize DAGs to automate ETL processes.

- ETL Development : Develop and manage end-to-end ETL (Extract, Transform, Load) lifecycles. Ensure data is accurately extracted from various APIs and SQL/NoSQL databases, transformed for business logic, and loaded into our data warehouse.

- Data Architecture : Maintain and improve the data architecture to support high-volume data ingestion and processing.

- Quality & Reliability : Implement monitoring and alerting for data pipelines to ensure high availability and data accuracy. Troubleshoot and resolve data issues promptly.

- Cloud Infrastructure : Work with AWS services (S3, Redshift, Glue, Lambda) to build scalable cloud-based data solutions (if applicable).

- Collaboration : Work closely with the construction tech team, data analysts, and software engineers to understand data requirements for our Digital Twin and operational dashboards.

What We Are Looking For :

- Experience : Minimum of 5 years of professional experience in Data Engineering.

- Core Competency (Airflow) : Strong hands-on experience with Apache Airflow. You should be comfortable writing custom operators, managing DAG dependencies, and troubleshooting scheduling issues.

- ETL Expertise : Proven track record of building and optimising ETL/ELT pipelines.

- Programming : Strong proficiency in Python (for scripting and Airflow) and SQL (for database querying and transformation).

- Database Knowledge : Experience with relational databases (PostgreSQL, MySQL) and Data Warehousing concepts.

Nice to Have (Plus) :

- AWS Experience : Hands-on experience with the AWS ecosystem, specifically S3, Redshift, Glue, or EMR.

- Experience with containerization tools like Docker or Kubernetes.

- Familiarity with Real Estate/Construction domain data or IoT data streams.

Experience Range : 5 - 8 years

Educational Qualifications : B.Tech/B.E

Skills Required : AWS , Airflow , MySQL , ETL