Location : Gurgaon (On-site)

Experience : 810 Years

Must Have Skills :

- Apache Airflow (810 yrs)

- AWS Data Services (810 yrs)

- Data Lake Architecture & Management

- Data Pipeline Design & Orchestration

- Advanced SQL Querying

Roles and Responsibilities :

- Design, develop and own robust data pipelines, ensuring optimal performance, scalability, and maintainability.

- Design and implement Data Lake, Dare Warehouse and Lakehouse solutions with different architecture patterns.

- Ensure data quality, integrity and governance across all stages of data lifecycle.

- Monitor and optimize performance of data engineering pipelines.

- Contribute to design principles, best practices, and documentation.

- Collaborate closely with cross-functional teams to deeply understand business requirements, translating them into effective technical design, implementations that support the organization's data-driven initiatives.

- Provide mentorship and guidance to other team members of the data engineering team, promoting knowledge transfer, a culture of continuous learning and skills development.

Requirements :

- Bachelor's degree in computer science, Information Systems, or a related field Master's degree is a plus.

- A seasoned Data Engineer with a minimum of 7-8 years of experience.

- Deep experience in designing and building robust, scalable data pipelines both batch and real-time using modern data engineering tools and frameworks.

- Proficiency in AWS Data Services (S3, Glue, Athena, EMR, Kinesis etc.).

- Strong grip on SQL queries, various file formats like Apache Parquet, Delta Lake, Apache Iceberg or Hudi and CDC patterns.

- Experience in stream processing frameworks like Apache Flink or Kafka Streams or any other distributed data processing frameworks like pySpark.

- Expertise in workflow orchestration using Apache Airflow.

- Strong analytical and problem-solving skills, with the ability to work independently in a fast-paced environment.

- In-depth knowledge of database systems (both relational and NoSQL) and experience with data warehousing concepts.

- Hands-on experience with data integration tools and a strong familiarity with cloud-based data warehousing and processing is highly desirable.

- Excellent communication and interpersonal skills, facilitating effective collaboration with both technical and non-technical stakeholders.

- A strong desire to stay current with emerging technologies and industry best practices in data landscape.