Moolya - Data Engineer

Description :

Job Title : Data Engineer

Experience : 5 Years

Employment Type : Full-Time

Job Summary :

We are looking for an experienced Data Engineer with around 5 years of hands-on experience in designing, building, and optimizing scalable data pipelines and data platforms. The ideal candidate should be strong in distributed data processing, cloud data solutions, and modern data engineering tools.

Key Responsibilities :

- Design, develop, and maintain ETL/ELT pipelines for structured and unstructured data.

- Build scalable and reliable data pipelines using PySpark / Spark / SQL.

- Develop and optimize workflows using Apache Airflow or similar orchestration tools.

- Work with various data sources and connectors such as JDBC, Hive, Impala, S3, Kafka, etc.

- Ensure data quality, accuracy, and consistency across systems.

- Collaborate with Data Scientists, Analysts, and Product teams to support data needs.

- Implement data validation, monitoring, and performance improvements.

- Work with cloud platforms like AWS / Azure / GCP for data storage, processing, and automation.

- Build and maintain data models, data marts, and data warehousing solutions.

- Troubleshoot data issues, optimize queries, and ensure robust data operations.

Required Skills & Experience :

- 5+ years of experience as a Data Engineer.

- Strong experience in PySpark, Spark, SQL.

- Hands-on experience with Airflow (or any workflow scheduler).

- Experience working with JDBC connectors, Hive, Impala, and other large-scale data sources.

- Strong understanding of data warehousing concepts, ETL design, and performance tuning.

- Experience with cloud services (AWS/Azure/GCP).

- Good knowledge of CI/CD for data pipelines.

- Strong analytical and problem-solving skills.

Preferred Skills :

- Experience with Big Data ecosystems (Hadoop, Kafka).

- Experience with containerization (Docker, Kubernetes).

- Knowledge of Data Lake / Lakehouse architectures.

- Familiarity with version control (Git) and DevOps practices.