HamburgerMenu
hirist

Cloud Data Engineer - PySpark/Python

NPG Consultants
Hyderabad
5 - 6 Years

Posted on: 07/08/2025

Job Description

Cloud Data Engineer :


Build scalable, cloud-native data solutions using 5+ years of hands-on experience. Transform complex datasets into reliable insights in a collaborative environment.

Responsibilities :

- Design, develop, and maintain scalable ETL/ELT pipelines.

- Design and optimize data solutions on AWS, Azure, or GCP, ensuring secure and efficient data access.

- Write clean, modular Python and PySpark code for data transformation.

- Build and maintain dimensional and relational data models.

- Implement data quality checks and resolve anomalies to ensure data integrity.

- Optimize pipeline throughput and query performance.

- Collaborate with data scientists, analysts, and business teams.

- Maintain clear documentation of pipelines, workflows, and data architecture.

- Apply data security best practices and ensure compliance.

Preferred Skills :

- DBT (Data Build Tool) experience.

- Docker and Kubernetes experience.

- Familiarity with Kafka, Spark Streaming, or similar technologies.

- Experience with Apache Airflow or other orchestration platforms.

- Cloud Certifications (AWS Certified Data Analytics, Azure Data Engineer, or equivalent).

- Understanding of metadata management, lineage, and cataloging tools.

info-icon

Did you find something suspicious?