Location : Gurgaon / Noida / Hyderabad

Mode : Hybrid

Job Description :

Data Pipeline Development :

- Design and build robust ETL/ELT pipelines for batch and stream processing using GCP tools.

Data Warehousing & Modeling :

- Build and optimize data models, data lakes, and data warehouses, primarily using BigQuery for analytical workloads.

Workflow Orchestration :

- Automate, schedule, and monitor data workflows using orchestration frameworks.

Pipeline Optimization :

- Enhance system performance and manage resource allocation for cost-efficiency.

Data Governance & Security :

- Enforce data quality frameworks, lineage tracking, security protocols, and compliance standards.

Cross-Functional Collaboration :

- Partner with DevOps, Software Engineering, and Business Intelligence teams to deploy data solutions into production environments.

Essential Qualifications & Skills :

1. Cloud Services :

- Proficiency with GCP ecosystem services such as :

1. BigQuery

2. Cloud Dataflow

3. Cloud Storage

4. Cloud Composer

5. Apache Airflow

6. Dataform/dbt

7. Pub/Sub

2. Programming :

- Advanced expertise in SQL and Python/Java.

3. Big Data Frameworks :

- Hands-on experience with distributed computing and processing frameworks, e.g., Apache Spark, PySpark, or Apache Beam.

4. Data Engineering Concepts :

Solid understanding of :

- Data Warehousing

- Star/Snowflake Schemas

- Dimensional Modeling

- Change Data Capture (CDC)

- DevOps Practices

Familiarity with :

- CI/CD pipelines

- Git version control

- Infrastructure as Code (e.g., Terraform)