Key Responsibilities :

Data Engineering & Development :

- Design, develop, and maintain large-scale data platforms and data pipelines.

- Build robust ETL/ELT frameworks for batch and real-time data processing.

- Develop scalable solutions using Python and GCP services.

- Design and implement data ingestion, transformation, and orchestration workflows.

- Build and optimize Airflow DAGs using Cloud Composer.

- Develop APIs and microservices using FastAPI.

Cloud & Platform Engineering :

- Design and implement cloud-native data solutions on Google Cloud Platform.

- Work extensively with :

1. BigQuery

2. Dataflow

3. Cloud Composer (Airflow)

4. GKE (Google Kubernetes Engine)

5. Cloud Run

- Support cloud migration initiatives from on-premise environments to GCP.

- Implement CI/CD pipelines and DevOps best practices.

Architecture & Solution Design :

- Conduct customer discovery workshops and requirement gathering sessions.

- Translate business requirements into scalable technical solutions.

- Create end-to-end solution architecture for enterprise data platforms.

- Design real-time and batch processing architectures.

- Drive technology selection, architecture reviews, and best practices.

Data Platform & Governance :

- Design and implement enterprise data lake and data warehouse solutions.

- Ensure data quality, governance, security, and compliance standards.

- Support production systems and troubleshoot critical issues.

- Optimize data performance, scalability, and reliability.

Mandatory Skills :

Programming & Frameworks :

- Python (Strong hands-on experience)

- FastAPI

- SQL (SQL Server, Oracle, PostgreSQL)

Google Cloud Platform :

- BigQuery

- Dataflow

- Cloud Composer (Airflow)

- GKE (Google Kubernetes Engine)

- Cloud Run

Data Engineering :

- Apache Spark

- Kafka

- ETL / ELT Development

- Data Migration (On-Premise to Cloud)

Databases :

- MongoDB

- Redis

- Bigtable

DevOps & Automation :

- GitHub

- CI/CD Pipelines

Architecture :

- Solution Architecture

- Enterprise Data Platform Design

- Real-Time Processing

- Batch Processing

Required Experience :

- Customer Requirement Gathering

- Discovery Workshops

- Solution Architecture Design

- Enterprise Data Platforms

- Data Governance & Data Quality

- Airflow DAG Development

- Production Support & Incident Management

Preferred Skills :

- Snowflake

- Databricks

- Azure Data Factory (ADF)

- Apigee API Management

- DLP (Data Loss Prevention)

- LLM / AI Integration Projects

- Cloud Security Best Practices

Education :

- Bachelor's or Master's degree in Computer Science.