Description :

Key Responsibilities :

- Data Pipeline Development & Expansion

- Design, implement, and support AWS & Snowflake-based data pipelines using Python and PySpark.

- Build out new pipelines for emerging business use cases, including RFC deployments.

- Implement automated CI/CD deployments for better organization and maintainability.

- Enhance existing pipelines for improved performance, reliability, and scalability.

Platform Integrations :

- Work with integration platforms and tools such as :

Falcon Shield (security & threat intelligence)
Collibra (data governance & cataloging)
XSOAR (security orchestration)
BIG ID (data privacy & compliance)
SPLUNK (SIEM & monitoring)

- Ensure smooth data flow between AWS/Snowflake pipelines and external platforms.

Evidence Collection & Compliance :

- Enable traceability of pipelines for audit readiness.

- Collaborate with ISS teams to collect and publish control implementation evidence.

- Support onboarding of new services, features, and regulatory requirements.

Architecture & Design :

- Assist in on-prem VM replacement with corresponding AWS/Snowflake integration architecture.

- Evaluate Snowflake-native and AWS-native solutions (e.g., AWS RDS, Glue, External managed tables).

- Ensure optimal performance, resiliency, replication, and control measures across all pipelines.

- Support architectural reviews and provide input on best practices for cloud-based data solutions.

Mandatory Skills :

- 5+ years of hands-on experience in AWS Data Engineering.

- Strong experience with Snowflake, Python, and PySpark.

- Experience with building, maintaining, and optimizing data pipelines.

- Knowledge of CI/CD deployments, Git, and pipeline versioning.

- Strong understanding of data architecture, performance tuning, and cloud-native solutions.

- Familiarity with data governance, security, and compliance tools (Collibra, BIG ID, Falcon Shield, etc.).