Role Overview:

We are seeking an experienced AWS Data Architect to design and implement a scalable data platform requirement. The candidate should have majorly worked on diverse formats, requiring extensive transformation, normalization, and standardization.

The architect will lead the creation of a robust, scalable, and governed data platform leveraging AWS services such as Redshift, AWS Glue, and S3, ensuring seamless ingestion, transformation, and analytics readiness.

Key Responsibilities:

- Design end-to-end data architecture for ingesting data from multiple external systems.

- Define data lake and data warehouse architecture using AWS services (S3, Redshift).

- Establish robust data ingestion and processing patterns (batch, ETL/ELT).

- Ensure scalability, performance, and cost optimization.

- Design ingestion pipelines for structured, semi-structured, and unstructured data.

- Handle various ingestions mechanisms like SFTP-based/API or event driven with varying file formats (CSV, XML, JSON, etc.).

- Implement data validation, error handling, and reconciliation processes.

- Lead complex data transformation workflows to unify disparate datasets into a consistent canonical format.

- Define and enforce data quality and consistency standards across all incoming datasets.

Optimize Redshift workloads:

1. Query tuning

2. Workload management (WLM)

3. Data distribution strategies

Define data governance frameworks:

1. Metadata management

2. Data lineage

3. Data quality checks

- Should have implemented monitoring, alerting, and audit mechanisms.

- Proven track record of designing end-to-end cloud data platforms.

- Experience building scalable, resilient architectures on AWS.

Technical Skills:

- Amazon Redshift

- AWS Glue (ETL, PySpark)

- Amazon S3

- Experience loading multiple format data ingestion config driven solution

- SQL (advanced)

- Python / PySpark

- Data warehousing concepts

- ETL/ELT design patterns