Data Architect - Python/Spark

Description :

Data Architecture & Design :

- Design scalable, performant, and secure data architectures ensuring integrity, availability, and observability.

- Translate business and analytical needs into logical and physical data models, aligned with governance and compliance standards.

- Define and enforce NFRs including scalability, performance, maintainability, and reliability in all solutions.

Data Integration & Processing :

- Build and optimize ETL/ELT pipelines using Spark and Python, ensuring efficiency and resilience.

- Enable seamless data movement across systems while maintaining data quality and consistency.

Data Modeling & Domain Understanding :

reusable data models.

- Develop semantic and analytical layers supporting BI, self-service analytics, and reporting.

- Ensure data models are intuitive, extensible, and aligned with business context.

Data Governance, Quality & Integrity :

- Establish data quality monitoring and validation processes to maintain trusted datasets.

- Partner with governance teams to enforce stewardship, retention, and compliance policies.

Security, Compliance & Non-Functional Requirements :

- Ensure compliance with organizational and regulatory requirements for data management.

- Continuously evaluate platform performance and scalability, driving architectural improvements.

Collaboration & Leadership :

- Provide technical leadership and architectural guidance to ensure best practices in data design and coding.

- Foster a collaborative, agile, and innovation-driven culture across teams.

Documentation & Continuous Improvement :

- Conduct architecture reviews and performance assessments to drive continuous improvement.

- Stay informed on emerging data technologies and patterns (e.g., lakehouse, data mesh) to guide platform evolution.

Qualifications :

- Minimum 5 years experience in design and architecture of data solutions spanning the entire data landscape.

- Proven leadership experience as a Data Engineering, with a strong background in designing and implementing data solutions.

- Proficiency in programming languages such as Python and Spark.

- Experience with Data Lake platforms preferably DataBricks

- Experience with Data visualization platforms like Sigma Computing, Thoughtspot, Cognos or other BI tools

- Experience with big data technologies (e.g., Hadoop, Spark) and cloud platforms (e.g., AWS, Azure, GCP).

- Solid understanding of database management systems, data warehousing, and ETL processes.

- Strong analytical and problem-solving skills, with attention to detail.

- Excellent communication, collaboration, and leadership skills.

Education Qualification :