Data Architect - Python/Spark/ETL

Description :

The core responsibilities for the job include the following :

Data Architecture and Design :

- Own the end-to-end architecture of the enterprise data ecosystem from source ingestion through the data lake to the analytics layer.

- Design scalable, performant, and secure data architectures ensuring integrity, availability, and observability.

- Translate business and analytical needs into logical and physical data models, aligned with governance and compliance standards.

- Define and enforce NFRs, including scalability, performance, maintainability, and reliability in all solutions.

Data Integration and Processing :

- Architect and oversee data ingestion, transformation, and orchestration for batch and real-time use cases.

- Build and optimize ETL/ELT pipelines using Spark and Python, ensuring efficiency and resilience.

- Enable seamless data movement across systems while maintaining data quality and consistency.

Data Modeling and Domain Understanding :

- Collaborate with business teams to understand domain entities and relationships, translating them into reusable data models.

- Develop semantic and analytical layers supporting BI, self-service analytics, and reporting.

- Ensure data models are intuitive, extensible, and aligned with business context.

Data Governance, Quality and Integrity :

- Define and implement data standards, lineage, and metadata frameworks for consistency and traceability.

- Establish data quality monitoring and validation processes to maintain trusted datasets.

- Partner with governance teams to enforce stewardship, retention, and compliance policies.

Security, Compliance, and Non-Functional Requirements :

- Collaborate with security and infrastructure teams to ensure access control, encryption, and auditability of data assets.

- Ensure compliance with organizational and regulatory requirements for data management.

- Continuously evaluate platform performance and scalability, driving architectural improvements.

Collaboration and Leadership :

- Partner with data engineering, analytics, and product teams to deliver impactful, reliable data solutions.

- Provide technical leadership and architectural guidance to ensure best practices in data design and coding.

- Foster a collaborative, agile, and innovation-driven culture across teams.

Documentation and Continuous Improvement :

- Maintain clear documentation of data architecture, models, and integration patterns.

- Conduct architecture reviews and performance assessments to drive continuous improvement.

- Stay informed on emerging data technologies and patterns (e. g., lakehouse, data mesh) to guide platform evolution.

Requirements :

- Bachelor's or higher degree in Computer Science, Information Technology, or a related field.

- Minimum 5 years of experience in the design and architecture of data solutions spanning the entire data landscape.

- Proven leadership experience in Data Engineering, with a strong background in designing and implementing data solutions.

- Proficiency in programming languages such as Python and Spark.

- Experience with Data Lake platforms, preferably Databricks

- Experience with Data visualization platforms like Sigma Computing, Thoughtspot, Cognos, or other BI tools

- Experience with big data technologies (e. g., Hadoop, Spark) and cloud platforms (e. g., AWS, Azure, GCP).

- Solid understanding of database management systems, data warehousing, and ETL processes.

- Strong analytical and problem-solving skills, with attention to detail.

- Excellent communication, collaboration, and leadership skills.