Data Architect - Data Governance & Quality

Description :

Work with us to build modern Insurtech AI underpinned solutions, we are a growing team of hands on architects striving to build high quality solutions for our internal and external customers.

The Data Architect is responsible for designing enterprise data solutions including data warehouses, data lakes, and analytics platforms.

This role defines data strategy, establishes data governance, and architects scalable data infrastructure.

Key Responsibilities :

Enterprise Data Architecture & Strategy :

Works with Chief Architect to define the enterprise data strategy and roadmap, designing comprehensive architectures encompassing data warehouses, data lakes, and lakehouse patterns using Azure Synapse Analytics, Azure Data Lake Storage, and Azure Databricks as primary platforms, with strategic use of Snowflake, AWS Redshift, and Google BigQuery where appropriate.

- Establishes architecture standards implementing bronze/silver/gold medallion patterns for raw ingestion, transformation, and business-ready consumption layers.

Data Modeling & Design :

- Designs sophisticated data models using dimensional modeling (star/snowflake schemas) for analytics, data vault methodology for enterprise warehousing with audit trails critical for insurance, and normalized models for operational systems.

- Creates master data management solutions ensuring single sources of truth for policies, claims, customers, agents, and products.

- Establishes naming conventions, data standards, and metadata practices enabling data discoverability and self-service analytics.

Data Integration & Pipelines :

- Architects robust ETL/ELT pipelines using Azure Data Factory, Apache Spark, Azure Databricks, and SSIS for legacy integration.

- Designs data ingestion supporting batch processing, real-time streaming via Azure Event Hubs and Kafka, change data capture for incremental updates, and API-based integration.

- Implements DataOps practices including CI/CD for data workflows, automated quality validation, version control, and fault-tolerant pipeline design with comprehensive error handling.

Data Governance & Quality :

- Leads governance initiatives implementing Azure Purview or Collibra for metadata management, data lineage, classification, and discovery.

- Designs data quality frameworks with profiling, validation, anomaly detection, and quality scorecards ensuring insurance data meets regulatory requirements.

- Implements privacy controls for GDPR, CCPA, and insurance regulations through data masking, anonymization, and access controls protecting PII and sensitive claim data.

Performance & Cost Optimization :

- Architects for scale designing partitioning strategies for large insurance datasets, indexing for common query patterns, compression reducing storage costs, and caching for frequently accessed data.

- Implements cost optimization through lifecycle policies, query optimization, and right-sized infrastructure balancing performance with budget.

Analytics & AI/ML Enablement :

- Designs platforms optimized for analytics and AI/ML workloads, creating feature stores for reusable ML features, data science sandboxes with governance, and training pipelines supporting insurance-specific models for fraud detection, claims prediction, and underwriting automation.

- Ensures architectures support traditional BI and advanced analytics including NLP on policy documents and computer vision for claims assessment.

Collaboration & Leadership :

- Works with business stakeholders, data engineers, analytics teams, AI/ML practitioners, and enterprise architects to deliver solutions meeting organizational needs.

- Mentors data engineering teams, leads technical design reviews, and ensures new systems align with enterprise data architecture principles from inception.

Required Skills :

Data Architecture & Modeling :

- Expert knowledge of dimensional modeling techniques (Kimball methodology, star/snowflake schemas), data vault 2.0 methodology for enterprise data warehousing, entity-relationship modeling for operational systems, and modern data lake architectures including medallion/multi-hop patterns.

- Deep understanding of master data management principles, data fabric and data mesh concepts for distributed data architectures, and normalization/denormalization trade-offs optimizing for different use cases.

Microsoft Data Platform Expertise :

- Comprehensive experience with Azure Synapse Analytics (dedicated SQL pools, serverless SQL, Spark pools), Azure Data Lake Storage Gen2 (hierarchical namespace, access control, lifecycle management), Azure Data Factory (mapping data flows, control flows, triggers, parameters), Azure Databricks (Delta Lake, Unity Catalog, notebooks, workflows), Azure Purview (data catalog, lineage, compliance), Azure Event Hubs and Azure Stream Analytics for real-time processing, and Power BI for business intelligence integration.

Multi-Platform Data Technologies :

- Familiarity with Snowflake (virtual warehouses, time travel, data sharing), AWS data services (Redshift, S3, Glue, Athena), Google Cloud Platform (BigQuery, Cloud Storage, Dataflow), and open-source technologies including Apache Spark (DataFrame API, Spark SQL, performance tuning), Apache Kafka for event streaming, Apache Airflow for workflow orchestration, and Delta Lake for ACID transactions on data lakes.

ETL/ELT & Data Integration :

- Proficiency in Azure Data Factory pipeline development, Apache Spark programming (PySpark, Scala), SQL Server Integration Services (SSIS) for legacy systems, database replication and CDC tools (Azure Data Sync, Debezium), API integration patterns (REST, GraphQL), and data validation/transformation frameworks ensuring data quality throughout pipelines.

Data Governance & Security :

- Expertise implementing data governance frameworks, metadata management solutions (Azure Purview, Collibra, Alation), data lineage and impact analysis, data quality tools and methodologies, data classification and sensitivity labeling, role-based access control (RBAC) and attribute-based access control (ABAC), encryption at rest and in transit, data masking and anonymization techniques, and compliance with GDPR, CCPA, HIPAA, and insurance-specific regulations.

Performance & Optimization :

- Deep understanding of database indexing strategies, partitioning and bucketing techniques for large datasets, query optimization and execution plan analysis, caching mechanisms (Redis, in-memory tables), compression algorithms and their trade-offs, and cost optimization approaches for cloud data platforms.

Insurance Domain Knowledge :

- Understanding of insurance data structures (policy administration, claims management, billing), actuarial data requirements (loss triangles, reserves, exposures), regulatory reporting (statutory accounting, Solvency II, state filings), reinsurance data flows, and agent/distribution channel data models.

Required Experience :

- Seven or more years in data engineering, data architecture, or database administration roles with demonstrable emphasis on cloud data platforms, including at least three years designing and implementing enterprise-scale data solutions.

- Proven experience architecting solutions on Microsoft Azure data services with preference for candidates having led migrations from on-premises to cloud or multi-cloud data platform implementations.

- Prior experience in designing lakehouse style architectures.

- Track record of designing data warehouses and data lakes for complex organizations, preferably in insurance or financial services environments with strict regulatory requirements.

- Hands-on experience implementing data governance frameworks, establishing data quality standards, and building data catalogs enabling self-service analytics.

- Demonstrated success architecting data platforms supporting AI/ML workloads including feature engineering pipelines and model training datasets.

- Experience leading cross-functional data initiatives, collaborating with business stakeholders to translate requirements into technical solutions, mentoring data engineering teams, and establishing architectural standards adopted across organizations.

- Evidence of optimizing data platform costs while maintaining performance SLAs and implementing DataOps practices including automated testing, CI/CD for data pipelines, and monitoring/alerting frameworks.

Key Competencies :

- Insurance Domain Expertise : Understanding of insurance business processes, data flows across policy administration and claims systems, actuarial data requirements, regulatory reporting obligations, and reinsurance data structures.

- Familiarity with insurance industry standards including ACORD data formats and common insurance platforms (Duck Creek, Guidewire, InsuranceSuite).

- Technical Leadership : Ability to lead architectural design reviews, establish technical standards and patterns, mentor junior data engineers and architects, communicate complex technical concepts to non-technical stakeholders, and balance competing priorities including cost, performance, security, and time-to-market.

- Strategic Thinking : Capability to define multi-year data platform roadmaps aligned with business strategy, evaluate emerging data technologies for organizational fit, design architectures accommodating future growth and changing requirements, and balance build-versus-buy decisions optimizing for long-term maintainability and total cost of ownership.

- Innovation & Continuous : Improvement Commitment to staying current with data architecture trends including data mesh, data fabric, lakehouse architectures, and real-time analytics platforms.

- Experience evaluating and adopting new technologies including streaming analytics, graph databases, time-series databases, and vector databases supporting AI/ML workloads.