YASH Technologies - Data Architect - Palantir Foundry

Position Title : Data Architect Data Engineering

Scope Of Responsibility :

Purpose Of The Position :

Roles & Responsibilities :

- Define and govern enterprise-wide standards for data modeling, metadata, lineage, and security across hybrid environments.

- Architect high-throughput data pipelines that support batch and real-time ingestion from APIs, structured/unstructured sources, and external platforms.

- Collaborate with engineering, analytics, and product teams to implement analytical-ready data layers across Foundry, Snowflake, and AWS-based lake houses.

- Define data optimization strategies including partitioning, clustering, caching, and materialized views to improve query performance and reduce cost.

- Ensure seamless data interoperability across Palantir objects (Quiver, Workshop, Ontology), Snowflake schemas, and AWS S3-based data lakes.

- Lead DataOps adoption including CI/CD for pipelines, automated testing, quality checks, and monitoring.

- Govern identity and access management using platform-specific tools (e.g., Foundry permissioning, Snowflake RBAC, AWS IAM).

- Drive compliance with data governance frameworks, including auditability, PII protection, and regulatory requirements (e.g., GxP, HIPAA).

- Evaluate emerging technologies (e.g., vector databases, LLM integration, Data Mesh) and provide recommendations.

- Act as an architectural SME during Agile Program Increment (PI) and Sprint Planning sessions.

Education & Certifications :

- AWS/ Palantir/ Snowflake Architect or Data Engineer Certification (preferred).

Experience :

- Hands-on experience with Palantir Foundry (Quiver pipelines, Workshop, Ontology design).

- Working knowledge on AWS data services (S3, Glue, Redshift, Lambda, IAM, Athena).

- Working with Snowflake (warehouse design, performance tuning, secure data sharing).

- Domain experience in life Science/Pharma or regulated environments preferred.

Technical Skills :

- Data Architecture : Experience with hybrid data lake/data warehouse architecture, semantic modeling, and consumption layer design.

- Palantir Foundry : Proficiency in Quiver pipelines, Workshop applications, Ontology modeling, and Foundry permissioning.

- Snowflake : Deep understanding of virtual warehouses, time travel, data sharing, access controls, and cost optimization.

- AWS : Strong experience with S3, Glue, Redshift, Lambda, Step Functions, Athena, IAM, and monitoring tools (e.g., CloudWatch).

- ETL/ELT : Strong background in batch/streaming data pipeline development using tools such as Airflow, dbt, or NiFi.

- Programming : Python, SQL (advanced), Shell scripting; experience with REST APIs and JSON/XML formats.

- Data Modeling : Dimensional modeling, third-normal form, NoSQL/document structures, and modern semantic modeling.

- Security & Governance : Working knowledge of data encryption, RBAC/ABAC, metadata catalogs, and data classification.

- DevOps & DataOps : Experience with Git, Jenkins, Terraform/CloudFormation, CI/CD for data workflows, and observability tools.

Soft Skills :

- Strong communication skills to articulate architecture to both technical and business audiences.

- Ability to work independently and collaboratively in cross-functional, global teams.

- Strong leadership and mentoring capability for junior engineers and architects.

- Skilled in stakeholder management, technical storytelling, and influencing without authority.

Good-to-have Skills :

- Integration of AI/ML models or LLMs with enterprise data architecture.

- Familiarity with business intelligence platforms (e.g., Tableau, Power BI) for enabling self-service analytics.

- Exposure to vector databases or embedding-based search systems.