About the Role :

We are looking for a highly hands-on Data Platform Engineer to join our AI Labs team in Bengaluru. This role will be responsible for architecting, building, and scaling our end-to-end data platform, bridging traditional data engineering with modern AI infrastructure.

You will own the design and implementation of data pipelines, lakehouse architecture, multi-region data systems, and AI/ML-powered data processing workflows. From Change Data Capture (CDC) and ETL/ELT pipelines to LLM inference and multimodal data redaction, you will build reliable, scalable, and cost-efficient data infrastructure.

Roles & Responsibilities :

1. End-to-End Data Architecture & Engineering :

- Design, build, and maintain end-to-end data pipelines, covering CDC ingestion from production databases to production-grade lakehouse storage.

- Develop and scale workflow orchestration using Temporal and data transformation pipelines using dbt.

- Build standardized dimensional data models and optimized relational/materialized views to support analytics and business intelligence.

- Modernize legacy data workflows by replacing traditional DAG runners and cron-based batch jobs with scalable orchestration frameworks.

2. Multi-Region Architecture & Data Residency :

- Design and implement multi-region data architectures across cloud environments, ensuring compliance with data residency and sovereignty requirements.

- Establish secure data isolation mechanisms to ensure sensitive patient and transaction data remains within permitted geographical boundaries.

- Build compliant cross-region replication, synchronization, failover, and data-routing mechanisms.

3. AI/ML Integration & Multimodal Data Processing :

- Integrate machine learning models into data pipelines to identify and redact sensitive PII/PHI across text, images, audio, and video.

- Build and operationalize embedding generation, LLM inference, and semantic search pipelines.

- Develop scalable workflows for AI-powered data processing and multimodal content analysis.

4. Lakehouse Infrastructure & Platform Enablement :

- Design and optimize open lakehouse architectures using Apache Iceberg.

- Implement efficient partitioning, ACID transactions, and time-travel capabilities for large-scale datasets.

- Provision and manage secure, scalable Jupyter Notebook environments for data science and exploratory analytics.

5. Cloud Infrastructure, DevOps & FinOps :

- Design, deploy, and manage AWS infrastructure using Infrastructure as Code (Terraform or AWS CDK).

- Establish cost-monitoring frameworks to track cost-per-outcome across data and AI pipelines.

- Own production reliability through monitoring, alerting, data quality checks, and pipeline freshness tracking.

Required Skills & Technical Expertise :

- Strong experience in data engineering, data platform development, and large-scale distributed data systems.

- Proficiency in SQL and Python for data engineering and pipeline development.

- Hands-on experience building CDC-based ingestion and ETL/ELT pipelines.

- Experience with workflow orchestration frameworks such as Temporal.

- Working knowledge of dbt for data transformation and modeling.

- Experience with Apache Iceberg or other modern lakehouse technologies.

- Experience with AWS cloud infrastructure and Infrastructure as Code tools such as Terraform or AWS CDK.

- Exposure to ML/LLM inference pipelines, embedding generation, and AI-powered data processing.

Hiring Note :

We are hiring for two complementary SE II positions from the same candidate pool. Both positions are individual contributor roles focused on deep technical architecture and execution, with no formal people-management or team-mentoring responsibilities.