Role : Data Engineering Lead

Total Exp : 8+ yrs

Location : Dehli

Designation : Data Engineering Lead

Educational Qualification :

- B.Tech / M.Tech / MS in Computer Science, Data Engineering, or related fields

- Professional certifications in cloud data platforms (AWS, Azure, GCP) desirable

Experience :

- 8-12 years of professional experience in data engineering or data platform architecture

- At least 4-5 years in designing and managing large-scale data pipelines for analytics or AI systems

- 5-7 years designing and implementing enterprise data architectures for analytics and AI use cases

- Experience working with structured, semi-structured, and unstructured data from diverse sources

Key Responsibilities :

- Architect and implement scalable data pipelines to support AI/ML models and analytical workloads

- Define and manage data warehouse architectures compliant with NeGD and MeitY standards

- Implement data ingestion frameworks for real-time and batch processing of government datasets

- Establish robust metadata management, lineage tracking, and data governance frameworks

- Optimize data storage, compression, and retrieval strategies for large-scale AI datasets

- Collaborate with AI/ML teams to ensure clean, high-quality, and versioned datasets for model training and inference

- Integrate APIs and connectors to unify data from multiple Digital India platforms

- Maintain compliance with MeitY's data retention, privacy, and security policies

Technical Competencies :

- Cloud Services : AWS Redshift, Glue, S3; Azure Synapse; GCP BigQuery and Dataflow

- Big Data Technologies : Apache Spark, Hadoop, Kafka, Airflow, Databricks, Snowflake

- Programming : Python, Scala, Java, SQL

- Databases : PostgreSQL, MongoDB, Cassandra, Elasticsearch

- Data Management : ETL/ELT design, schema evolution, DataOps, CI/CD pipelines

- Infrastructure : Docker, Kubernetes, Terraform, Jenkins for data automation

- Data Governance : Data cataloging, access control, encryption, and compliance logging