Job Description :

Hiring : Senior Data Engineer

Location : Hyderabad (Hybrid)

Experience : 7 to 14 Years

Employment Type : Full-Time

Role Overview :

We are looking for a highly skilled Senior Data Engineer to lead the design, development, and optimization of scalable, secure, and high-performance data platforms.

The ideal candidate will bring a strong software engineering mindset, building data pipelines as reusable products with automation, governance, observability, and data quality embedded throughout the lifecycle.

This role will drive modern data architecture initiatives across batch and real-time processing environments while enabling analytics, AI, and business intelligence capabilities.

Key Responsibilities :

- Design and develop reusable data pipeline frameworks for batch and streaming workloads using standardized templates and best practices.

- Build scalable and efficient data models using Star Schema, Snowflake Schema, Data Vault 2.0, and Slowly Changing Dimensions (SCD).

- Optimize cloud data warehouse performance and cost across platforms such as Snowflake, BigQuery, Redshift, Synapse, and Databricks SQL.

- Develop and manage real-time data streaming solutions using Kafka, Kinesis, Pub/Sub, Spark Structured Streaming, and Apache Flink.

- Implement data quality frameworks, validation rules, anomaly detection, reconciliation processes, and automated quality gates.

- Drive metadata-driven engineering by managing schema evolution, lineage, data ownership, sensitivity classification, and business glossary integration.

- Create semantic layers, metric definitions, conformed dimensions, and analytical models for enterprise reporting and self-service BI.

- Prepare AI-ready data foundations by building feature-friendly datasets, knowledge layers, semantic models, and data structures for future AI/ML initiatives.

- Establish observability, monitoring, logging, tracing, performance profiling, and FinOps practices to ensure platform reliability and cost efficiency.

- Mentor engineering teams and promote DevOps best practices, CI/CD automation, Infrastructure as Code (IaC), containerization, and secure development standards.

Required Skills & Qualifications :

- Advanced expertise in SQL, Python, Data Engineering, and Data Processing Frameworks

- Hands-on experience with Snowflake, BigQuery, Amazon Redshift, Azure Synapse Analytics, Databricks SQL

- Strong knowledge of Data Modeling, Data Vault 2.0, Star Schema, Snowflake Schema, SCD Types, Schema Evolution

- Experience with Apache Kafka, Google Pub/Sub, AWS Kinesis, Apache Spark, Spark Structured Streaming, Apache Flink

- Proficiency in Apache Airflow, Cloud Composer, Managed Workflows, dbt, ELT Development

- Strong understanding of CI/CD Pipelines, Git, Terraform, CloudFormation, Docker, Kubernetes

- Expertise in Data Quality Frameworks, Data Contracts, Great Expectations, dbt Testing, Data Governance, Data Catalogs, Lineage Tools

- Experience building Semantic Layers, KPI Frameworks, Metric Stores, Enterprise BI Solutions

- Knowledge of Machine Learning Data Pipelines, Feature Engineering, GenAI Data Readiness, Knowledge Graphs, Ontologies

- Strong background in Cloud Security, IAM, Data Encryption, Tokenization, Data Masking, Compliance, Audit Controls

Preferred Skills :

- Scala or Java for Spark/Flink Development

- Golang Development Experience

- Experience with enterprise-scale Data Lakehouse architectures

- Exposure to AI/ML, Generative AI, and modern data product frameworks