Data Engineer Lakehouse & Analytics Infrastructure

Experience : 4 - 6 Years

Location : Noida, India

Employment Type : Full-Time

About the Role :

We are looking for an experienced Data Engineer to design, build, and optimize our modern data lakehouse infrastructure. The ideal candidate will have hands-on expertise with PostgreSQL, MinIO, Dremio, and Apache Iceberg, and will play a key role in building scalable, high-performance data pipelines and enabling self-service analytics across the organization.

Key Responsibilities :

Data Architecture & Lakehouse Management :

- Design, implement, and maintain Apache Iceberg table formats for large-scale analytical datasets, ensuring schema evolution, partitioning, and time-travel capabilities are optimally configured.

- Manage and optimize object storage on MinIO, including bucket policies, lifecycle rules, versioning, and access controls for the data lake.

- Architect and maintain the lakehouse layer to ensure ACID compliance, data consistency, and efficient query performance.

Query & Analytics Layer :

- Configure, administer, and optimize Dremio as the semantic and query acceleration layer, including reflections, data source connections, and virtual datasets.

- Tune Dremio query performance for BI tools and ad-hoc analytics, ensuring low-latency access to Iceberg tables.

- Build and maintain virtual datasets, views, and data curation layers to support self-service analytics for business teams.

Database Engineering :

- Design, develop, and optimize PostgreSQL schemas, queries, indexes, and stored procedures for transactional and metadata-driven workloads.

- Monitor and tune PostgreSQL performance (query plans, indexing strategies, vacuuming, replication) to ensure high availability and reliability.

- Implement backup, recovery, and disaster recovery strategies for PostgreSQL instances.

Data Pipeline Development :

- Build and maintain robust ETL/ELT pipelines to ingest data from multiple sources into the Iceberg-based lakehouse on MinIO.

- Ensure data quality, validation, and governance across ingestion and transformation pipelines.

- Collaborate with data analysts, BI developers, and data scientists to understand data requirements and deliver reliable datasets.

Infrastructure, Monitoring & Best Practices :

- Monitor system health, storage utilization, and query performance across MinIO, Dremio, and PostgreSQL environments.

- Implement security best practices including access control, encryption, and audit logging across the data stack.

- Document architecture, data flows, and operational runbooks; contribute to establishing engineering best practices.

- Troubleshoot and resolve production issues related to storage, query engines, and databases.

Required Skills & Qualifications :

- 4 - 6 years of experience in Data Engineering or a related field.

- Strong hands-on experience with Apache Iceberg (table maintenance, compaction, partitioning, schema evolution).

- Practical experience with MinIO or other S3-compatible object storage solutions.

- Proven experience administering and optimizing Dremio (or similar query engines like Trino/Presto/Starburst) is a strong plus.

- Solid expertise in PostgreSQL schema design, query optimization, indexing, and performance tuning.

- Proficiency in SQL and at least one scripting/programming language (Python preferred).

- Experience with data lakehouse architecture and modern data stack concepts (data lake, data warehouse, ELT/ETL).

- Familiarity with containerization (Docker/Kubernetes) and cloud/on-prem infrastructure is a plus.

- Understanding of data governance, security, and access control principles.

- Strong problem-solving skills and ability to work independently in a fast-paced environment.