Description :

Role : Data Engineer 2.0

Location - Remote

Experience : Min. 5 Years

Notice Period- Immediate to 15 Days or serving notice period

Key Responsibilities :

- Design and implement manual test strategies for real-time streaming use cases using Azure Service Bus, Event Hubs, Kafka, and Azure Functions.

- Validate Spark Streaming applications, including unbounded data flows, streaming DataFrames, checkpoints, and streaming joins.

- Develop test plans for containerized microservices deployed on Kubernetes, ensuring scalability and fault tolerance.

- Test data ingestion and transformation workflows across open table formats like Delta Lake, Apache Iceberg, and Hudi.

Good to Have :

- Monitoring and troubleshooting system performance using observability stacks such as Prometheus, Grafana, and ELK.

- functional and performance testing on analytical databases and query engines such as Trino, StarRocks, and ClickHouse.

- Testing and validation of data products designed under data mesh architecture, ensuring domain-oriented data quality and governance.