Role : Kafka Administrator

Location : Pune , India

Exp : 10-15 years

Experience Required :

- 10+ years overall, with 5+ years in Kafka infrastructure management and operations.

- Must have successfully deployed and maintained Kafka clusters in production environments, with proven experience in securing, monitoring, and scaling Kafka for enterprise-grade data streaming.

Overview :

We are seeking an experienced Kafka Administrator to lead the deployment, configuration, and operational management of Apache Kafka clusters supporting real-time data ingestion pipelines.

The role involves ensuring secure, scalable, and highly available Kafka infrastructure for streaming flow records into centralized data platforms.

Key Responsibilities :

- Architect and deploy Apache Kafka clusters with high availability.

- Implement Kafka MirrorMaker for cross-site replication and disaster recovery readiness.

- Integrate Kafka with upstream flow record sources using IPFIX-compatible plugins.

- Configure Kafka topics, partitions, replication, and retention policies based on data flow requirements.

- Set up TLS/SSL encryption, Kerberos authentication, and access control using Apache Ranger.

- Monitor Kafka performance using Prometheus, Grafana, or Cloudera Manager and ensure proactive alerting.

- Perform capacity planning, cluster upgrades, patching, and performance tuning.

- Ensure audit logging, compliance with enterprise security standards, and integration with SIEM tools.

- Collaborate with solution architects and Kafka developers to align infrastructure with data pipeline needs.

- Maintain operational documentation, SOPs, and support SIT/UAT and production rollout activities.

Required Skills & Qualifications :

- Proven experience in Apache Kafka, Kafka Connect, Kafka Streams, and Schema Registry.

- Strong understanding of IPFIX, nProbe Cento, and network flow data ingestion.

- Hands-on experience with Apache Spark (Structured Streaming) and modern data lake or DWH platforms.

- Familiarity with Cloudera Data Platform, HDFS, YARN, Ranger, and Knox.

- Deep knowledge of data security protocols, encryption, and governance frameworks.

- Excellent communication, documentation, and stakeholder management skills.