Job Description : Confluent Kafka L3 Engineer / Kafka SME

Position Title : Confluent Kafka L3 Engineer / Subject Matter Expert (SME)

Job Location : Mumbai, India (100% Onsite)

Department : Payments Applications & Financial Infrastructure

Domain/Industry : Banking & Financial Services (BFSI)

Work Schedule : Day Shift (6 Days a Week)

Notice Period : Immediate Joiners to 30 Days Maximum

Compensation : (commensurate with experience)

Role Overview :

We are looking for an experienced Confluent Kafka L3 Engineer / Subject Matter Expert to manage, optimize, and support our mission-critical distributed streaming platform. In this role, you will be responsible for ensuring high availability, performance, and security across large-scale, real-time payment processing systems within a fast-paced banking environment.

Key Responsibilities :

1. Platform Architecture & Administration :

- Oversee the end-to-end administration, configuration, deployment, and tuning of enterprise Apache Kafka and Confluent Platform clusters in high-throughput production environments.

- Manage multi-cluster replication, MirrorMaker, Kafka Connect, Schema Registry, and ksqlDB/Kafka Streams.

- Ensure maximum uptime, resilience, and fault tolerance for critical payment application data pipelines.

2. Level 3 Support & Incident Management :

- Act as the senior escalation point (L3/SME) for high-severity production incidents, complex cluster failures, performance bottlenecks, and data loss scenarios.

- Lead root cause analysis (RCA) and implement permanent resolutions and preventive actions for recurring issues.

- Design and maintain robust Disaster Recovery (DR) plans, backup protocols, and failover mechanisms across distributed data centers.

3. Performance Optimization & Security :

- Benchmark and optimize Kafka cluster performance, JVM parameter tuning, OS kernel parameters, and broker/producer/consumer configurations.

- Enforce enterprise security standards across Kafka clusters, including SSL/TLS encryption, SASL authentication (Kerberos/SCRAM), ACLs, and RBAC policies.

- Monitor resource utilization, cluster health, lag metrics, and capacity planning using Prometheus, Grafana, Confluent Control Center, or similar tools.

4. Technical Leadership & Collaboration :

- Partner with application development teams to optimize Kafka producer/consumer code, topic configurations, schema management, and serialization protocols.

- Mentor L1/L2 engineering teams and establish best practices for event streaming standards across banking applications.

- Participate in architecture review boards for new payment applications integrating with the Kafka platform.

Technical & Functional Qualifications :

Education & Experience :

- Education : Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

- Total Experience : 7 - 12 years in IT infrastructure/data platform engineering.

- Relevant Experience : 4 - 5 years of direct, hands-on experience configuring, troubleshooting, and supporting Apache Kafka or Confluent Platform in large-scale, enterprise production environments.

Core Competencies :

- Kafka Expertise : Advanced knowledge of Kafka Architecture (Brokers, Zookeeper/KRaft, Topics, Partitions, Offset Management, Consumer Groups).

- Confluent Ecosystem : Hands-on experience with Confluent Control Center, Schema Registry, Kafka Connectors, and Enterprise Security Plugins.

- Domain Experience : Prior experience working in Banking or Financial Services, specifically with Payment Applications (e.g., UPI, NEFT/RTGS, core payment gateways, real-time fraud detection pipelines) is mandatory.

- Operating Systems & Scripting : Strong proficiency in Linux/Unix administration, bash scripting, Python, or Ansible for automation.