Kafka Engineer - Cloud Services

Job Title : Kafka Engineer

Experience : 6- 11 yrs

Location : Hyderabad or Bangalore (WFO-24-7 Shift)

Job Summary :

We are seeking an experienced Kafka Engineer with hands-on expertise in Confluent Kafka, Kafka on Cloud (MSK or Confluent Cloud), and strong proficiency in AWS services. The ideal candidate will be responsible for designing, implementing, and maintaining scalable event-streaming architectures that support our real-time data needs.

Role and Responsibilities :

- Design, build, and manage Kafka clusters using Confluent Platform and Kafka Cloud services (AWS MSK, Confluent Cloud).

- Develop and maintain Kafka topics, schemas (Avro/Protobuf), and connectors for data ingestion and processing pipelines.

- Monitor and ensure the reliability, scalability, and security of Kafka infrastructure.

- Collaborate with application and data engineering teams to integrate Kafka with other AWS-based services (e.g., Lambda, S3, EC2, Redshift).

- Implement and manage Kafka Connect, Kafka Streams, and ksqlDB where applicable.

- Optimize Kafka performance, troubleshoot issues, and manage incident response.

Required Skills :

- 5- 8 years of experience working with Apache Kafka and Confluent Kafka.

- Strong knowledge of Kafka internals (brokers, zookeepers, partitions, replication, offsets).

- Experience with Kafka Connect, Schema Registry, REST Proxy, and Kafka security.

- Hands-on experience with AWS (EC2, IAM, Cloud Watch, S3, Lambda, VPC, Load balancers).

- Proficiency in scripting and automation using Terraform, Ansible, or similar tools.

- Familiarity with DevOps practices and tools (CI/CD pipelines, monitoring tools like Prometheus/Grafana, Splunk, Datadog etc).

- Experience with containerization (Docker, Kubernetes) is a plus.

Nice to Have :

- Confluent Certified Developer or Administrator.

- AWS Certified

- CICD (AWS Code Pipeline, Harness)

- Containers ( Docker, Kubernetes)