Mycom - Technical Lead - DataLake Development

Job Title : Technical Lead - Data Lake Developer

Experience Required : 8 to 10 years

Employment Type : Full-Time

About the Role :

You will lead the development of robust data pipelines and work with cutting-edge Big Data and Cloud technologies to enable real-time and batch data processing across the organization.

Key Responsibilities :

Design & Implement Scalable Data Lakehouse Architectures :

- Ensure scalability, reliability, and performance of data platforms.

Develop High-Performance Data Pipelines :

- Build and optimize real-time and batch data pipelines using Spark, Kafka, and streaming technologies.

- Implement data ingestion, transformation, and storage strategies.

Collaborate Across Teams :

- Work closely with data engineers, data scientists, and business stakeholders to understand data requirements.

- Mentor junior developers and contribute to technical leadership.

Ensure Data Quality & Governance :

- Implement data validation, monitoring, and governance frameworks.

- Ensure compliance with data security and privacy standards.

Required Technical Skills :

Big Data & Streaming Technologies :

- Apache Kafka

- Apache Pekko (formerly Akka)

- Apache Spark & Spark Streaming

Lakehouse & Storage Technologies :

- AWS S3 or MinIO S3

Query Engines & Databases :

- Trino (PrestoSQL)

- MongoDB

Cloud & Containerization :

- Kubernetes or OpenShift

Preferred Qualifications :

- Proven experience in designing and implementing large-scale data platforms.

- Strong understanding of distributed systems and microservices architecture.

- Experience with CI/CD pipelines and DevOps practices.

- Excellent problem-solving and communication skills.

Why Join Us?

- Lead impactful projects that drive data-driven decision-making.

- Collaborate with a passionate and talented team.

- Opportunities for continuous learning and career growth