What will you be doing here ?

- Responsible for the maintenance and growth of a 50TB+ data pipeline serving global SaaS products for businesses, including onboarding new data and collaborating with pre-sales to articulate technical solutions

- Solves complex problems across large datasets by applying algorithms, particularly within the domains of Natural Language Processing (NLP) and Large Language Models (LLM)

- Leverage bleeding-edge technology to work with large volumes of complex data

- Be hands-on in development - Python, Pandas, NumPy, ETL frameworks.

- Preferred exposure to distributed computing frameworks like Apache Spark , Apache Kafka, Apache Airflow, Elasticsearch

- Along with individual data engineering contributions, actively help peers and junior team members on architecture and code to ensure the development of scalable, accurate, and highly available solutions

- Collaborate with teams and share knowledge via tech talks and promote tech and engineering best practices within the team.

Requirement:

- B-Tech/M-Tech in Computer Science preferably from Tier 1/2 Colleges.

- 2-4 years of relevant work experience in data engineering or related roles.

- Proven ability to efficiently work with a high variety and volume of data (50TB+ pipeline experience is a plus).

- Solid understanding and preferred exposure to NoSQL databases, including Elasticsearch, MongoDB, and GraphDB.

- Basic understanding of working within Cloud infrastructure and Cloud Native Apps (AWS, Azure, IBM , etc.

- Exposure to core data engineering concepts and tools: Data warehousing, ETL processes, SQL, and NoSQL databases.

- Great problem-solving ability over a larger set of data and the ability to apply algorithms, with a plus for experience using NLP and LLM.

- Willingness to learn and apply new techniques and technologies to extract intelligence from data, with prior exposure to Machine Learning and NLP being a significant advantage.

- Sound understanding of Algorithms and Data Structures.

- Ability to write well-crafted, readable, testable, maintainable, and modular code.

Desired Profile :

- A hard-working, humble disposition.

- Desire to make a strong impact on the lives of millions through your work.

- Capacity to communicate well with stakeholders as well as team members and be an effective interface between the Engineering and Product/Business team.

- A quick thinker who can adapt to a fast-paced startup environment and work with minimum supervision