About the Role :

Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support the development of data infrastructure for our clients by participating in activities that may include starting from upstream and downstream technology selection to designing and building different components. Candidate will also be involved in projects like integrating data from various sources, managing big data pipelines that are easily accessible with optimised performance of the overall ecosystem.

Responsibilities :

- Responsible for the design, deployment, configuration, and operations of a multi-node big data cluster.

- Work with open source and/or commercial stacks to support the full SDLC.

- Deploy, manage, and maintain development, test and production environments for the big data platform.

- Develop scripts to automate and streamline operations and configurations in the infrastructure.

- Specify, design, build, and support BI solutions by working closely with the datalake team.

- Create dashboards and KPIs to show the business performance to management.

- Design and maintain data models used for reporting and analytics.

- Work to identify infrastructure needs and provide support to developers and business users.

- Research performance issues and optimize platform for performance.

- Troubleshoot and resolve issues in all operational environments.

- Work with a cross-functional team delivering software deployments.

- Own the design and development of automated solutions for recurring reporting and in-depth analysis.

Requirements :

- Strong experience in Data lake, Spark, distributed file system, Yarn, and Cloud services (GCP / AWS).

- Strong experience in SQL tools like Vertica, Dremio, or any big data SQL.

- Scripting knowledge in Shell and Python.

- Experience with ETL and OLAP concepts in building highly scalable data pipelines.

- Exposure to visualization systems like Apache Superset or Tableau is a plus.

- Experience with Agile, data structures, data analysis, and wrangling tools.

- Familiarity with version control and relational databases.

- Strong experience in monitoring, debugging, troubleshooting, and providing on-call support.

Qualifications :

- Bachelor's / Master's Degree in Computer Science or related field from a reputed institution.

- 8+ years of professional experience in software, with most of them from a product company.

- Experience working in a Linux based environment.

- Good communication and design skills.