Key Result Areas And Activities :

Data Pipeline Development and Maintenance :

- Design, develop, and maintain scalable data pipelines to ensure efficient data flow and processing.

- Implement ETL (Extract, Transform, Load) processes to integrate data from various sources.

Data Quality and Integrity :

- Ensure the accuracy, completeness, and reliability of data by implementing robust data validation and cleansing processes.

- Monitor data quality metrics and address any discrepancies or issues promptly.

Performance Optimization :

- Optimize data storage and retrieval processes to enhance system performance and reduce latency.

- Implement indexing, partitioning, and other techniques to improve query performance.

Collaboration and Communication :

- Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.

- Communicate technical concepts and findings effectively to non-technical team members.

Security and Compliance :

- Implement data security measures to protect sensitive information and ensure compliance with relevant regulations and standards.

- Conduct regular audits and assessments to identify and mitigate potential security risks.

Essential Skills :

- Understanding of enterprise data warehousing principles of ETL/ELT and the tools/technologies used to implement them.

- Experience with Treasure Data, including designing and maintaining complex workflows.

- Strong knowledge of Dig Dag files.

- Strong knowledge of Presto SQL and Optimisation techniques.

- Experience with Python and relational databases.

- Familiarity with cloud platforms (e.g., AWS) and their data services.

Desirable Skills :

- Experience with version control systems such as Git.

- knowledge of Airflow or other orchestration tool.

Qualifications :

- Overall work experience 3+ years good focus on Treasure data.

- Technical certifications that attribute to continuous learning aspirations.

Qualities :

- Inherent curiosity and empathy for customer success.

- Obsessive about solving customer problems.

- Think long and act short.

- Collaborative with your peers, partners and your team.

- Excited about the mission and milestones not titles and hierarchies.

- Nurture an environment of experimentation and learning.

- Strong problem-solving skills and attention to detail.

- Excellent communication and collaboration skills.