Role : Senior Data Engineer

Reporting to : Lead Data Engineer

Place of Posting : Mumbai

Job Purpose :

This is an intermediate-level position responsible for establishment and implementation of new developments, along with potential enhancements within the Technology & Transformation vertical. The primary objective is to act as a catalyst in Avanse's growth journey, through impactful contribution to Application Development & programming role.

Key Responsibilities :

- Design and develop scalable, batch and real time data pipelines, addressing business requirements.

- Identify the key problems, analyze information and recommend precise solutions to various technical & domain-specific issues.

- Act as an SME to various technical and functional stakeholders with minimum direct supervision

Key Accountabilities :

- Drive best practice in Data platform management and software development, ensuring high quality in terms of data, architecture, testing & deployment.

- Manage team responsibilities of addressing production issues promptly, if any.

- Work closely with product managers, Business Stakeholders to develop technical roadmaps.

- Build effective relationships with cross-functional teams to swiftly address issues.

Required Experience :

- 4-6 years of robust experience in Python programming, demonstrating a strong command of Object-Oriented Programming (OOP) principles and their application in scalable solutions.

- Proven expertise in designing, building, and maintaining comprehensive data platforms, encompassing Data Governance, ETL processes, Data Warehouses, and Data Lakes.

- Deep proficiency in complex SQL queries, with a demonstrated ability to optimize query performance for large datasets.

- Hands-on experience with Big Data technologies, specifically Apache Spark and the Hadoop ecosystem.

- Expertise in Spark performance tuning and optimization techniques to ensure efficient data processing.

- 4-6 years of practical experience with Amazon Web Services (AWS) data engineering services, including Glue, EMR, Lambda, Redshift, Athena, Glue Catalog, and Lake Formation

- Familiarity with UI and Business Dashboard development using tools such as Tableau, Qlik, QuickSight, or other similar frontend visualization platforms (Good to have)

- Experience working within an Agile methodology, utilizing tools like JIRA, Git, Confluence, and Jenkins for project management and CI/CD pipelines.

- Good to have domain expertise on BFSI/NBFC/FinTech

Required Qualification :

- Bachelor's degree/University degree or equivalent of B.E/B.Tech

Technical Competencies :

- Design and develop scalable, batch and real time data pipelines, addressing business requirements.

- Continuously enhance team practices and technical processes aligning with broader organizational goals.

Behavioral Competencies :

- Collaboration & Teamwork : Working effectively with cross-functional teams (data scientists, analysts, software engineers, business stakeholders) to achieve common goals.

- Problem-Solving & Analytical Thinking : The ability to dissect complex data challenges, identify root causes, and scalable solutions.

- Adaptability & Flexibility : The willingness to learn new technologies, adapt to changing project requirements, and navigate evolving data landscapes.

- Communication Skills : Clearly articulating technical concepts to non-technical audiences, documenting processes, and effectively discussing solutions with peers.

Personal Attributes :

- Integrity : Handling sensitive data responsibly and maintaining ethical standards.

- Patient & Persistent : Tackling complex, often time-consuming data integration and debugging tasks with resilience.

- Curious & Inquisitive : A natural inclination to explore data and understand underlying systems.

- Self-Motivated : Driven to achieve results and improve processes independently.