Job Title : Lead Data Engineer

Location : Marathalli, Bangalore

Job Mode : Remote

Key Responsibilities :

- Create and maintain data sources inside our data lake on Azure

- Assemble and analyze large, complex data sets that meet both functional and non-functional business requirements

- Identify, design, and implement internal process improvements : automating manual processes; optimizing data delivery; re-designing infrastructure for greater scalability.

- Work with stakeholders including the Executive, Tech, Product, Data and Design teams to optimize data driven decisions

- Keep our data separated and secure by understanding and applying best practice access policies

- Create tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader

- Ensure data integrity with thorough data quality test coverage and data validation

- Improve, create and implement CI/CD processes and tooling inside the data lake

Required Qualifications :

- Join immediately

- Databricks

- Databricks hands on experience (esp. Pipelines PySpark & SQL, Delta Lake, Unity Catalog, DLT-)

- Ingesting & transforming data using Databricks - Min 3 years continuous hands-on

- Advanced SQL and Python knowledge

- Data Lake housing and Modelling experience (e.g. medallion architecture & query performance optimization)

- Experience with backend systems ERP, CRM (e.g. SAP, Oracle, Salesforce)

- Exposure of working with Azure (ADLS, ADF)

- Good Communication skills