Data Engineer

Job Title : Data Engineer Unity Catalog

Employment Type : Full-Time (Hybrid 2 Days/Week in Office)

Experience Required : 5+ Years

Notice Period : Immediate to 15 Days

Location : Pune/ Bangalore / Hyderabad

About the Role :

We are seeking a highly skilled Data Engineer with strong experience in Unity Catalog and Databricks to join our growing data team. This role is critical for building a unified metadata and governance framework across our enterprise data platform. You will collaborate cross-functionally to enable data discovery, access control, and lineage tracking, empowering teams to build and scale data products with confidence and compliance.

Key Responsibilities :

- Partner with data architects, analysts, and scientists to define and maintain data lineage, ownership, and classification strategies.

- Implement automated data cataloging and metadata ingestion pipelines using PySpark and Databricks workflows.

- Establish and enforce data stewardship, governance, and compliance best practices across all layers of the data stack.

- Monitor, debug, and optimize catalog and metadata system performance for scalability and reliability.

- Provide training and support to data consumers and producers on catalog usage and access patterns.

- Continuously evaluate new capabilities in Databricks Unity Catalog, Delta Lake, and Azure ecosystem to improve data discoverability and trust.

Required Skills and Experience :

- Hands-on experience with Databricks Unity Catalog, including access control, data lineage, and metadata management.

- Proficient in PySpark and SQL for building scalable data pipelines and catalog integration.

- Solid understanding of Azure Data Services (e.g., Azure Data Lake, Azure Data Factory, Synapse, Key Vault).

- Deep knowledge of data governance principles, security, compliance, and privacy standards (GDPR, HIPAA, etc.).

- Strong troubleshooting skills and the ability to optimize performance for large-scale data platforms.

- Excellent communication and collaboration skills across technical and non-technical stakeholders.

Preferred Qualifications :

- Exposure to tools such as Purview, Alation, or Collibra is a plus.

- Experience working in regulated industries (Finance, Healthcare, etc.) is a strong advantage.

Why Join Us?

- Work on cutting-edge data governance and metadata solutions

- Collaborative team culture and growth-focused environment

- Flexible hybrid model with only 2 days per week in the office

- Opportunity to lead strategic data initiatives and own the Unity Catalog platform