Description :

Job Title : Data Engineer - Databricks (Unity Catalog Migration)

Experience : 1+ Year

Job Summary :

We are looking for an experienced Data Engineer with strong hands-on experience in Databricks and Unity Catalog to support a Unity Catalog migration project. The role involves migrating existing Databricks workspaces, data assets, and ETL pipelines to Unity Catalog while ensuring data security, governance, and minimal disruption to ongoing workflows.

Key Responsibilities :

- Work on end-to-end Unity Catalog migration for Databricks environments.

- Migrate schemas, tables, views, notebooks, and jobs to Unity Catalog.

- Refactor and optimize existing ETL pipelines to be Unity Catalogcompatible.

- Configure and manage data access controls, permissions, and governance using Unity Catalog.

- Collaborate with data architects, analytics teams, and stakeholders to ensure smooth migration.

- Perform impact analysis and resolve migration-related issues.

- Ensure best practices for data security, compliance, and performance.

- Document migration steps, architecture changes, and operational guidelines.

Required Skills & Experience :

- Strong hands-on experience with Databricks (Azure / AWS).

- Proven experience in Unity Catalog implementation and migration projects.

- Solid understanding of ETL/ELT pipelines using Databricks (PySpark / Spark SQL).

- Experience working with Delta Lake.

- Good knowledge of data governance, access control, and metadata management.

- Experience with cloud platforms (Azure or AWS).

- Strong problem-solving and debugging skills.

Good to Have :

- Experience with CI/CD for Databricks.

- Exposure to large-scale data migration or platform modernization projects.

- Knowledge of data quality frameworks and monitoring.