HamburgerMenu
hirist

D-TechWorks - Data Engineering Lead - ETL

D-TechWorks Pvt Ltd
Pune
5 - 9 Years

Posted on: 17/07/2025

Job Description

Data Engineer Lead - Pune

Job Title : Data Engineer Lead

Experience : 5 to 9 Yrs

Location : Pune

Responsibilities :


- Lead and mentor a team of data engineers, providing technical guidance, setting best practices, and overseeing task execution for the migration project.

- Design, develop, and architect scalable ETL processes to extract, transform, and load petabytes of data from on-premises SQL

Server to GCP Cloud SQL PostgreSQL.

- Oversee the comprehensive analysis of existing SQL Server schemas, data types, stored procedures, and complex data models, defining strategies for their optimal conversion and refactoring for PostgreSQL.

- Establish and enforce rigorous data validation, quality, and integrity frameworks throughout the migration lifecycle, ensuring accuracy and consistency.

- Collaborate strategically with Database Administrators, application architects, business stakeholders, and security teams to define migration scope, requirements, and cutover plans.

- Lead the development and maintenance of advanced scripts (primarily Python) for automating large-scale migration tasks, complex data transformations, and reconciliation processes.

- Proactively identify, troubleshoot, and lead the resolution of complex data discrepancies, performance bottlenecks, and technical

challenges during migration.

- Define and maintain comprehensive documentation standards for migration strategies, data mapping, transformation rules, and

post-migration validation procedures.

- Ensure data governance, security, and compliance standards are meticulously applied throughout the migration process, including data encryption and access controls within GCP.

- Implement Schema conversion or custom schema mapping strategy for SQL Server to PostgreSQL shift

- Refactor and translate complex stored procedures and T-SQL logic to PostgreSQL-compatible constructs while preserving functional equivalence.

- Develop and execute comprehensive data reconciliation strategies to ensure consistency and parity between legacy and migrated datasets post-cutover.

- Design fallback procedures and lead post-migration verification and support to ensure business continuity.

- Ensuring metadata cataloging and data lineage tracking using GCP-native or integrated tools.


info-icon

Did you find something suspicious?