About the Role :

We are seeking a highly skilled Lead Databricks Engineer to join our team at DXC Technology, where you will play a pivotal role in building and scaling enterprise-grade data solutions on the Azure ecosystem. Youll lead the development of robust data pipelines, analytics frameworks, and cloud-native solutions using cutting-edge tools like Azure Databricks (ADB), Azure Data Factory (ADF), Delta Lake, and PySpark.

This role is ideal for professionals who thrive in fast-paced, collaborative environments and are passionate about data engineering, cloud transformation, and data-driven business impact.

Key Responsibilities :

- Design, develop, and maintain scalable data ingestion and transformation pipelines using Azure Databricks, PySpark, and ADF.

- Write, optimize, and manage complex SQL queries and transformations across structured and unstructured data sources.

- Implement data lake solutions using Delta Lake, Unity Catalog, and Azure Synapse Analytics.

- Apply data profiling, cataloging, and lineage techniques to ensure data quality and governance compliance.

- Enable secure and governed access to data assets using Azures data governance features and role-based access control (RBAC).

- Serve as technical lead on large-scale data engineering initiatives, guiding junior developers and ensuring architectural best practices.

- Lead Proof of Concepts (PoCs), prototype development, and pilot solution implementations to validate technical feasibility.

- Work closely with solution architects, business analysts, DevOps teams, and client stakeholders to translate requirements into data-driven outcomes.

- Take part in business process mapping and contribute to data architecture decisions aligned with business needs.

- Utilize Azure DevOps for CI/CD, automation of data pipelines, and environment management.

- Establish and enforce coding standards, data modeling conventions, and deployment workflows.

- Monitor and optimize the performance of data pipelines, ETL/ELT processes, and data storage systems.

- Troubleshoot and resolve performance bottlenecks and errors in production environments.

Required Qualifications :

- 5+ years of hands-on experience as a Data Engineer in Azure cloud environments.

- Azure Databricks (ADB)

- Azure Data Factory (ADF)

- Delta Lake / Delta Tables

- Synapse Analytics

- Unity Catalog

- Strong programming proficiency in Python, PySpark, and Spark SQL.

- Deep understanding of SQL with the ability to write complex queries, joins, window functions, etc.

- Experience with Azure DevOps for pipeline automation, code repositories (Git), and CI/CD integration.

- Demonstrated ability to conduct data profiling, data cataloging, and data lineage tracking.

- Strong knowledge of data governance, security, and compliance within cloud data environments.

- Excellent analytical thinking, problem-solving, and communication skills.