Azure Databricks Developer - Data Engineering

About Client :

Our Client is a multinational IT services and consulting company headquartered in USA, With revenues 19.7 Billion USD, with Global work force of 3,50,000 and Listed in NASDAQ, It is one of the leading IT services firms globally, known for its work in digital transformation, technology consulting, and business process outsourcing, Business Focus on Digital Engineering, Cloud Services, AI and Data Analytics, Enterprise Applications ( SAP, Oracle, Sales Force ), IT Infrastructure, Business Process Out Source.

Major delivery centers in India, including cities like Chennai, Pune, Hyderabad, and Bengaluru.

Offices in over 35 countries.

India is a major operational hub, with as its U.S. headquarters.

Job Title : Azure Databricks

Key Skills : Databricks,Power bi

Job Locations : Hyderabad,Pune

Experience : 10 Years to 18 Years

Education Qualification : Any Graduation

Work Mode : Hybrid

Employment Type : Contract

Notice Period : Immediate

Job Summary :

We are seeking a skilled and experienced Azure Databricks Developer to join our data engineering team.

The ideal candidate will have a strong background in big data processing, Spark programming, and working on the Azure cloud platform.

You will be responsible for designing, building, and maintaining scalable data pipelines and solutions using Azure Databricks and other related technologies.

Key Responsibilities :

- Design and develop scalable data pipelines using Apache Spark on Azure Databricks.

- Ingest, transform, and analyze large datasets from multiple sources.

- Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions.

- Optimize Databricks jobs for performance and cost efficiency.

- Implement CI/CD pipelines for Databricks notebooks and jobs.

- Manage data quality, monitoring, and governance within the Databricks environment.

- Integrate Databricks with other Azure services such as Data Lake Storage (ADLS), Synapse Analytics, Azure Data Factory, and Azure Key Vault.

- Troubleshoot issues and perform root cause analysis of failed data jobs or processes.

- Document architecture, workflows, and best practices.

Required Skills & Qualifications :

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- 38 years of experience in data engineering or a similar role.

- Hands-on experience with Azure Databricks and Apache Spark (PySpark preferred).

- Strong knowledge of Azure ecosystem : ADF, ADLS, Synapse, Azure DevOps, etc.

- Proficiency in Python and SQL.

- Experience with Delta Lake, versioned data storage, and streaming (Structured Streaming).

- Familiarity with CI/CD, Git, and Databricks Repos.

- Understanding of data modeling, ETL/ELT processes, and data warehousing principles.