Consultant Data Engineer Databricks

Location : Bangalore, Hyderabad, Gurgaon

Experience : 4 to 8 Years

Employment Type : Full-time

Job Summary :

We are looking for an experienced Consultant Data Engineer Databricks with 4 to 8 years of experience in designing, developing, and optimizing scalable data engineering solutions on Azure and Databricks. The ideal candidate should have strong hands-on experience with PySpark, Azure Data Factory (ADF), Databricks, ETL pipelines, SQL, and cloud-based data platforms.

Key Responsibilities :

- Design, develop, and maintain scalable ETL/ELT data pipelines using Azure and Databricks.

- Develop and optimize data processing solutions using PySpark.

- Build and manage data pipelines using Azure Data Factory (ADF).

- Work with Databricks for data ingestion, transformation, processing, and optimization.

- Write complex and optimized SQL queries for data extraction and transformation.

- Implement data integration solutions across multiple data sources.

- Perform data cleansing, transformation, validation, and quality checks.

- Troubleshoot and optimize data pipelines for performance, reliability, and scalability.

- Collaborate with data architects, developers, business teams, and clients to understand requirements.

- Participate in technical discussions and contribute to solution design.

- Ensure adherence to data engineering best practices, coding standards, and project requirements.

- Support deployment, monitoring, and troubleshooting of data engineering solutions across environments.

Required Skills :

- 48 years of relevant experience in Data Engineering.

- Strong hands-on experience with Databricks.

- Strong proficiency in PySpark.

- Hands-on experience with Azure Data Factory (ADF).

- Strong knowledge of ETL/ELT concepts and data pipeline development.

- Strong SQL skills.

- Good understanding of Microsoft Azure data services.

- Experience working with large datasets and distributed data processing.

Preferred Qualifications :

- Experience with Azure-based data engineering projects.

- Knowledge of data lake/data warehouse concepts.

- Experience in performance tuning of Spark/Databricks jobs.

- Experience working in client-facing consulting environments.

Key Skills : Databricks, PySpark, Azure Data Factory, ETL, Azure, SQL, Data Engineering