GyanSys - Databricks Consultant/Senior Consultant/Lead - Data Engineering Solutions

Role Overview :

We are looking for a skilled Databricks Consultant / Senior Consultant / Lead to design and deliver scalable, cloud-based data engineering solutions.

The ideal candidate will have strong hands-on expertise in Databricks and Apache Spark, with experience building high-performance ETL/ELT pipelines and working with large-scale structured and unstructured datasets.

This role involves close collaboration with analytics, reporting, and business teams to enable data-driven decision-making.

Key Responsibilities :

- Design, develop, and implement Databricks-based data engineering solutions

- Build scalable ETL/ELT pipelines using Apache Spark

- Ingest, transform, and validate large volumes of structured and unstructured data

- Optimize Spark jobs for performance, scalability, and cost efficiency

- Work with cloud-based data lakes and modern data platforms

- Ensure adherence to data quality, governance, and security standards

- Collaborate with analytics and reporting teams to support downstream use cases

- Support implementation, migration, and modernization of data platforms

Technical Requirements (Must-Have) :

- Strong hands-on experience with Databricks

- Proficiency in : Apache Spark SQL Python

- Experience working with cloud-based data platforms (AWS / Azure / GCP)

- Experience with data lakes and large-scale data processing

- Strong understanding of data modeling and analytics workflows

Good-to-Have Skills :

- Experience with Delta Lake

- Exposure to streaming frameworks (Kafka / Structured Streaming)

- Knowledge of CI/CD and DevOps practices in data engineering

- Experience with BI/reporting tools (Power BI, Tableau, etc)