Azure Databricks Engineer - Scala/Spark

Description :

Role : Databricks Developer

Location : Hyderabad

Role Overview :

- Build data ingestion, transformation, and processing workflows with Apache Spark

- Work with structured and unstructured data from multiple sources

- Optimize Spark jobs for performance, scalability, and cost efficiency

- Implement CI/CD pipelines and manage version control

- Collaborate with Data Engineers, Data Scientists, and Business Analysts

- Manage Databricks security, access controls, and best practices

- Troubleshoot performance and scalability issues

Required Skills :

- Proficiency in SQL

- Experience with cloud platforms (Azure, AWS, or GCP)

- Experience with ETL pipelines and data warehousing

- Knowledge of CI/CD, Git, and DevOps tools

- Strong understanding of Delta Lake, Lakehouse architecture, and performance tuning

- Experience handling large-scale datasets

Preferred Skills :

- Knowledge of Kafka, Airflow, or Snowflake

- Understanding of data governance, compliance, and security policies

Education & Experience :

- Bachelors or Masters degree in Computer Science or a related field

- 5 - 7 years of Databricks development experience

- Retail industry experience is mandatory