Key Responsibilities :

- Design, develop and maintain scalable data engineering solutions using Apache Spark and Scala.

- Build and manage data pipelines using Airflow and related orchestration technologies.

- Work extensively with Hive, Kubernetes, Docker and GitHub Actions.

- Develop and optimize data solutions using Apache Iceberg.

- Design and implement data engineering solutions on Azure Databricks.

- Work with ADLS, ADF, Delta Lake, Databricks Jobs & Pipelines, Lakeflow Connect and Unity Catalog.

- Work with both Classic and Serverless Databricks Compute environments.

- Implement scalable, reliable and high-performance data pipelines.

- Follow CI/CD and DevOps best practices using GitHub Actions.

- Explore and implement AI-assisted development practices across SDLC and data engineering.

- Collaborate with cross-functional teams to deliver innovative data solutions aligned with business requirements.

Must-Have Skills :

- Spark, Scala, Airflow, Hive

- Azure Databricks, ADLS, ADF, Delta Lake

- Kubernetes, Docker, GitHub Actions

Strongly Preferred Skills :

- Apache Iceberg

- Databricks Jobs & Pipelines

- Lakeflow Connect

- Unity Catalog

- Databricks Classic & Serverless Compute

- Azure Data Engineering

- CI/CD

- AI-assisted SDLC / AI for Data Engineering

Candidate Profile :

- 6+ years of overall experience in Data Engineering.

- Strong hands-on experience with Spark and Scala.

- Good experience in Azure Databricks ecosystem.

- Strong understanding of data pipelines, orchestration and distributed data processing.

- Experience with containerization and DevOps tools.

- Ability to work from Whitefield office, Bangalore, 3 days per week.

- Candidates should be available for the Saturday interview.