Persistent Systems - Spark/Scala Developer - ETL/Azure Databricks

About Position :

We are looking for Spark Scala developer with hands on experience on ETL, Azure Databricks.

Role : Spark Scala Developer

Location : All Persistent Locations

Experience : 8+ Years

Job Type : Full Time Employment

What You'll Do :

- Design and Development : Design and develop scalable data processing applications using Apache Spark and Scala.

- Data Processing : Develop Spark jobs to process large datasets, including data ingestion, transformation, and aggregation.

- Data Pipeline Development : Build and maintain data pipelines using Spark, Scala, and other related technologies.

- Data Quality : Ensure data quality and integrity by implementing data validation, data cleansing, and data normalization techniques.

- Performance Optimization : Optimize Spark jobs for performance, scalability, and reliability.

- Troubleshooting : Troubleshoot and resolve technical issues related to Spark and Scala applications.

Expertise You'll Bring :

- Minimum 8 years experience of Data Engineering

- Experience in ETL/Pipeline Development using tools such as Azure Databricks/Apache Spark and Azure DataFactory with development expertise on batch and real-time data integration

- Experience in programming using Scala or Python

- Experience in writing the Store Procedures

- Experience in data ingestion, preparation, integration and operationalization techniques in optimally addressing the data requirements

- Experience in Cloud data warehouse like Azure Synapse, Snowflake analytical warehouse

- Experience with Orchestration tools, Azure DevOps and GitHub

- Experience in building end to end architecture for Data Lakes, Data Warehouses and Data Marts

- Experience in relational data processing technology like MS SQL, Delta lake, Spark SQL

- Experience to own end-to-end development, including coding, testing, debugging and deployment

- Extensive knowledge of ETL and Data Warehousing concepts, strategies, methodologies.

- Ability to provide solutions that are forward-thinking in data and analytics

- Must be team oriented with strong collaboration, prioritization, and adaptability skills required

- Excellent written and verbal communication skills including presentation skills

- Familiarity with Azure services like Azure functions, Azure Data Lake Store, Azure Cosmos

- Familiarity with Healthcare business data models

Benefits :

- Competitive salary and benefits package

- Culture focused on talent development with quarterly promotion cycles and company-sponsored higher education and certifications

- Opportunity to work with cutting-edge technologies

- Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards

- Annual health check-ups

- Insurance coverage : group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents