Description :

Azure Databricks Engineer

Notice Period : Immediate Or Serving NP (< 30 Days)

Primary Skills : As an Azure Data bricks Data Engineer (Mandatory)

- At least 1 real-time project Exp is a must

- DBT Core / DBT Cloud

- Spark SQL, Python Programming Skills

- Databricks certifications (Associate or Professional) Preferred

- ADF (Azure Data Factory) & Azure DevOps Exp.- Preferred

Job Description :

- 5+ yrs. in Azure Databricks technologies.

- Experience in Medallion Architecture is mandatory.

- Good understanding of ETL , BI and DWH technologies.

- Hands-on experience on Jason or XML files as sources.

- Experience in autoloader for incremental loading and explode options for flattening data

- Experience in loading Delta live tables

- Experience in PySpark or Spark using data frames.

- Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure.

- Work together with data scientists and analysts to understand the needs for data and create effective data workflows.

- Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage.

- Utilizing Azure Data Bricks to create and maintain ETL (Extract, Transform, Load) operations.

- Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data.

- Improve the scalability, efficiency, and cost-effectiveness of data pipelines.

- Monitoring and resolving data pipeline problems to guarantee consistency and availability of the data.

- Good communication skills Oral & Written