Azure Data Engineer - Synapse Analytics/Apache Airflow

Job Title : Azure data engineer

Location : Bangalore

Experience : 6- 8 years

Notice Period : Immediate to 15 days

Job Description :

- Overall, 6 to 8 years of experience in IT Industry.

- Min 6 years of experience working on Data Engineering using Azure Databricks, Synapse, ADF/Airflow.

- At least 4 Project experience in Building and maintaining ETL / ELT pipelines for large data sets, complex data processing, transformations, business logics, cost monitoring & performance optimization, and feature engineering processes.

Must Have skills :

- Extensive experience with Azure Databricks (ADB), Delta Lake, Azure Data Lake Storage (ADLS), Azure Data factory (ADF), Azure SQL Database (SQL DB), SQL, ELT / ETL Pipeline Development in Spark based environment.

- Extensive Experience with SparkCore, PySpark, Python, SparkSQL, Scala, Azure Blob Storage.

- Experience with CI/CD, Version control tools like GitHub, Azure DevOps.

- Experience with Version control tools

- Experience with Azure cloud platform.

- Experience in managing cross-functional team, requirements gathering, day-to-day relationships with clients, and stakeholders supporting clients to achieve better outcomes

Good to have :

- Experience / Knowledge on Containerizations - Docker, Kubernetes

- Experience working in Agile Methodology

- Experience in Real-Time Data Processing using Apache Kafka/EventHub/IoT, Structured Streaming and Stream analytics.

Qualifications :

Qualifications - BE, MS, M.Tech or MCA

Additional Information :

Certifications : Azure Big Data, Databricks Certified Associate