Role : Data Engineer - Azure (Associate Architect / Senior)

Experience Level : 4 to 7 Years

Work location : Mumbai, Bangalore & Trivandrum

Notice Period : 0-15 days

Role & Responsibilities :

- Support the planning and implementation of data flow design, provide sizing and configuration assistance.

- Assisting the Solution Architects on programs for transformations and modernizations of data solutions using Azure cloud data technologies.

- Write production quality code to deploy ML workflows in a distributed environment.

- Design and Build Azure Data Pipelines and Data Streams.

- Design and build a hybrid data solution using a combination of on-premise and in-cloud services.

- Diagnose and troubleshoot complex distributed systems problems and develop solutions with a significant impact on our massive scale.

- Build tools to ingest and process terabyte-scale data on a daily basis.

- Communicate with a wide set of teams, including product owners, business analysts, data and enterprise architects, platform engineers, and cloud vendors, etc.

- Compilation of detailed design documents, technical specification documents, and test plans as required.

- Independently manage development tasks as per technical specification documents, prod deployment, post-prod support, etc.

- Managing support to Unit Testing, SIT & UAT.

- Coordinate with business and support teams on testing and migration activities.

Skills expectation :

- A strong hands-on knowledge of Spark/Pyspark.

- Expertise in writing SQL queries and performance optimization.

- Strong hands-on experience with a programming language like Python (preferred), java, or scala.

- Prior experience with working on cloud platforms like Azure (preferred), AWS or GCP.

- Prior experience of ADF is a plus.

- DataBricks experience is good to have.

- Data Warehouse: Snowflake or Synapse.

- Design Dimensional Models for Data Warehouse and Data Marts.

- Knowledge of version control systems and build automation tools like Azure DevOps and/or MLOps.

- Experience with operating system internals, file systems, disk/storage technologies, and storage Protocol.