Job Summary :

We are seeking a Senior IT Data Engineer (5+ years experience) with mandatory expertise across the Azure Technology stack for data engineering, ingestion, and consumption layers. The ideal candidate will implement data engineering solutions using core Azure services (ADLS, Data Factory, Databricks, Event Hub), providing technical leadership across the entire development lifecyclefrom design and cloud engineering (network/security) to data modeling, CI/CD, and production support.

This role requires strong proficiency in SQL, Python/Spark scripting, and experience with data platforms like Teradata and Snowflake, all while leading the development of coding standards and mentoring junior team members.

Key Responsibilities and Technical Deliverables :

Azure Data Engineering and Implementation :

- Implement data migration and data engineering solutions using a comprehensive set of Azure products and services including: Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), Azure Functions, Event Hub, Azure Stream Analytics (ASA), and Azure Databricks.

- Implement batch and streaming data pipelines using cloud technologies, ensuring high throughput, reliability, and optimal performance.

- Expertise with the Azure Technology stack for data management, data ingestion, capture, processing, curation and creating consumption layers is mandatory.

- Provide practical direction within the Azure Native cloud services to optimize architecture and resource utilization.

Full Development Lifecycle and DevOps Ownership :

- Perform multiple aspects involved in the development lifecycle including technical design, cloud engineering (Infrastructure, network, security, and administration), ingestion, preparation, data modeling, rigorous testing, performance tuning, deployments, consumption, BI integration, alerting, and continuous prod support.

- Be a part of a DevOps team that completely owns and supports their product, driving operational excellence and automation.

- Implement and manage CI/CD pipelines using tools like GitHub, Jenkins, Azure DevOps, and Terraform.

- Leads development of coding standards, best practices, and privacy and security guidelines for the data platform.

Data Platforms and Analytics :

- Apply 5+ years of data analytics experience using SQL and working with diverse data platforms, including Teradata, Cassandra, MongoDB, Oracle, SQL Server, Snowflake, ADLS, and Azure Data Explorer (ADX).

- Work closely with source data application teams and product owners to design, implement and support analytics solutions that provide actionable business insights.

- Utilize Scripting (Python, Spark, Unix, SQL) for data processing, transformation, and automation tasks.

Technical Leadership and Mentorship :

- Provide technical leadership and collaborate within a team environment as well as work independently to drive complex initiatives to completion.

- Mentors others on technical and domain skills (e.g., airline industry data) to create multi-functional teams.

Mandatory Skills & Qualifications :

- Experience : 5+ years of experience in software development (Agile, DevOps) and data analytics using SQL.

- Data Platforms : 5+ years working with data platforms like Teradata, SQL Server, Snowflake, or ADLS.

- Cloud & Data Lake : 5+ years of cloud development and data lake experience, primarily in Microsoft Azure (EventHub, Data Factory, Databricks, Blob Storage).

- Core Skills : Expertise with the Azure Technology stack for data management and consumption layers.

- Scripting : Proficiency in Python, Spark, and SQL.

Preferred Skills and Certifications :

- Experience : 3+ years full-stack development experience (preferably in Azure) and 3+ years of Teradata Vantage development and administration.

- Domain : Airline Industry Experience.

Certifications : Azure Development Track Certification, Spark Certification (Preferred).

BI/Governance Tools : Familiarity with BI Analytics tools (Cognos, Tableau, Power BI) and Data Governance platforms (Alation, Informatica, BigID).