Company is seeking an innovative and detail-oriented Data Engineering Lead to design, develop, and optimize enterprise data infrastructure across multiple business domains. The role involves building scalable data pipelines, integrating manufacturing and IoT data, and developing a structured medallion data lakehouse architecture on the Azure platform. The successful candidate will work closely with data scientists, analysts, AI engineers, and business stakeholders to ensure reliable, secure, and scalable data solutions that support advanced analytics and digital transformation initiatives.

Key Responsibilities :

1. Data Pipeline Ownership :

- Design, develop, and maintain scalable data pipelines integrating data from multiple enterprise systems.

- Enable efficient data flow to support real-time monitoring, predictive analytics, and business intelligence initiatives.

- Ensure pipeline performance, reliability, and scalability.

2. Collaboration with Data Teams :

- Partner with data analysts, data scientists, and AI engineers to provide clean, structured, and accessible datasets.

- Develop and maintain Silver-layer transformation pipelines and Gold-layer semantic models for business analytics.

- Support advanced analytics and machine learning use cases.

3. IoT and Manufacturing Data Integration :

- Integrate IoT-generated data with manufacturing systems to provide real-time operational insights.

- Drive IT-OT convergence initiatives to improve connectivity and visibility across manufacturing environments.

- Support Industry 4.0 initiatives through data-driven decision-making.

4. Data Infrastructure Management :

- Optimize data storage, processing, and retrieval systems for AI and analytics workloads.

- Design scalable architectures capable of handling high-volume transactional and enterprise data.

- Manage and enhance Azure-based data platforms and services.

5. ETL & Workflow Automation :

- Design, develop, and automate ETL (Extract, Transform, Load) processes.

- Ensure timely, accurate, and efficient integration of data from multiple sources.

- Improve operational efficiency through workflow automation.

6. Data Quality, Governance & Compliance :

- Establish frameworks to ensure data accuracy, consistency, integrity, and security.

- Implement data governance standards and best practices.

- Ensure compliance with ISO 9001, GDPR, and other applicable regulations.

Required Skills :

1. Data Engineering & Analytics :

- Strong proficiency in SQL, Python, and ETL development frameworks.

- Experience designing and managing enterprise-scale data pipelines.

- Expertise in data modeling and data warehousing concepts.

2. Azure Data Platform :

- Hands-on experience with : 1. Azure Databricks 2. Azure Data Factory (ADF) 3. Microsoft Fabric.

- Strong understanding of Azure-based data architecture and services.

3. IoT Data Integration :

- Knowledge of IoT communication protocols such as MQTT and OPC-UA.

- Experience processing and integrating IoT data for real-time analytics.

4. Database Technologies :

- Expertise in relational databases such as MySQL and PostgreSQL.

- Knowledge of NoSQL and non-relational databases.

5. Cloud Technologies :

- Strong experience with cloud platforms, preferably Microsoft Azure.

- Understanding of cloud deployment, automation, and infrastructure management.

6. Data Governance & Security :

- Knowledge of data governance frameworks, security standards, and regulatory compliance requirements.

- Experience implementing data quality controls and access management practices.

Nice-to-Have Skills :

Industry 4.0 Knowledge :

- Understanding of smart manufacturing concepts including : 1. Predictive Maintenance 2. Digital Twins 3. Connected Manufacturing.

DevOps & Modern Data Engineering :

- Familiarity with: 1. CI/CD Pipelines 2. DevOps Practices 3. Docker 4. Kubernetes 5. Workflow orchestration tools.

Qualifications :

Education :

- Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related field.

- Specialized coursework or certifications in Azure Data Engineering, Data Architecture, or Analytics preferred.

Experience :

- 5- 8 years of experience in Data Engineering with a strong focus on Azure technologies.

- Proven experience integrating data from :

. MES (Manufacturing Execution Systems) 2. ERP Systems 3. CRM Platforms 4. Manufacturing Applications.

- Experience developing and optimizing large-scale data pipelines and ETL processes.

Preferred Industry Experience :

- Manufacturing, Energy, Industrial Automation, and Industry 4.0 environments.

Technical Expertise :

- Strong experience with SQL, Python, Databricks, Apache Airflow, Azure Data Factory, Microsoft Fabric, MQTT, and OPC-UA.