Job Description :

We are seeking an innovative, forward-thinking Data Engineer with a strong footprint in Azure Databricks and Artificial Intelligence (AI) solutions. This position is a full-time, permanent remote role based out of Pune, India, within our IT Services & Consulting division.

In this role, you will bridge the gap between traditional data engineering and modern AI. You will be responsible for building high-performance data pipelines on Microsoft Azure while actively embedding machine learning (ML) and AI capabilitiessuch as predictive forecasting, anomaly detection, and automated optimization modelsinto our global clients' Manufacturing and Supply Chain ecosystems.

Role & Responsibilities :

1. Azure Databricks & Cloud Data Engineering :

- Architect, develop, and maintain automated end-to-end data pipelines using Azure Databricks (PySpark, Spark SQL, Delta Lake).

- Implement and mature Medallion Architecture data lakes (Bronze/Silver/Gold layers) to store and process huge streams of IoT, manufacturing, and logistics data.

- Integrate Databricks seamlessly with the broader Azure stack, including Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), and Azure Key Vault.

- Monitor, tune, and optimize Spark clusters to manage computing costs, minimize execution lag, and ensure enterprise-grade security.

2. AI Engineering & ML Ops Implementation :

- Collaborate heavily with Data Science teams to operationalize AI/ML models within production pipelines using Databricks Lakehouse AI (MLflow, Feature Store).

- Build pipelines optimized for AI workloads, including data preprocessing, feature engineering, and high-frequency model scoring.

- Deploy, monitor, and track the performance of predictive models (e.g., predictive maintenance for factory assets, automated demand forecasting, supply chain bottleneck prediction).

- Incorporate LLM-driven or Generative AI components where applicable for supply chain risk summarizing or automated operational reporting.

3. Supply Chain & Manufacturing Domain Integration :

- Design custom data models capable of handling complex manufacturing data concepts : Bill of Materials (BOM), OEE (Overall Equipment Effectiveness), factory floor IoT sensor telemetry, inventory turnover, and multi-tier logistics transit times.

- Ingest and harmonize data from legacy silos like ERP (SAP, Oracle), MES (Manufacturing Execution Systems), and WMS (Warehouse Management Systems) into a singular AI-ready source of truth.

Requirements & Qualifications :

Technical Requirements :

- Experience : 4+ years of dedicated data engineering experience, with a heavy focus on cloud architectures.

- Azure & Databricks : Exceptional hands-on expertise with Azure Databricks and advanced scripting in Python/PySpark and SQL.

- AI/ML Foundations : Practical experience with machine learning operationalization frameworks (MLflow) and familiarity with Python ML libraries (e.g., Scikit-Learn, TensorFlow, or XGBoost).

- Data Pipelines : Strong experience orchestrating complex enterprise workflows using Azure Data Factory (ADF).

- Engineering Practices : Proficient with CI/CD tools (Azure DevOps, GitHub Actions), Git version control, and infrastructure as code concepts.

Domain & Soft Skills :

- Domain Expertise : Clear understanding of manufacturing operations, logistics, or supply chain planning mechanics.

- Consulting Mindset : Strong communication skills to interact with clients, decipher ambiguous business problems, and present technical architectures clearly.