About the Role:

We are looking for an experienced Data Engineer with strong Azure Data Engineering and AI capabilities to design, develop, and deploy scalable, production-ready data and AI solutions. The ideal candidate will have strong hands-on expertise in Azure Databricks, Python, and Azure Data Factory, along with practical experience in Azure AI Foundry, LLMs, Prompt Engineering, and AI Agent development.

Key Responsibilities:

- Design, develop, and maintain scalable data pipelines and data processing solutions using Azure Databricks, Python, and Azure Data Factory (ADF).

- Build and deploy AI applications and AI agents using Azure AI Foundry.

- Work with LLMs and Prompt Engineering to develop and integrate AI-powered solutions into data and business workflows.

- Implement scalable data architectures using Medallion Architecture and ensure reliable data ingestion, transformation, and processing.

- Develop and manage data solutions using Azure SQL Database, Databricks Apps, and related Azure services.

- Implement CI/CD pipelines and GitHub-based development and deployment practices.

- Ensure data and AI solutions meet requirements for scalability, security, governance, performance, and production readiness.

- Collaborate with Data Engineering, AI/ML, Architecture, and other technology teams to deliver enterprise data and AI solutions.

- Troubleshoot data pipelines, applications, and production issues and contribute to continuous improvement.

Required Skills & Experience:

- 5 - 9 years of overall experience in Data Engineering, Cloud Data Engineering, or related roles.

- 5+ years of hands-on experience with Azure Data Factory (ADF), Python, and Azure Databricks.

- Strong hands-on experience with Azure Databricks for data engineering and pipeline development.

- Strong programming and data-processing skills in Python.

- Strong experience with Azure Data Factory and enterprise data pipeline development.

- Hands-on experience with Azure AI Foundry and AI application development.

- Practical experience with LLMs, Prompt Engineering, and AI Agents.

- Experience with Azure SQL Database and Databricks Apps.

- Strong understanding of Medallion Architecture and modern data engineering practices.

- Experience with CI/CD and GitHub.

- Good understanding of data security, governance, scalability, and production deployment.

- Strong analytical, troubleshooting, communication, and problem-solving skills.

Good to Have:

- Experience with Databricks Mosaic AI.

- Knowledge of newer Databricks AI capabilities and AI/ML services.

- Experience integrating AI capabilities with enterprise data platforms.