Posted on: 10/07/2026



Key Responsibilities:
- Design, develop, and maintain scalable data platforms on Microsoft Azure.
- Build and optimize ETL/ELT pipelines using Azure Databricks, Azure Data Factory, and Azure Data Lake Storage (ADLS Gen2).
- Develop robust data processing workflows using PySpark, Python, and SQL.
- Implement metadata-driven data engineering frameworks and reusable components.
- Design and maintain Delta Lake architectures using Unity Catalog, Delta Live Tables, Databricks Workflows, and SQL Warehouse.
- Develop and support enterprise data models using both Dimensional Modeling and 3NF methodologies.
- Collaborate with data scientists, analysts, and business stakeholders to deliver data-driven solutions.
- Build and support LLM/Generative AI-powered applications and integrate AI capabilities into enterprise data platforms.
- Leverage AI-assisted development tools such as GitHub Copilot, Claude Code, Gemini Code, or Codex to improve engineering productivity.
- Participate in code reviews, performance tuning, testing, debugging, and deployment activities.
- Implement CI/CD pipelines using Azure DevOps, Git, or Jenkins.
- Ensure data quality, governance, security, and compliance across enterprise data platforms.
Required Skills:
- Strong hands-on experience with:
1. Azure Databricks
2. Azure Data Factory (ADF)
3. Azure Data Lake Storage Gen2 (ADLS Gen2)
4. Azure SQL Database
5. Microsoft Fabric
6. Azure Event Hub
7. Azure Stream Analytics
8. Azure Cosmos DB
9. Azure Purview
10. Azure Log Analytics
11. Azure Data Explorer
- Deep expertise in the Databricks ecosystem:
1. PySpark
2. Databricks Notebooks
3. Unity Catalog
4. Delta Live Tables
5. Databricks Workflows
6. SQL Warehouse
7. Mosaic AI
8. AI/BI Genie
- Strong programming skills in Python and SQL.
- Experience designing metadata-driven data engineering frameworks.
- Strong understanding of Dimensional Data Modeling and 3NF Data Modeling.
- Hands-on experience with Git, Azure DevOps, or Jenkins for CI/CD.
- Excellent analytical, debugging, and performance optimization skills.
Preferred Skills:
- Experience developing Generative AI or LLM-based applications.
- Familiarity with AI-assisted software engineering using:
1. GitHub Copilot
2. Claude Code
3. Gemini Code
4. Codex
- Knowledge of prompt engineering, AI code review, testing, debugging, and responsible AI-assisted development.
- Experience with Neo4j, Elasticsearch, or Vector Databases.
- Exposure to Cloudera (CDH) or Hortonworks (HDP).
- Knowledge of Azure infrastructure, networking, security, and governance.
Educational Qualification:
- Bachelor's degree (B.E./B.Tech) in Computer Science, Information Technology, or a related discipline from a recognized institution.
Did you find something suspicious?
Posted by
Gopalram.d
Last Active: NA as recruiter has posted this job through third party tool.
Posted in
Data Engineering
Functional Area
Data Engineering
Job Code
1652901