Data Architect - ETL Tools

Description :

- Design and Develop Data Solutions.

- Build scalable data pipelines and workflows using Databricks and Azure services.

- Leverage low-code/no-code tools (e.g., Power Apps, Azure Data Factory, Logic Apps) for rapid solution development.

Data Integration :

- Integrate data from multiple sources (structured, semi-structured, unstructured) into Databricks.

- Performance Optimization.

- Optimize Spark jobs and workflows for cost and performance efficiency.

Collaboration :

- Work closely with data analysts, business users, and citizen developers to enable self-service analytics.

Automation :

- Implement automation for data ingestion, transformation, and deployment using low-code/no-code frameworks.

- Governance & Security.

- Ensure compliance with data governance, security, and privacy standards.

Required Skills & Qualifications :

- Technical Expertise.

- Hands-on experience with Databricks, Apache Spark, and Azure ecosystem (Data Lake, Synapse, ADF).

- Proficiency in Python, SQL, and familiarity with Delta Lake.

- Low-Code/No-Code Experience.

- Experience with platforms like Power Apps, Power Automate, Azure Logic Apps, or similar.

- Ability to design workflows and integrate APIs using low-code tools.

- Data Engineering Fundamentals.

- Strong understanding of ETL/ELT processes, data modeling, and performance tuning.

- Excellent communication and collaboration skills.

- Ability to work in agile environments and manage multiple priorities.

Preferred Qualifications :

- Knowledge of MLflow and machine learning workflows.

- Familiarity with data governance tools like Azure Purview.