Azure Data Engineer - Python/ETL

Description :

Role : Azure Data Engineer

We are seeking a highly skilled and experienced Azure Data Engineer to join our growing data engineering team.

The ideal candidate will have 5+ years of experience designing, building, and managing scalable data platforms in Microsoft Azure, with a strong focus on data integration, transformation, and pipeline development.

You will work closely with data architects, analysts, and stakeholders to deliver robust data solutions that support strategic business initiatives.

Location : Hyderabad (Onsite)

Key Responsibilities :

- Design and develop end-to-end data pipelines in Azure using services such as Azure Data Factory (ADF), Azure Synapse Analytics, Azure Databricks, and Azure Data Lake Storage (ADLS).

- Build and maintain ETL/ELT processes for structured and unstructured data from diverse sources.

- Implement data modeling, data warehousing, and performance tuning techniques using tools like Synapse, SQL Server, and Power BI.

- Ensure data quality, governance, and security compliance through the implementation of appropriate policies and frameworks.

- Collaborate with data scientists, analysts, and business units to understand data needs and deliver solutions.

- Monitor and troubleshoot data pipelines and perform root cause analysis for any issues in data workflows.

- Optimize costs and performance of Azure data services.

- Maintain detailed documentation of system design, processes, and procedures.

- Mentor junior engineers and participate in code reviews and best practice discussions.

Required Skills & Qualifications :

- 5+ years of hands-on experience in data engineering, with a significant focus on Azure cloud platform.

- Exp on hadoop ,hive

- Proficiency in Azure Data Factory, Azure Synapse, Azure Databricks, Data Lake Gen2, and Azure Functions.

- Strong SQL skills and experience with T-SQL, stored procedures, and query optimization.

- Familiarity with Spark, Python, and PySpark for data processing.

- Experience with CI/CD pipelines using tools like Azure DevOps or GitHub Actions.

- Strong understanding of data warehousing concepts, star/snowflake schemas, and data lakes.

- Knowledge of data governance, security, and compliance best practices (e.g., GDPR, HIPAA).

- Excellent problem-solving skills and attention to detail.