Data Engineer - ETL/Python

Job Description :

We are looking for an experienced and highly motivated Data Engineer to join our data and analytics team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and cloud-based data solutions. The role requires strong expertise in big data technologies, cloud platforms, and distributed data processing frameworks to support enterprise data initiatives and analytics requirements.

Key Responsibilities :

- Design, build, and maintain scalable and efficient data pipelines for batch and real-time data processing.

- Develop and optimize ETL/ELT workflows using Python, PySpark, and SQL.

- Work extensively with cloud platforms such as :

i. Microsoft Azure

ii. Amazon Web Services (AWS)

iii. Google Cloud Platform (GCP)

- Develop and manage data solutions using Databricks for large-scale data processing and analytics.

- Implement robust data engineering solutions to support reporting, analytics, machine learning, and business intelligence initiatives.

- Collaborate with data scientists, analysts, architects, and business stakeholders to understand data requirements and deliver scalable solutions.

- Perform data ingestion, transformation, cleansing, and validation across multiple data sources.

- Optimize data workflows and query performance for large datasets.

- Ensure data quality, security, governance, and compliance standards are maintained.

- Monitor and troubleshoot data pipeline issues and provide timely resolutions.

- Participate in architecture discussions and recommend best practices for cloud-based data engineering solutions.

- Develop reusable frameworks, automation scripts, and documentation for data processes.

Required Skills & Qualifications :

- 6-15 years of experience in Data Engineering or related fields.

- Strong hands-on experience with :

i. Python

ii. PySpark

iii. SQL

iv. Databricks

- Experience working with at least one major cloud platform :

i. Azure

ii. AWS

iii. GCP

- Strong understanding of distributed data processing and big data technologies.

- Experience building scalable ETL/ELT pipelines and data integration solutions.

- Proficiency in data modeling, query optimization, and performance tuning.

- Familiarity with data warehousing concepts and large-scale data processing.

- Strong analytical, debugging, and problem-solving skills.

- Ability to work in a collaborative and fast-paced environment.

- Good communication and stakeholder management skills.