Data Unveil - Data Engineer - ETL/Python

Description :

About us :

At Data Unveil, we believe in delivering the best for our clients (Pharma Companies).

We use the latest technology and tools to aggregate and analyze specialty healthcare data received from various data partners.

We provide clear and hassle-free business insights to enhance the clients vision and drive business success.

Key Responsibilities :

- Develop and optimize ETL workflows and data pipelines using Python and PySpark.

- Write and optimize SQL queries for Oracle and MySQL databases.

- Perform distributed data processing and large-scale data transformations with PySpark.

- Implement data ingestion, transformation, validation, and integration procedures.

- Work with AWS services like S3, Lambda, EC2, RDS, Glue, and others.

- Ensure data integrity, security, and performance optimization.

- Collaborate with data teams to meet project requirements.

- Debug and optimize Spark jobs and database queries.

- Use version control (Git) and participate in code reviews.

Required Skills :

- Strong Python and PySpark programming skills.

- Experience with Linux environment.

- Hands-on experience with AWS services (S3, Lambda, EC2, RDS, Glue).

- Strong SQL skills for Oracle and MySQL.

- Knowledge of ETL processes and data pipeline development.

- Familiarity with Git and CI/CD practices.

- Ability to optimize queries and troubleshoot data issues.

Education :

- Bachelors degree in Computer Science, IT, or related field.

- Masters degree is a plus