AWS/Azure Data Engineer

Description :

Company Overview :

Role Overview :

As an AWS/Azure Data Engineer at Pan Asia HR Solutions, you will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support our clients' data-driven initiatives. You will collaborate with data scientists, analysts, and business stakeholders to understand their data requirements and translate them into robust and efficient solutions. Your work will directly impact the ability of our clients to leverage data for informed decision-making and improved business outcomes.

Key Responsibilities :

- Develop and maintain data models and schemas to ensure data quality and consistency for downstream analytics and reporting.

- Build and optimize data warehouses and data lakes to support efficient data retrieval and analysis for business intelligence.

- Automate data integration processes to streamline data flow and reduce manual effort for data users.

- Monitor and troubleshoot data pipeline performance to ensure data availability and accuracy for critical business operations.

- Collaborate with data scientists and analysts to provide data access and support their analytical needs for data-driven insights.

- Implement data governance and security policies to protect sensitive data and ensure compliance with regulatory requirements for client data.

Required Skillset :

- Demonstrated ability to design and implement data pipelines using AWS services such as S3, EC2,

Lambda, Glue, Redshift, and Azure services such as Azure Data Lake Storage, Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.

- Proven expertise in programming languages such as Python and Spark for data processing and transformation.

- Strong understanding of data modeling, data warehousing, and database concepts.

- Excellent problem-solving and analytical skills with the ability to troubleshoot complex data issues.

- Effective communication and collaboration skills to work with cross-functional teams.

- Bachelor's or Master's degree in Computer Science, Data Science, or a related field.