About the Role :

We are looking for an experienced Data Engineer with 6+ years of experience in designing, developing, and managing scalable data pipelines and cloud-based data solutions. The ideal candidate should have strong expertise in Python, Cloud SQL, PostgreSQL, and Cloud Pub/Sub, along with a solid understanding of data engineering principles and cloud technologies.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines for processing and transforming large volumes of data.

- Develop and manage cloud-based data solutions to support business and analytics requirements.

- Build data ingestion and processing workflows using Python.

- Work extensively with PostgreSQL and Cloud SQL for data storage, processing, and optimization.

- Implement event-driven data pipelines and messaging solutions using Cloud Pub/Sub.

- Perform data extraction, transformation, validation, and loading across various data sources.

- Optimize database queries, data pipelines, and processing workflows for performance and scalability.

- Ensure data quality, accuracy, consistency, and reliability across data platforms.

- Develop monitoring and error-handling mechanisms for data pipelines.

- Collaborate with Data Scientists, Analysts, Software Engineers, and other stakeholders to understand data requirements.

- Troubleshoot data pipeline, database, and production issues.

- Maintain technical documentation related to data pipelines, architecture, and processes.

- Follow best practices for data security, governance, scalability, and cloud infrastructure.

Required Skills :

- 6+ years of professional experience in Data Engineering.

- Strong programming experience with Python.

- Hands-on experience with Cloud SQL and PostgreSQL.

- Experience with Cloud Pub/Sub and event-driven data processing.

- Strong understanding of ETL/ELT processes and data pipeline development.

- Experience working with cloud-based data engineering environments.

- Good understanding of database design, SQL, data modeling, and query optimization.

- Experience with data integration and processing from multiple data sources.

- Strong troubleshooting and problem-solving skills.

- Good understanding of data quality and data governance practices.