- Collaborate with business and technology stakeholders to understand current and future data requirements and translate them into actionable data solutions.
- Design, build, and maintain reliable, efficient, and scalable data infrastructure for data collection, storage, transformation, and analysis.
- Plan, design, build, and maintain scalable data solutions including data pipelines, data models, and applications for efficient and reliable data workflow.
- Design, implement, and maintain existing and future data platforms such as data warehouses, data lakes, and data lakehouses for both structured and unstructured data.
- Design and develop analytical tools, algorithms, and programs to support data engineering activities including writing scripts and automating tasks.
- Ensure optimum performance of data systems and identify improvement opportunities through monitoring and optimization.
- Design and implement data architecture with specialized knowledge in various data transformation mechanisms.
- Leverage big data tools and programming frameworks that meet organizational requirements for data storage, processing, analytics, and data science while considering long-term strategy, tool selection, and cost-effectiveness.
- Deliver appropriate analytical data modeling techniques based on data usage needs for Data Science, Machine Learning, Visualization, and other business requirements.
- Develop scalable and maintainable data solutions that support organizational growth.
- Incorporate CI/CD pipelines to improve efficiency and quality of development and deployment processes, accelerate release cycles, and enhance overall product stability.
- Establish benchmarks and adopt proactive monitoring and reliability techniques to ensure data system performance and availability.
- Adopt and implement appropriate data governance processes to ensure compliance with data protection and data safeguard requirements at both regional and organizational levels.
- Own and execute initiatives and projects within timelines, mitigate risks, and ensure delivery of high-quality solutions on schedule.
- Participate in periodic retrospectives to identify opportunities for continuous improvement.
- Mentor junior team members and contribute to team excellence by proactively identifying and resolving roadblocks.
- Productionize components and models at scale, working effectively with large data volumes.
- Stay current with emerging technologies and introduce innovative solutions and best practices to the organization.
- Maintain strong communication with cross-functional teams and stakeholders through clear oral, written, and interpersonal communication.
Requirements :
- 7+ years of experience in Data Engineering.
- Strong hands-on experience with Google Cloud Platform (GCP).
- Expertise in BigQuery, Dataflow, Dataproc, and Data Fusion.
- Experience with Terraform and Tekton (CI/CD & Infrastructure as Code).
- Strong knowledge of Cloud SQL and PostgreSQL.
- Hands-on experience with Apache Airflow (including Airflow + PySpark).
- Proficiency in Python for data engineering.
- Experience building and consuming APIs.
- Experience with Google Cloud Pub/Sub and Apache Kafka.