We are seeking a talented and motivated Data Engineer with 3+ years of experience to join our growing data team. In this role, you will be instrumental in building, maintaining, and optimizing our data ingestion pipelines, ensuring high data quality and reliability. You'll work with a variety of data sources and technologies, with a strong emphasis on Google Cloud Platform (GCP) services, particularly BigQuery, contributing to the foundation of our data-driven initiatives.

What You'll Do :

- Design & Develop Data Pipelines : Build robust, scalable, and efficient data ingestion pipelines from various sources (e.g., databases, APIs, streaming data, files) into our data lake/warehouse, with a focus on ingesting data into BigQuery.

- Ensure Data Quality : Implement and maintain data quality checks, validation rules, and monitoring mechanisms to ensure accuracy, completeness, and consistency of data, particularly within BigQuery tables. Identify and resolve data anomalies and inconsistencies proactively.

- Data Modeling : Collaborate with data analysts and data scientists to understand data requirements and design optimal data models for analytics and reporting within BigQuery (e.g., partitioned and clustered tables, views, external tables).

- Performance Optimization : Optimize existing data pipelines and BigQuery queries for performance and cost-efficiency, leveraging BigQuery features like partitioning, clustering, and query optimization techniques.

- Automation : Automate data extraction, transformation, and loading (ETL/ELT) processes, potentially utilizing GCP services like Cloud Functions, Cloud Dataflow, or Cloud Composer (Apache Airflow) for orchestration.

- Documentation : Create and maintain comprehensive documentation for data pipelines, data models, and data quality standards.

- Troubleshooting & Support : Provide support for data-related issues, debug pipeline failures, and ensure timely resolution, including troubleshooting BigQuery job failures and performance issues.

- Collaboration : Work closely with cross-functional teams including product, engineering, and business intelligence to understand data needs and deliver effective data solutions.

- Stay Current : Research and evaluate new data technologies and tools to improve our data infrastructure and processes, especially within the GCP ecosystem.

What You'll Bring :

- Bachelor's degree in Computer Science, Engineering, Information Technology, or a related quantitative field.

- 3+ years of professional experience in data engineering or a similar role, with a strong focus on building and maintaining data pipelines.

- Proficiency in at least one programming language commonly used in data engineering (e.g., Python, Java, Scala). Python is highly preferred.

- Strong experience with SQL and working with relational databases (e.g., PostgreSQL, MySQL, SQL Server, Oracle).

- Hands-on experience with ETL/ELT tools and concepts.

- Proven experience with Google Cloud Platform (GCP) data services, specifically BigQuery.

- Competency in loading data into BigQuery using various methods (e.g., batch loading from Cloud Storage, streaming inserts).

- Strong SQL querying skills within BigQuery, including understanding of BigQuery's unique SQL dialect and functions.

- Familiarity with BigQuery table optimizations like partitioning and clustering.

- Experience with BigQuery's data quality features or applying data quality best practices to BigQuery data.

- Familiarity with data warehousing concepts and experience with data lake architectures.

- Understanding of data quality principles and experience implementing data validation techniques.

- Experience with version control systems (e.g., Git).

- Excellent problem-solving skills and attention to detail.

- Strong communication and interpersonal skills, with the ability to explain complex technical concepts to non-technical stakeholders.

Bonus Points If You Have :

- Experience with other GCP data services such as Cloud Dataflow, Cloud Composer (Apache Airflow), Cloud Storage, Pub/Sub, or Dataproc.

- Familiarity with BigQuery ML or other machine learning concepts.

- Experience with data orchestration tools (e.g., Apache Airflow, Prefect, Dagster).

- Knowledge of data governance and data security best practices within GCP