Coffeebeans.io - Snowflake Data Engineer - ETL

Description :

Role : Snowflake Data Engineer

Role Overview :

The Snowflake Data Engineer is a seasoned technical role requiring 5+ years of experience, responsible for taking end-to-end ownership of complex data engineering engagements, spanning architecture, design, development, and delivery.

Based in Bangalore, the incumbent will be crucial in building modern, scalable data platforms (Data Lakes, Lakehouses, and Data Warehouses) primarily utilizing Snowflake, to support advanced analytics, Business Intelligence (BI), and AI/ML applications.

This role demands strong technical leadership, client communication skills, and the ability to mentor junior team members while remaining highly adaptable to new technologies.

Key Responsibilities and Data Platform Deliverables :

- Architectural Leadership : Lead the design and implementation of scalable, secure, and high-performance data architectures, ensuring solutions meet demanding enterprise-grade requirements.

- Snowflake Implementation : Drive hands-on development and optimization within the Snowflake environment, leveraging advanced features for performance and cost efficiency.

- Pipeline Development : Architect and optimize complex data pipelines using ETL/ELT patterns for both batch and real-time processing across varied and heterogeneous source systems.

- Data Modeling and Storage : Model and implement enterprise-grade data lakes, lakehouses, and warehouses to efficiently support diverse analytical workloads and consumption patterns.

- Orchestration and Automation : Utilize orchestration tools like Airflow or similar to build and manage robust, production-grade data workflows, ensuring reliability and observability.

- Collaboration and Strategy : Collaborate with client teams and internal stakeholders to define data strategies, tooling, and standards, while also leading technical implementations and contributing to business discussions.

- Technical Governance : Set up and enforce best practices around code quality, pipeline observability, automated testing, and CI/CD (Continuous Integration/Continuous Delivery) processes for data environments.

- Troubleshooting and Guidance : Troubleshoot complex data issues and provide architectural solutions across heterogeneous systems. Guide technical decision-making and mentor junior/mid-level engineers.

Mandatory Technical Skills and Expertise :

- Core Languages : Strong expertise in Python and advanced proficiency in SQL (essential for Snowflake development).

- Cloud Data Warehousing : Hands-on experience with Snowflake is mandatory, including performance tuning, security, and resource optimization.

- Data Engineering Fundamentals : Deep understanding of data engineering architecture, including ETL/ELT pipelines, data modeling (dimensional, normalized, and vault), streaming systems, and enterprise data integration patterns.

- Distributed Processing : Proven experience with distributed data processing frameworks (e.g., Apache Spark, Flink) for handling large volumes of data.

- Orchestration : Proficient with orchestration tools like Airflow or similar platforms for managing production data workflows.

- Cloud Native Services : Solid grounding in cloud-native data services (compute, storage, IAM, monitoring, networking) experience with AWS (Amazon Web Services) is highly preferred.

- Leadership and Stakeholder Management : Experience leading technical implementations and effectively collaborating with business stakeholders.

Good to Have Skills :

- Multi-Cloud Familiarity : Multi-cloud experience (AWS preferred; GCP or Azure familiarity is valued).

- Modern Tooling : Hands-on experience with Databricks, dbt (Data Build Tool), and collaborative data development environments.

- Data Governance : Awareness of data governance, security, lineage, and quality frameworks.

- Certifications : Certifications such as AWS Certified Data Analytics, Databricks Certified Data Engineer Professional, or equivalent.