Position : Senior Data Engineer with Python and Snowflake

Experience : 9+ Years

Location : Hinjewadi, Pune

Shift Timings : 12 : 30 PM - 9 : 30 PM (IST)

Work Model : Hybrid (3 days WFO : Tuesday, Wednesday, Thursday)

Job Summary :

We are seeking a highly skilled and experienced Senior Data Engineer with 9+ years of experience. The ideal candidate will have extensive hands-on expertise in Python and Snowflake, with a proven track record of developing data engineering and data science projects on AWS. You must have a deep understanding of implementing object-oriented programming in Python and hands-on experience in building Snowflake applications using its AI/Cortex platform. This role requires a professional who can handle the entire data lifecycle, from building scalable pipelines and managing data governance to leveraging cutting-edge AI features to create innovative solutions.

Key Responsibilities :

Data Engineering & Development :

- Design and develop data engineering and data science projects on Snowflake/AI Cloud platform on AWS.

- Write and maintain clean, scalable, and object-oriented code in Python, utilizing data structures like Pandas and data frames.

- Build and deploy Python applications like Lambda on AWS, and deploy web applications using Docker containers or Kubernetes with CI/CD pipelines.

- Develop applications using Snowpipe and Snowpark to move and handle unstructured data from cloud sources like AWS S3 and data lakes.

Snowflake & AI/Cortex Development :

- Build applications using the Snowflake AI/Cortex platform, specifically with Cortex Agents, Cortex Search, and Cortex LLMs.

- Implement context enrichment using prompts or Retrieval-Augmented-Generation (RAG) methods.

- Design and implement applications for relevance search and custom interactions with LLMs.

Data Governance & Architecture :

- Implement Snowflake's Account hierarchy model and define robust account-role-permission strategies.

- Handle data sharing using Internal Data Marketplace and Data Exchanges.

- Apply data governance and security concepts, including dynamic row/column-level data masking using Snowflake Tags.

Problem-Solving & Collaboration :

- Continuously improve functionality through experimentation, performance tuning, and incorporating customer feedback.

- Use Agile and SCRUM methodologies with Jira for project management.

Required Skills & Qualifications :

Core Experience :

- 9+ years of experience in data engineering and data science.

- 7+ years of hands-on Python development experience.

Mandatory Technical Skills :

- Proven experience with Snowflake and AWS cloud platform.

- 1+ years of experience with Snowflake AI/Cortex platform.

- Deep understanding of Object-Oriented Programming (OOP) in Python, data structures, and multi-threading concepts.

- Experience with version control using Git or Bitbucket.

- Proficiency in Snowpipe, Snowpark, and developing applications for unstructured data.

- Strong knowledge of Snowflake's data governance and security features.

- Experience with containerization using Docker or Kubernetes.

Professional Attributes :

- Excellent analytical and problem-solving skills.

- The ability to communicate complex technical concepts clearly.

Preferred Skills :

- Experience with different data modeling techniques.

- Experience in building Snowflake native applications using Streamlit.

- Knowledge of application cache implementation within Python web applications.

- Experience with duckdb and Apache Arrow.

- Experience implementing CI/CD pipelines for Snowflake applications.