Sonata Software - Senior Data Engineer - Snowflake DB

Job Description :

We are seeking a highly skilled Senior Data Engineer with deep experience in Snowflake and modern cloud-based data platforms. The ideal candidate will design, build, optimize, and maintain scalable data pipelines and data warehouse solutions while collaborating closely with analytics, platform, and application teams.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines using Snowflake

- Architect and optimize data warehouse and data lake solutions

- Lead and support data migration initiatives from on-prem or legacy systems to cloud platforms

- Build and manage ETL/ELT pipelines using Python and cloud-native services

- Implement event-driven data processing using AWS services

- Ensure data quality, reliability, performance tuning, and cost optimization

- Manage and support Kafka clusters, including topic design, partitioning, replication, and monitoring

- Work with MongoDB and other NoSQL databases for data ingestion and integration

- Deploy and manage data workloads in Kubernetes environments

- Collaborate with DevOps and Security teams to implement DevSecOps best practices

- Implement CI/CD pipelines for data engineering workloads

- Monitor, troubleshoot, and resolve production issues

- Mentor junior engineers and contribute to technical design discussions

Required Technical Skills :

Data & Warehousing :

- Strong expertise in Snowflake (architecture, performance tuning, clustering, micro-partitions, streams & tasks)

- Solid understanding of data warehousing concepts (star/snowflake schemas, slowly changing dimensions, fact/dimension tables)

- Experience with data migration and modernization projects

Cloud & Programming :

- Strong experience with AWS, including :

a. S3

b. Lambda

- Event-driven services (EventBridge, SNS/SQS, or similar)

- Proficient in Python for data processing, automation, and pipeline development

Streaming & Messaging :

- Hands-on experience managing Apache Kafka

- Kafka cluster administration, monitoring, scaling, and troubleshooting

- Experience with real-time or near real-time data pipelines